Using an Ensemble of Deep Neural Networks to Detect Human Keypoints in the Workspace of a Collaborative Robotic System

It is suggested that the use an ensemble of deep neural networks can determine the spatial position of the operator using keypoints with a multicamera sensor system. The advantage of the algorithm is the use of a multicamera system that allows keypoints to be linked to the local coordinate system of an industrial robotic complex. The testing of this work was made on the basis of modern embedded computing hardware and software. The effectiveness of the proposed approach is demonstrated even when only a subset of key points is found in the frame, as well as when they partially overlap. A software module in Python has been developed for detecting and localizing key points of the operator and industrial manipulator. The proposed approach will make it possible to plan the robot’s trajectories for the safe execution of joint operations in one workspace. The developed algorithm will be used to predict the operator’s actions in the workspace and detect abnormal situations and possible intersections in the trajectories of the collaborative robot.