Learning synergies based in-hand manipulation with reward shaping

In-hand manipulation is a fundamental ability for multi-fingered robotic hands that interact with their environments. Owing to the high dimensionality of robotic hands and intermittent contact dynamics, effectively programming a robotic hand for in-hand manipulations is still a challenging problem. To address this challenge, this work employs deep reinforcement learning (DRL) algorithm to learn in-hand manipulations for multi-fingered robotic hands. A reward-shaping method is proposed to assist the learning of in-hand manipulation. The synergy of robotic hand postures is analysed to build a low-dimensional hand posture space. Two additional rewards are designed based on both the analysis of hand synergies and its learning history. The two additional rewards cooperating with an extrinsic reward are used to assist the in-hand manipulation learning. Three value functions are trained jointly with respect to their reward functions. Then they cooperate to optimise a control policy for in-hand manipulation. The reward shaping not only improves the exploration efficiency of the DRL algorithm but also provides a way to incorporate domain knowledge. The performance of the proposed learning method is evaluated with object rotation tasks. Experimental results demonstrated that the proposed learning method enables multi-fingered robotic hands to learn in-hand manipulation effectively.