Robotic Grasping for Instrument Manipulations

This project focuses on the grasp requirements derived from the voluntary and involuntary physical interactions in instrument manipulations. The manipulation-oriented grasp requirements include interactive wrench requirements and motion requirements that are required to accomplish a manipulation task. The manipulation-oriented grasp requirements are directly associated with the functionality of the instrument and the manipulation task, but independent from the robotic hardware. Grasp quality measures developed from the manipulationoriented grasp requirements can be used as search criteria for optimal grasps.

A grasp of the instrument in the interactive task should be able to hold on to the instrument and provide the interactive wrench (force and torque) during the task. Therefore, how much the desired task wrench space is covered by the grasp wrench (capability of this grasp) should be used to evaluate how good the grasp is.

To perform a task, the instrument motion trajectory can either be computed by a motion planner or generated from a learned motion model. Grasping with a robotic hand give flexibility in “mounting” the instrument onto the robotic arm -- a different grasp will connect the instrument to the robotic arm with a different pose, then the inverse kinematics approach will result in a different joint motion to achieve the same functional motion. Therefore, the grasp and the functional motion decide the manipulator’s motion. With a desired functional tool motion, the grasp determines the manipulator’s motion. Since different manipulator motions will have different efficiency rates in transferring the motion from joints to the instrument, the efficiency of the manipulation motion should be used to evaluate the grasp.

The quality measures in both wrench coverage and manipulation efficiency are determined by contact points of the robotic hand on the tool, and contact points are further determined by the hand posture as well as the relative hand position and orientation. Therefore, a grasp G can be defined with an array of finger joint angles and hand position and orientation. When a robotic hand with high DOF is used, strategies learned from demonstrations are introduced to reduce the search space.

Instead of learning grasping points, we extract and use two more abstract strategies: grasp types and thumb placement. They confine the configuration space, but leave enough room for grasp planning to find the optimal stable grasp that is adapted to different robotic hands.

A grasp type abstracts the manner in which a human grips an object for a manipulation. It can either be input by a user or recognized in a demonstration. For robotic hands with less DOF, fewer grasp types can be defined. Taking the Barrett hand as an example, we defined only five grasp types, much fewer than the human hand: power grasp, power sphere, precision grasp, precision sphere, and lateral pinch. To map from the learned grasp types from human grasps, some grasp types can be grouped into one.

For a particular instrument manipulation task, a grasp type is learned from human demonstration using machine learning techniques and then mapped to the robotic hand’s grasp type.

There is general agreement in anthropology that thumb plays a key role in gripping an object efficiently. The crucial feature distinguishing human hand from other apes is the opposable thumb. The same situation applies to robots that almost all robotic hands have a long and strong opposable thumb. Mapping only thumb position from a human hand to a robot hand is simple because there is little correspondence problem and it can be easily generalized to different robotic hand models.

Both the grasp types and the thumb placement dramatically reduce the hand configuration space for searching an optimal grasp. For example, the constrains introduced by the thumb placement of a Barrett Hand reduces its configuration dimension from ten (six for the wrist and four for the fingers) to two or three depending on grasp types.

Two different grasps would requires two very different arm motion to pour water

Grasp quality measures of different grasps for different tasks




Lin, Y. and Sun, Y., (2015) Grasp Planning to Maximize Task Coverage, Intl. Journal of Robotics Research, vol. 34 no. 9 1195-1210.
Lin, Y., and Sun, Y. (2015) Robot Grasp Planning Based on Demonstrated Grasp Strategies, Intl. Journal of Robotics Research, 34(1): 26-42.
Sun, Y., Yun Lin, and Yongqiang Huang (2016) Robotic Grasping for Instrument Manipulations, URAI, 1-3 (invited)
Lin, Y. and Sun, Y. (2015) Task-Based Grasp Quality Measures for Grasp Synthesis, IROS, 485-490.
Lin, Y., Sun, Y. (2014) Grasp Planning Based on Grasp Strategy Extraction from Demonstration, IROS, pp. 4458-4463.
Lin, Y. and Sun, Y., 2016. Task-oriented grasp planning based on disturbance distribution. In Robotics Research (pp. 577-592). Springer International Publishing. (presented at ISRR)
Lin Y., Ren S., Clevenger M., and Sun Y. (2012) Learning Grasping Force from Demonstration, IEEE Intl. Conference on Robotics and Automation (ICRA), pp. 1526-1531.


Y. Sun, Y. Lin, Systems and Methods for Planning a Robot Grasp Based upon a Demonstration Grasp, US patent #9,321,176 Issued on April 26, 2016.

This work is supported by NSF. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation.