Our Projects


  • All
  • Artificial Intelligence
  • Robotics
  • Vision
  • VR
  • Deep Learning
Manipulation Motion Generating (NSF)
Automated Pain Assessment in Neonates (USF)
Robotics Modeling of Skilled Hand Tasks (USF)
Virtually Transparent Epidermal Imagery (NSF)
Telemedicine Robot (Moffit Cancer Center)



Paper: Learning to Pour

Pouring is a simple task people perform daily. It is the second most frequently executed motion in cooking scenarios, after pick-and-place. We present a pouring trajectory generation approach, which uses force feedback from the cup to determine the future velocity of pouring. The approach uses recurrent neural networks as its building blocks. We collected the pouring demonstrations which we used for training. To test our approach in simulation, we also created and trained a force estimation system. The simulated experiments show that the system is able to generalize to single unseen element of the pouring characteristics.

Read more

Paper: An Approach for Automated Multimodal Analysis of Infants’ Pain

In the paper, we propose an automated multimodal approach that utilizes a combination of both behavioral and physiological pain indicators to assess infants’ pain. We also present a unimodal approach that depends on a single pain indicator for assessment.

Read more

Patent: Systems And Methods For Projecting Images Onto An Object (US patent #9,520,072)

In one embodiment, a method for projecting images on a subject includes determining a pose and position of the subject, adjusting a three-dimensional model of an anatomical structure of the subject to match the determined pose and position, and projecting an image of the anatomical structure onto the subject in registration with the actual anatomical structure of the subject to illustrate the location of the structure on or within the subject.

Read more

Paper: Recent Datasets on Object Manipulation: A Survey

In the paper, we take a significant step forward by reviewing datasets that were published in the last 10 years and that are directly related to object manipulation and grasping. We report on modalities, activities, and annotations for each individual dataset and we discuss our view on its use for object manipulation. We also compare the datasets and summarize them. Finally, we conclude the survey by providing suggestions and discussing the best practices for the creation of new datasets.

Read more

Paper: Robotic Grasping for Instrument Manipulations

The paper introduces two grasp quality measures that are derived from the two manipulation requirements: interactive wrench requirements and motion requirements for accomplishing a manipulation task

Read more

Paper: Functional Object-Oriented Network for Manipulation Learning

This paper presents a novel structured knowledge representation called functional object-oriented network (FOON) to model the connectivity of the functional-related objects and their motions in manipulation tasks. The graphical model FOON is learned by observing object state change and human manipulations with the objects.

Read more

Patent: Systems and Methods for Planning a Robot Grasp Based upon a Demonstration Grasp (US patent #9,321,176)

In one embodiment, planning a robot grasp of an object includes determining a grasp type that would be used by a human being to grasp the object, determining a position and orientation of the human being's thumb relative to the object, and planning the robot grasp based upon the determined grasp type and thumb position and orientation.

Read more






Book & Book Chapters

Our Team

1 Professor, 7 Ph.D students, and 1 REUs

Yu Sun

Associate Professor

Lab Director

Yongqiang Huang

Ph.D. Student

B.S.: Beijing University of Post and Telecommunications

M.S.: North Carolina State University

John Rippetoe

Ph.D. Student

B.S.: University of South Florida

David Paulius

Ph.D. Student

B.S.: University of the Virgin Islands

Troi Williams

Ph.D. Student

B.S.: University of the Virgin Islands

M.S.: Norfolk State University

Ahmad Babaeian Jelodar

Ph.D. Student

B.S.: Amirkabir University of Technology - Tehran Polytechnic

M.S.: Amirkabir University of Technology - Tehran Polytechnic

Tianze Chen

Ph.D. Student

B.S.: Zhejiang University

M.S.: University of Florida

Md Sirajus Salekin

Ph.D. Student

B.S.: Islamic University of Technology

M.S.: Islamic University of Technology

Jason Melancon


B.S.: University of South Florida

RPAL Group

Lab Alumni

Our sponsors

Contact Info

  • 4202 E Fowler Ave., ENB 223

  • yusun AT usf.edu

  • 813 974 7506 (813 385 4612)