• Patent Issued: Y. Sun, Y. Huang, Learning and Generalizing Movement Policies for Robotic Manipulation, US Patent #10,611,026, issued on 04/07/2020.
  • Congratulations! David Paulius successfully defends his PhD dissertation on "Functional Object-Oriented Network: A Knowledge Representation for Service Robotics" on March 06, 2020 [ Flyer ] [ Photo ]
  • Patent Issued: Y. Sun, T. Williams, Learning State-Dependent Sensor Measurement Models for Localization, US Patent # 10,572,802, Issued on 02/25/2020.
  • Congratulations! Troi Williams receives Microsoft Research Dissertation Grant 2019 [ More Details ]
  • Congratulations! Hailey Baez and Jean-Luc Hayes have been awarded Graduate Fellowships [ More Details ]
  • Congratulations! John Rippetoe successfully defends his PhD dissertation on "Force Feedback and Intelligent Workspace Selection for Legged Locomotion Over Uneven Terrain" on March 20, 2019 [ Flyer ] [ Photo ]
  • Congratulations! Yongqiang Huang successfully defends his PhD dissertation on "Generating Robotic Manipulation Motions by Learning from Human Demonstrations" on March 18, 2019 [ Flyer ] [ Photo ]
  • Drinking Game Just Become Easier with a Beer-Pouring Robot [ More Details ]
  • New grant: In collaboration with Stanford University, Dr. Sun was awarded a NSF grant in support of a project entitled "CHS: Small: Collaborative Research: Wearable Fingertip Haptic Devices for Virtual and Augmented Reality: Design, Control, and Predictive Tracking" [ More Details ]
  • Dr. Sun awarded USF Excellence in Innovation Award [ More Details ]
  • ENB 223, 4202 E Fowler Ave, Tampa 33620, FL, USA

  • yusun AT

  • 813-974-7506 (813-385-4612)

Our Projects


  • All
  • Artificial Intelligence
  • Robotics
  • Vision
  • VR
  • Deep Learning



Paper: Long Activity Video Understanding

Video understanding is one of the most challenging topics in computer vision. In this project, a four-stage video understanding pipeline was presented to simultaneously recognize all atomic actions and the single ongoing activity in a video. The pipeline used objects and motions from the video and a graph-based knowledge representation network as prior reference.

Read more

Paper: Learning to Pour

Pouring is a simple task people perform daily. It is the second most frequently executed motion in cooking scenarios, after pick-and-place. We present a pouring trajectory generation approach, which uses force feedback from the cup to determine the future velocity of pouring.

Read more

Paper: An Approach for Automated Multimodal Analysis of Infants’ Pain

In the paper, we propose an automated multimodal approach that utilizes a combination of both behavioral and physiological pain indicators to assess infants’ pain. We also present a unimodal approach that depends on a single pain indicator for assessment.

Read more

Patent: Systems And Methods For Projecting Images Onto An Object (US patent #9,520,072)

In one embodiment, a method for projecting images on a subject includes determining a pose and position of the subject, adjusting a three-dimensional model of an anatomical structure of the subject to match the determined pose and position, and projecting an image of the anatomical structure onto the subject

Read more

Paper: Recent Datasets on Object Manipulation: A Survey

In the paper, we take a significant step forward by reviewing datasets that were published in the last 10 years and that are directly related to object manipulation and grasping. We report on modalities, activities, and annotations for each individual dataset and we discuss our view on its use for object manipulation.

Read more

Paper: Robotic Grasping for Instrument Manipulations

The paper introduces two grasp quality measures that are derived from the two manipulation requirements: interactive wrench requirements and motion requirements for accomplishing a manipulation task

Read more

Paper: Functional Object-Oriented Network for Manipulation Learning

This paper presents a novel structured knowledge representation called functional object-oriented network (FOON) to model the connectivity of the functional-related objects and their motions in manipulation tasks. The graphical model FOON is learned by observing object state change and human manipulations with the objects.

Read more

Patent: Systems and Methods for Planning a Robot Grasp Based upon a Demonstration Grasp (US patent #9,321,176)

In one embodiment, planning a robot grasp of an object includes determining a grasp type that would be used by a human being to grasp the object, determining a position and orientation of the human being's thumb relative to the object, and planning the robot grasp based upon the determined grasp type and thumb position and orientation.

Read more






Book & Book Chapters

Our Team

1 Professor, 9 PhD students, 1 MS student, 1 REU

Yu Sun

Associate Professor

Lab Director

David Paulius

Ph.D. Student

B.S.: University of the Virgin Islands

Troi Williams

Ph.D. Student

B.S.: University of the Virgin Islands

M.S.: Norfolk State University

Ahmad Babaeian Jelodar

Ph.D. Student

B.S.: Amirkabir University of Technology - Tehran Polytechnic

M.S.: Amirkabir University of Technology - Tehran Polytechnic

Tianze Chen

Ph.D. Student

B.S.: Zhejiang University

M.S.: University of Florida

Md Sirajus Salekin

Ph.D. Student

B.S.: Islamic University of Technology

M.S.: Islamic University of Technology

Juan Ricardo Wilches Cortina

Ph.D. Student

B.S.: Universidad del Norte

M.S.: Universidad del Norte

Hailey Baez

Ph.D. Student

B.S.: Universidad del Turabo

M.S.: University of South Florida

Jean-Luc Kristoff Hayes

Ph.D. Student

B.S.: University of South Florida

no picture

Anzhelika Kolinko

Ph.D. Student

B.S.: Ukraine

no picture

Maxat Alibayev

M.S. Student

B.S.: University of South Florida

Adheesh Shenoy


B.S.: University of South Florida

Lab Alumni

Our sponsors