• Troi William has received a CIFellow and will start his Postdoc at the University of Maryland. His Postdoc mentor is Dr. Pratap Tokekar.
  • Congratulations! Ahmad Babaeian Jelodar successfully defends his PhD dissertation on "Knowledge Extraction and Inference Based on Visual Understanding of Cooking Contents" on June 30, 2021. [ Flyer ]
  • Congratulations! Md Sirajus Salekin for being awarded USF RPAL Research Excellence Award on Jan, 2021.
  • Congratulations! Troi Williams receives Koerner Family Foundation (KFF) Research Award on Jan, 2021. [ More Details ]
  • Congratulations! Dr. David Paulius for being awarded USF RPAL Research Excellence Award on July, 2020.
  • Congratulations! Maxat Alibayev successfully defends his Master's thesis on "Action Recognition Using the Motion Taxonomy" on June 17, 2020 [ Flyer ]
  • Patent Issued: Y. Sun, Y. Huang, Learning and Generalizing Movement Policies for Robotic Manipulation, US Patent #10,611,026, issued on 04/07/2020.
  • Congratulations! David Paulius successfully defends his PhD dissertation on "Functional Object-Oriented Network: A Knowledge Representation for Service Robotics" on March 06, 2020 [ Flyer ] [ Photo ]
  • Patent Issued: Y. Sun, T. Williams, Learning State-Dependent Sensor Measurement Models for Localization, US Patent # 10,572,802, Issued on 02/25/2020.
  • ENB 223, 4202 E Fowler Ave, Tampa 33620, FL, USA

  • yusun AT

  • 813-974-7506 (813-385-4612)

Our Projects


  • All
  • Artificial Intelligence
  • Robotics
  • Vision
  • VR
  • Deep Learning

Our Contributions to Science and Technology

Full List

Paper: A Motion Taxonomy for Manipulation Embedding

To represent motions from a mechanical point of view, this paper explores motion embedding using the motion taxonomy. With this taxonomy, manipulations can be described and represented as binary strings called motion codes. Motion codes capture mechanical properties, such as contact type and trajectory, that should be used to define suitable distance metrics between motions or loss functions for deep learning and reinforcement learning.

Read more

Paper: Long Activity Video Understanding

Video understanding is one of the most challenging topics in computer vision. In this project, a four-stage video understanding pipeline was presented to simultaneously recognize all atomic actions and the single ongoing activity in a video. The pipeline used objects and motions from the video and a graph-based knowledge representation network as prior reference.

Read more

Paper: Learning to Pour

Pouring is a simple task people perform daily. It is the second most frequently executed motion in cooking scenarios, after pick-and-place. We present a pouring trajectory generation approach, which uses force feedback from the cup to determine the future velocity of pouring.

Read more

Paper: An Approach for Automated Multimodal Analysis of Infants’ Pain

In the paper, we propose an automated multimodal approach that utilizes a combination of both behavioral and physiological pain indicators to assess infants’ pain. We also present a unimodal approach that depends on a single pain indicator for assessment.

Read more

Patent: Systems And Methods For Projecting Images Onto An Object (US patent #9,520,072)

In one embodiment, a method for projecting images on a subject includes determining a pose and position of the subject, adjusting a three-dimensional model of an anatomical structure of the subject to match the determined pose and position, and projecting an image of the anatomical structure onto the subject

Read more

Paper: Recent Datasets on Object Manipulation: A Survey

In the paper, we take a significant step forward by reviewing datasets that were published in the last 10 years and that are directly related to object manipulation and grasping. We report on modalities, activities, and annotations for each individual dataset and we discuss our view on its use for object manipulation.

Read more

Paper: Robotic Grasping for Instrument Manipulations

The paper introduces two grasp quality measures that are derived from the two manipulation requirements: interactive wrench requirements and motion requirements for accomplishing a manipulation task

Read more

Paper: Functional Object-Oriented Network for Manipulation Learning

This paper presents a novel structured knowledge representation called functional object-oriented network (FOON) to model the connectivity of the functional-related objects and their motions in manipulation tasks. The graphical model FOON is learned by observing object state change and human manipulations with the objects.

Read more

Patent: Systems and Methods for Planning a Robot Grasp Based upon a Demonstration Grasp (US patent #9,321,176)

In one embodiment, planning a robot grasp of an object includes determining a grasp type that would be used by a human being to grasp the object, determining a position and orientation of the human being's thumb relative to the object, and planning the robot grasp based upon the determined grasp type and thumb position and orientation.

Read more






Book & Book Chapters

Our sponsors