RI Seminar: Weiming (William) Zhi
Diagrammatic Teaching: A Novel Paradigm for Robots to Learn from Us
Presenter: Weiming (William) Zh of the Robotics Institute, Carnegie Mellon University
Bio: Weiming (William) Zhi is a postdoctoral research fellow with Matt Johnson-Roberson at the Robotics Institute, Carnegie Mellon University.
His research lies at the intersection of machine learning and robotics. His work has tackled problems in motion planning and control, learning from demonstration, motion prediction, and robot mapping. He was recognized as a Robotics: Science and Systems (RSS) Pioneer in 2020 and won the best paper award at L4DC 2022. Before CMU, he completed his PhD with Fabio Ramos at the University of Sydney, where his thesis won the School of Computer Science Outstanding Thesis Award.
Abstract: Humans have a remarkable ability to infer instructions from crude diagrammatic sketches—we wish to endow robots with this same capability. Here, we introduce an alternative paradigm for robots to acquire new skills called Diagrammatic Teaching, where robots learn from sketches provided by the user. By doing so, we can circumvent kinesthetic teaching and teleoperation, allowing robots with high degrees of freedom, such as quadruped mounted manipulators, to attain novel skills from sketched human demonstrations. In this talk, we introduce two frameworks under this paradigm:
(1) Ray-tracing Probabilistic Trajectory Learning (RPTL), which fits a probabilistic model of motion trajectories to ray-traced projections of user sketches over the scene;
(2) Stable Diffeomorphic Diagrammatic Teaching (SDDT), which models the robot's motion as an Orbitally Asymptotically Stable (O.A.S.) dynamical system that learns periodic motion patterns from a sketch over the robot's egocentric view.
