Festo.IO_2023 - Toward Object Manipulation Without Explicit Models

Speaker: Dieter Fox (University of Washington) The prevalent approach to object manipulation is based on the availability of explicit 3D object models. By estimating the pose of such models in a scene, a robot can readily reason about how to pick up an object, place it in a stable position, or avoid collisions. Unfortunately, assuming the availability of object models constrains the settings in which a robot can operate, and noise in estimating a model's pose can result in brittle manipulation performance. In this talk, I will discuss work on learning to manipulate unknown objects directly from visual (depth) data. Without any explicit 3D object models, these approaches are able to segment unknown object instances and manipulate them even in cluttered scenes. I will also present recent work on combining language instructions with action-centric representations to efficiently teach robots a variety of manipulation tasks. I'll conclude with a discussion of the role simulation can play in the future of robotics. https://io.festo.com/