Self-Supervised Learning of Lidar Segmentation for Autonomous Indoor Navigation
In collaboration with University of Toronto
AuthorsHugues Thomas, Ben Agro, Mona Gridseth, Jian Zhang, Timothy D. Barfoot
In collaboration with University of Toronto
AuthorsHugues Thomas, Ben Agro, Mona Gridseth, Jian Zhang, Timothy D. Barfoot
We present a self-supervised learning approach for the semantic segmentation of lidar frames. Our method is used to train a deep point cloud segmentation architecture without any human annotation. The annotation process is automated with the combination of simultaneous localization and mapping (SLAM) and ray-tracing algorithms. By performing multiple navigation sessions in the same environment, we are able to identify permanent structures, such as walls, and disentangle short-term and long-term movable objects, such as people and tables, respectively. New sessions can then be performed using a network trained to predict these semantic labels. We demonstrate the ability of our approach to improve itself over time, from one session to the next. With semantically filtered point clouds, our robot can navigate through more complex scenarios, which, when added to the training pool, help to improve our network predictions. We provide insights into our network predictions and show that our approach can also improve the performances of common localization techniques.
September 12, 2023research area Computer Vision
A household robot should be able to navigate to target locations without requiring users to first annotate everything in their home. Current approaches to this object navigation challenge do not test on real robots and rely on expensive semantically labeled 3D meshes. In this work, our aim is an agent that builds self-supervised models of the world via exploration, the same as a child might. We propose an end-to-end self-supervised embodied agent...
November 18, 2022research area Computer Vision, research area Methods and Algorithmsconference NeurIPS
This paper was accepted at the workshop "Self-Supervised Learning - Theory and Practice" at NeurIPS 2022.
Many state of the art self-supervised learning approaches fundamentally rely on transformations applied to the input in order to selectively extract task-relevant information. Recently, the field of equivariant deep learning has developed to introduce structure into the feature space of deep neural networks, specifically with respect to such...