Facebook AI Open-Sources ‘Droidlet’, A Platform For Building Robots With Natural Language Processing And Computer Vision To Understand The World Around Them

Robots today have been programmed to vacuum the floor or perform a preset dance, but there is still much work to be done before they can achieve their full potential. This mainly has something to do with how robots are unable to recognize what is in their environment at a deep level and therefore cannot function properly without being told all of these details by humans. For instance, while it may seem like backup programming for when bumping into an object that would help prevent unwanted collisions from happening again, this idea isn’t actually based on understanding anything about chairs because the robot doesn’t know exactly what one is!

Facebook AI team just released Droidlet, a new platform that makes it easier for anyone to build their smart robot. It’s an open-source project explicitly designed with hobbyists and researchers in mind so you can quickly prototype your AI algorithms without having to spend countless hours coding everything from scratch.

Droidlet is a platform for building embodied agents capable of recognizing, reacting to, and navigating the world. It simplifies integrating all kinds of state-of-the-art machine learning algorithms in these systems so that users can prototype new ideas faster than ever before!


People using droidlet can quickly test out different computer vision algorithms with their robot or replace one natural language understanding model with another. Droidlets enable researchers to easily build agents that can accomplish complex tasks in the real world or in simulated environments like Minecraft or Habitat.

For researchers or hobbyists, droidlet is a fully-developed set of modules that includes primitives for visual perception and language building. These components are available to be used by anyone with programming experience who would like to build robots or simulated agents in the future without worrying about how these systems work individually.

The droidlet platform is powerful and flexible, it can be used outside of the full agent. Over time Droidlet will become even more robust as they add new tasks based on sensory modalities or other hardware setups that others have contributed to.

Paper: https://arxiv.org/pdf/2101.10384.pdf?

Github: https://github.com/facebookresearch/droidlet?

Source: https://ai.facebook.com/blog/droidlet-a-one-stop-shop-for-modularly-building-intelligent-agents

Asif Razzaq is an AI Journalist and Cofounder of Marktechpost, LLC. He is a visionary, entrepreneur and engineer who aspires to use the power of Artificial Intelligence for good.

Asif's latest venture is the development of an Artificial Intelligence Media Platform (Marktechpost) that will revolutionize how people can find relevant news related to Artificial Intelligence, Data Science and Machine Learning.

Asif was featured by Onalytica in it’s ‘Who’s Who in AI? (Influential Voices & Brands)’ as one of the 'Influential Journalists in AI' (https://onalytica.com/wp-content/uploads/2021/09/Whos-Who-In-AI.pdf). His interview was also featured by Onalytica (https://onalytica.com/blog/posts/interview-with-asif-razzaq/).