Current technologies enable robots to execute very short, hard-coded commands, but they struggle with learning to perform long-horizon tasks and reasoning about abstract goals. Language models (LMs) let to perform a wide range of language understanding and generation tasks.
A recent paper by Google Research looks for an effective way to combine advanced language models with robot learning algorithms.
Researchers present a novel approach that uses language model knowledge to enable a robot to follow high-level textual instructions for physically grounded tasks. The language model is grounded in tasks that are feasible within a specific real-world context.
In an evaluation, robots are placed in a real kitchen setting and given tasks in natural language. Highly interpretable results are observed for temporally extended complex and abstract tasks, like “I just worked out, please bring me a snack and a drink to recover”.