Autonomous mobile robots (AMRs) have become a common sight in modern factories and warehouses. However, conventional robots rely on predefined algorithms and sensor configurations. When tasks or environments change, adapting them usually requires reprogramming by a technician.
What if operators could simply give commands in everyday language, such as “move right and avoid the obstacle,” and the robot translated them into movement?
In a paper published in the IAES International Journal of Robotics and Automation (IJRA), Mishra et al. build a low-cost AMR prototype (Raspberry Pi 4, webcam, YOLOv5 object detection) connected to an LLM through a cloud API.
Abstract
The dynamic evolution of industrial automation has created an imperative need to transition from inflexible, rule-based systems to flexible, intelligent agents that can facilitate human-robot collaboration. The purpose of this research was to integrate large language models (LLMs) with autonomous mobile robots (AMRs) to improve context-aware decision-making. The inflexibility of traditional systems has often hindered performance in dynamic environments, as systems often rely on predefined algorithms and sensor configurations. To improve this, a modular framework was created, consisting of a central processing unit and an LLM API to interpret natural language and process environmental information. Quantitative results have been clearly specified in the abstract, which states that the success rate in resolving navigation exceptions by the proposed framework was 89% with a 5% localization error rate. Moreover, substantial savings were observed in token usage and computational resources. This study provided an imperative framework for smarter industrial automation, filling the gap between mechanical precision and artificial intelligence. The practical experimentation of an AMR model gives an outcome of a successful navigation exception resolution rate of 89% by means of the proposed framework, with an error rate of 5% localized to each exception. Additionally, significant reductions in the number of tokens used and the time taken to process tokens provide a scalable means for developing contextually-based, robust, autonomous mobile platforms’ decisions.
According to the authors, the system resolved 89% of navigation exceptions in their tests. They also acknowledge that reliance on cloud APIs introduces latency and potential risks when the network connection is poor.
Curious about the testing methods and full results? Read more in IJRA Vol. 15, No. 3.
IJRA invites academics, researchers, and practitioners to contribute to the fields of coordination algorithms and autonomous multi-robot systems.