In a pioneering endeavor led by Prof. Angela Schoellig from the Technical University of Munich (TUM), ChatGPT is utilized to orchestrate synchronized aerial performances of drone swarms to musical accompaniment.
This application integrates a critical safety filter to avert mid-air collisions, showcasing the integration of large language models (LLMs) like ChatGPT into robotics.
Using ChatGPT to Choreograph Flying Robots
The team claims that this technology's operational interface is streamlined and user-friendly. Doctoral student Martin Schuck chooses a music track and inputs a textual request for suggested choreography using OpenAI's widely used chatbot.
Additional prompts can be provided to the drone swarm before an algorithm assesses the feasibility of the proposed flight paths.
Within the Learning Systems and Robotics Lab, a simulated airfield on screens showcases six drones executing circular flight patterns in sync with the selected music.
Upon approval of the choreography, palm-sized drones take flight from the lab floor, illustrating the seamless translation of virtual choreography to physical execution.
To enable these aerial showcases, Schoellig's team has strategically positioned six ceiling-mounted cameras in a 40-square-meter room, with marked positions on the floor indicating the drones' starting points.
According to the team, this setup enables real-time tracking of quadrocopters equipped with four propellers and motors, ensuring the precise execution of predetermined flight trajectories.
They also report that the choreographic precision of these "airshows" is safeguarded by the system's ability to monitor drone positions 200 times per second, comparing them against the desired flight paths.
The team's approach ultimately harmonizes ChatGPT's choreographic capabilities with the safety filter, aptly termed "SwarmGPT," an intermediary between the AI tool and human operators.
Schoellig said in a press release that "The ChatGPT AI tool was primarily created to generate texts, but it can also suggest choreographies."
"However, it initially knows nothing about the properties of drones and physical limits for the flight paths. So it is clear that ChatGPT makes mistakes."
30 Different Choreographies
The research team has explored over 30 choreographies involving up to nine drones in the past three months. They can develop a secure choreography for 30-second music clips featuring three drones in approximately five minutes.
As the number of drones increases, ChatGPT's computational demands grow, prolonging the choreography proposal process. Nonetheless, Schoellig remains confident that this scalable concept will continue to evolve.
Robots engaged in tasks like object retrieval, cable laying, or door opening using voice control have success rates of 63%, 56%, and 80%.
According to researchers, applications in other robotics contexts have demonstrated varying reliability.
Schoellig views these challenges as motivation, expressing optimism that their methodology will enhance performance across diverse scenarios.
The research team's findings were published in the proceedings of the 2023 IEEE International Conference on Robotics and Automation (ICRA).
