Authors: Diwij Dev, Kai Huang, Jae Lee, Dongjie Liu, and Tian Xia
💻 Source Code: Glitch
📝 Full Paper in PDF
Abstract
We present CodeSwarm-AR, an interactive system for programming education that utilizes swarm robotics and augmented reality (AR). While other projects have explored the use of AR and robotics for general STEM education purposes or block-based programming education, CodeSwarm-AR offers a unique approach with the use of multiple robotics, bi-directional input methods, and directly learning programming concepts like data structures. Through bi-directional code interaction, visualization, and step-by-step demonstration, CodeSwarm-AR provides users with a comprehensive learning experience. Existing code examples are demonstrated through the movements of swarm robots, allowing users to both follow and visualize the construction of a linked list data structure, as well as interact with it using robots to generate new code by manipulating the linked list. CodeSwarm-AR presents a promising new approach to programming education through the integration of swarm robotics and AR.
Modality 1 - Code-to-Robotic Demonstration
In the code demonstration web page terminal, the learner can view the assigned code sample from the instructor. The user can click on the play button next to each step of the code sample, the AR display shows animated virtual objects and texts attached to Toio robots and their tracking mat to supplement the code demonstration.
Modality 2 - Robotic-to-Code Generation
The user physically manipulates the Toio robots on their tracking mat, and the code is generated based on the movements and positions of the Toio robots. The user can view the generated code in the code generation web page and has the option to optimize the code displayed in the terminal.
System Implementation
- Server Connection, Data Transmission, and Hosting
- Desktop Web Page Interface
- AR Interface
- Connection, Control and Tracking of Swarm Robotics
Please refer to the report in PDF for details on design space, prototypes, system setup, and design critiques.