Master's Thesis Award Winners 2024

Bridging the Sim-to-Real Gap in Collective Robotic Construction: A Mixed Reality Reinforcement Learning Approach

Niki Kentroti, Rabih Koussa, Che Chen Hu
Supervisors: Prof. Thomas Wortmann, Prof. Achim Menges
Advisors: Lasath Siriwardena, Shermin Sherkat

The transition of collective robotic construction from controlled laboratory settings to real-world applications faces a significant barrier: limited autonomy. Advances in learning-based approaches offer promising potential to allow robots to autonomously discover optimal construction strategies through cost-effective, scalable, and safe simulation training. Nonetheless, simulation environments cannot entirely capture the complexities of real-world conditions, leading to the phenomenon known as the "reality gap." To address these limitations, this research leverages a mixed-reality (MR) learning environment for augmenting simulation-based training with real-world observations, action executions, and environment state changes. To this end, a sim-real-sim workflow was developed as a feedback loop between the Unity simulation environment (ML-Agents Toolkit), the robot operating system and the OptiTrack motion capture system. The effectiveness of the MR approach is demonstrated through the execution of an isolated task from the assembly of a bending-active structure by a team of Roaming Autonomous Distributed robots. The experiment involves initial simulation training to manipulate a rod and reach a target collaboratively, and consists of two distinct training streams (S) for additional training. S1 involves the continuation of the simulation-only training, and S2 the transition into MR training instead. Results validate that the MR stream outperforms the simulation-only one during deployment, with agents exhibiting the emergence of error-recovery behaviours in response to previously unseen environment states. Ultimately, the proposed workflow leverages the cost-effective nature of simulation training while incorporating essential real-world variability to diminish the reality gap and increase the feasibility of deploying CRC systems in construction tasks.