Spokesperson
Phone:
+49 711 685 82786
TOWARDS REAL-TIME ADAPTIVE ROBOTIC BEHAVIOR: USING MULTIMODAL FEEDBACK AND REINFORCEMENT LEARNING FOR HUMAN-ROBOT COLLABORATION, BY AND FOR HUMANS AND ROBOTS
This project addresses key challenges in human–robot collaboration (HRC) in prefabrication and construction: uncertainty, error propagation, and communication breakdowns that limit efficiency, safety, and user trust. State-of-the-art HRC systems often rely on static task assignments and one-directional interaction models, tasking users with error detection and recovery. In dynamic construction environments, characterised by material variation, interruptions, and shifting human roles, such approaches increase cognitive load, frustration, and reduce collaboration quality.
Resilient HRC requires co-adaptive processes where humans and robots jointly detect errors, negotiate corrective strategies, and communicate actions and intentions. While advances in reinforcement learning (RL), sensing, and multimodal interfaces show promise, significant gaps remain: existing error taxonomies insufficiently address uncertainty-rich workflows; adaptive robotic behaviours are rarely tested in construction; and multimodal communication is underexplored under site conditions, like noise and movement.
Building on prior studies in collaborative timber assembly, we aim to develop real-time adaptive robotic behaviours and multimodal communication methods for effective feedback loops in HRC. Previous work revealed persistent error propagation, mismatched collaboration speeds, limited coordination channels, and high cognitive demands, motivating a shift toward systems that adapt to both environmental conditions and human collaborators.
The project pursues three objectives: (1) studying how diverse users recognise and resolve errors and extending error taxonomies; (2) translating human strategies into adaptive robotic behaviours using sensor-supported reinforcement learning; and (3) designing and evaluating multimodal communication methods—visual, auditory, and haptic—that provide clear, low-cognitive-load feedback. Outcomes include validated HRC strategies, design guidelines, and open-source tools supporting resilient and inclusive collaboration in construction.
PRINCIPAL INVESTIGATORS
Prof. Achim Menges
Institute for Computational Design and Construction (ICD), University of Stuttgart
Prof. Dr. Thomas Wortmann
Department for Computing in Architecture, Institute for Computational Design and Construction (ICD/CA), University of Stuttgart
PARTICIPATING RESEARCHER
Prof. Dr. Michael Sedlmair
Institute for Visualization (VISUS), University of Stuttgart
RESEARCHERS
PARTNER
Dr. Tanja Blascheck (VIS)
FUNDING
IRIS-HISIT Az.: MWK11-0430.0-1/7/5
