Visual Semantic Scene Understanding for Collaborative
Wood Building System

Research Project 31-1 (RP 31-1)

Weiße Schrift

VISUAL SEMANTIC SCENE UNDERSTANDING FOR COLLABORATIVE INTCDC WOOD BUILDING SYSTEM

Image processing and computer vision enable the automatic digitalisation and advanced analysis of the construction prefabrication process, in particular improving the safety and efficiency of human-robot collaboration. However, the data-intensive nature of modern visual recognition methods, their susceptibility to changes in data distribution, and their primary focus on recognising large structures pose significant challenges in practical collaborative scenarios. Furthermore, data collection and annotation for all possible situations is challenging, and data quality is often compromised by occlusion or sensor failure.

This project aims to develop a comprehensive deep learning based visual recognition and post-analysis approach for complex, multi-party prefabrication environments in timber construction prefabrication. The objectives are to:

  • develop transfer learning based computer vision algorithms for fine-grained recognition of people, materials, and their interactions in prefabrication environments;
  • implement methods to accurately assess algorithmic limitations through reliable estimates of aleatory and epistemic uncertainty at different levels of granularity;
  • detect and signal unexpected events and anomalies, and verify the digital twin on-the-fly against the visually captured state;
  • perform in-depth post-analysis of collaborative assembly, focusing on human-robot-material interactions, movement trajectories, and undefined anomalous behaviours, thereby enhancing automatic exploration of worker activities in highly collaborative dynamic scenes.

The ultimate goal of our visual recognition system is to improve the safety, operational efficiency and adaptability of human-robot co-agency in the IntCDC wood building system.

 

PARTICIPATING RESEARCHER

Jun.-Prof. Dr.-Ing. Alina Roitberg
Institute for Artificial Intelligence (KI), University of Stuttgart 

TEAM

n.n.

 

PEER-REVIEWED PUBLICATIONS

    OTHER PUBLICATIONS

      DATA SETS

            

        To the top of the page