Collaborative Human-Robot Teams for On-Site Construction

Research Project 26-2 (RP 26-2)

COLLABORATIVE TEAMS OF MOBILE MANIPULATORS WITH ADVANCED CAPABILITIES AND HUMANS FOR ON-SITE CONSTRUCTION AND RENOVATION

The project will study how heterogeneous mobile manipulator teams collaborate with human workers during on-site renovation and construction of existing buildings, aiming to create an integrated human-machine co-agency framework for safe, efficient, meaningful collaboration in complex settings. Combining social science, design, and engineering, it will embed robots into current workflows and organizations. Guided by empirical studies and stakeholder input, it will define co-agency scenarios linking human skill and judgment with robotic autonomy and precision. Technically, it will develop highly adaptive, autonomous robots that will perceive, coordinate, and purposefully improvise, with Mixed-Reality interfaces enabling workers to interpret and collaborate effectively with these systems. Validation will focus on ecologically valid renovation use cases, notably the assembly and disassembly of timber stud wall systems, which comprise representative tasks transferable across a broad range of interior construction and renovation applications. Depending on project opportunities, the frameworks may be further examined in serial facade renovation. These realistic renovation scenarios will demonstrate gains in productivity, precision, and sustainability. The results will yield socio-technical design principles and integration guidelines for human-centered, flexible, and sustainable robotic construction.

PRINCIPAL INVESTIGATORS

Prof. Dr. Kai Arras
Principal Investigator
Institute for Artificial Intelligence (KI)
Cluster of Excellence IntCDC

Prof. Dr. Cordula Kropp
Board of Directors
Institute for Social Sciences (SOWI)
Cluster of Excellence IntCDC

Prof. Achim Menges
Spokesperson
Institute for Computational Design and Construction (ICD)
Cluster of Excellence IntCDC

Prof. Dr.-Ing. habil. Dr. h.c. Oliver Sawodny
Board of Directors
Institute for System Dynamics (ISYS)
Cluster of Excellence IntCDC

PARTICIPATING RESEARCHERS

Prof. Dr. Michael Sedlmair
Participating Researcher
Visualization Research Center (VISUS)
Cluster of Excellence IntCDC

RESEARCHERS

M.Sc. Kaan Aytekin
Doctoral Researcher
Institute for Artificial Intelligence (KI)
Cluster of Excellence IntCDC

M.A. Amelie Schreck
Doctoral Researcher
Institute for Social Sciences (SOWI)
Cluster of Excellence IntCDC

B.Sc. Lena Kerber
Maternity/paternity leave cover
Institute for Social Sciences (SOWI)
Cluster of Excellence IntCDC

M.Sc. Xiliu Yang
Postdoctoral Researcher
Institute for Computational Design and Construction (ICD)
Cluster of Excellence IntCDC

M.Sc. Samuel Slezak
Doctoral Researcher
Institute for Computational Design and Construction (ICD)
Cluster of Excellence IntCDC

M.Sc. Alice Hierholz
Doctoral Researcher
Institute for System Dynamics (ISYS)
Cluster of Excellence IntCDC

M.Sc. Jonas Vogelsang
Doctoral Researcher
Visualization Research Center (VISUS)
Cluster of Excellence IntCDC

Dipl.-Ing. Jan Krieglstein
Doctoral Researcher
Visualization Research Center (VISUS)
Cluster of Excellence IntCDC

 

PEER-REVIEWED PUBLICATIONS

  1. 2026

    1. Abolhasani, S., Balangé, L., Zhang, L., Reß, V., Haala, N., Elshani, D., Wortmann, T., Sörgel, U., & Schwieger, V. (2026). Capturing and Modeling of Existing Buildings from Point Clouds Under the Application of Stochastic Information. In A. Kopácik, P. Kyrinovic, J. Erdélyi, & J. Bures (Eds.), Contributions to International Conferences on Engineering Surveying (pp. 111–124). Springer Nature Switzerland. https://doi.org/10.1007/978-3-032-12070-0_10
    2. Hierholz, A., Reß, V., Hentschel, N., Gienger, A., Haala, N., & Sawodny, O. (2026). Active visual SLAM using potential fields and model predictive control. Control Engineering Practice, 172, 106897. https://doi.org/10.1016/j.conengprac.2026.106897
    3. Hierholz, A., Röhm, E., Gienger, A., & Sawodny, O. (2026). Distributed Trajectory Generation based on ADMM for Collaborative Workpiece Transport with Mobile Robots. IEEE 19th International Conference on Advanced Motion Control (AMC). https://doi.org/10.1109/AMC67705.2026.11435816
    4. Kropp, C., & Renner, T. (2026). Automation, Co-Agency, and Distributed Responsibility: Caring for Hybrid Therapeutic Networks. Analyse & Kritik, 48, Article 1. https://doi.org/10.1515/auk-2026-3005
    5. Wehmeier, M., Hain, S., Hagedorn, M., Pfeifer, D., & Sawodny, O. (2026). A Modeling Approach for Robotic Handling of Deformable Objects in Food Retail to Enable Virtual Commissioning. IEEE 19th International Conference on Advanced Motion Control (AMC).
  2. 2025

    1. Friz, F., Hierholz, A., Sawodny, O., & Uchiyama, N. (2025). Function-Parameterized Optimal Trajectory Generation for Mobile Robots. IEEE/IFAC 9th International Conference on Control Automation and Diagnosis (ICCAD′25).
    2. Ress, V., Meyer, J., Zhang, W., Skuddis, D., Soergel, U., & Haala, N. (2025). 3D Gaussian Splatting aided Localization for Large and Complex Indoor-Environments. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLVIII-G-2025, 1283–1290. https://doi.org/10.5194/isprs-archives-XLVIII-G-2025-1283-2025
    3. Zhang, W., Cheng, Q., Skuddis, D., Zeller, N., Cremers, D., & Haala, N. (2025). HI-SLAM2: Geometry-Aware Gaussian SLAM for Fast Monocular Scene Reconstruction. IEEE Transactions on Robotics, 41, 6478–6493. https://doi.org/10.1109/TRO.2025.3626627
    4. Zhang, X., Lin, D., & Soergel, U. (2025). Target-aware attentional network for rare class segmentation in large-scale LiDAR point clouds. ISPRS Journal of Photogrammetry and Remote Sensing, 220, 32–50. https://doi.org/10.1016/j.isprsjprs.2024.11.012
  3. 2024

    1. Collmar, D., Walter, V., & Soergel, U. (2024). Crowd Controls Crowd: Quality Improvement of Polygon Integration in Paid Crowdsourcing. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, X-4-2024, 83–90. https://doi.org/10.5194/isprs-annals-X-4-2024-83-2024
    2. Gienger, A., Stein, C., Lauer, A. P. R., Sawodny, O., & Tarín, C. (2024). Data-Based Reachability Analysis and Optimized Robot Positioning for Co-Design of Construction Processes. 2024 IEEE/SICE International Symposium on System Integration (SII).
    3. Hierholz, A., Gienger, A., & Sawodny, O. (2024). Improving Data-based Trajectory Generation by Quadratic Programming for Redundant Mobile Manipulators*. Proceedings of the 2024 IEEE International Conference on Advanced Intelligent Mechatronics (AIM), 771–776. https://doi.org/10.1109/AIM55361.2024.10636991
    4. Ress, V., Brändle, M., & Haala, N. (2024). Analysis and Implementation of Rotation-invariant Neural Network Architectures for Feature Extraction. 44. Wissenschaftlich-Technische Jahrestagung Der DGPF in Remagen – Publikationen Der DGPF, 32, 12–19. https://www.dgpf.de/src/tagung/jt2024/proceedings/paper/02_KKNP_dgpf2024_Ress_et_al.pdf
    5. Ress, V., Zhang, W., Skuddis, D., Haala, N., & Soergel, U. (2024). SLAM for Indoor Mapping of Wide Area Construction Environments. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, X-2-2024, 209–216. https://doi.org/10.5194/isprs-annals-X-2-2024-209-2024
  4. 2023

    1. Collmar, D., Walter, V., Koelle, M., & Soergel, U. (2023). FROM MULTIPLE POLYGONS TO SINGLE GEOMETRY: OPTIMIZATION OF POLYGON INTEGRATION FOR CROWDSOURCED DATA. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, X-1/W1-2023, 159–166. https://doi.org/10.5194/isprs-annals-X-1-W1-2023-159-2023
    2. Collmar, D., Walter, V., Koelle, M., & Soergel, U. (2023). A two-step approach for the acquisition of individual tree outlines using paid crowdsourcing. 43. Wissenschaftlich-Technische Jahrestagung Der DGPF, 22.-23. März 2023 in München, 163–173. https://doi.org/10.24407/KXP:1841079707
    3. Haala, N., Zhang, W., Joachim, L., & Skuddis, D. (2023). SLAM für die Echtzeit-Erfassung von Innenräumen. 22. Internationale Geodätische Woche Obergurgl 2022, 188–179.
    4. Haala, N., Zhang, W., Joachim, L., Skuddis, D., Abolhasani, S., Schwieger, V., & Soergel, U. (2023). Zum Potenzial von SLAM-Verfahren für geodätische Echtzeit-Messaufgaben. Allgemeine Vermessungsnachrichten (Avn), 163–172. https://gispoint.de/artikelarchiv/avn/2023/avn-ausgabe-052023/7879-zum-potenzial-von-slam-verfahren-fuer-geodaetische-echtzeit-messaufgaben.html
    5. Hierholz, A., Gienger, A., & Sawodny, O. (2023). Cooperative Time-Optimal Trajectory Generation for a Heterogeneous Group of Redundant Mobile Manipulators. Proceedings of the 2023 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM), 1296–1301. https://doi.org/10.1109/AIM46323.2023.10196138
    6. Koelle, M., Walter, V., Schmohl, S., & Soergel, U. (2023). LEARNING ON THE EDGE: BENCHMARKING ACTIVE LEARNING FOR THE SEMANTIC SEGMENTATION OF ALS POINT CLOUDS. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, X-1/W1-2023, 945–952. https://doi.org/10.5194/isprs-annals-X-1-W1-2023-945-2023
    7. Parlapanis, C., Frontull, M., & Sawodny, O. (2023). Feed-Forward Control of a Construction Vehicle’s Hydro-Mechanical Powertrain to Prevent Engine Stalling. Proceedings of the 2023 IEEE Conference on Systems, Man, and Cybernetics (SMC).
    8. Zhang, X., Lin, D., Xue, R., & Soergel, U. (2023). TARGET-GUIDED LEARNING FOR RARE CLASS SEGMENTATION IN LARGE-SCALE URBAN POINT CLOUDS. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLVIII-1/W2-2023, 1693–1698. https://doi.org/10.5194/isprs-archives-XLVIII-1-W2-2023-1693-2023
  5. 2022

    1. Zhang, X., Xue, R., Kölle, M., & Sörgel, U. (2022). Point surfel transformer network for semantic segmentation of large-scale ALS point clouds. 42. Wissenschaftlich-Technische Jahrestagung Der DGPF, 5.-6. Oktober 2022 in Dresden, 290–296. https://doi.org/10.24407/KXP:1796046701

DATA SETS

  1. 2026

    1. Hierholz, A., Röhm, E., Gienger, A., & Sawodny, O. (2026). Replication Data for: Distributed Trajectory Generation based on ADMM for Collaborative Workpiece Transport with Mobile Robots [DaRUS]. https://doi.org/10.18419/DARUS-5722
  2. 2025

    1. Balangé, L., Kerekes, G.-A., Frolow, R., Yang, Y., Abolhasani, S., & Schwieger, V. (2025). Point Clouds of the livMatS Biomimetic Shell at Various Stages of the Construction Process. https://doi.org/10.18419/darus-5093
    2. Hierholz, A., Gienger, A., & Sawodny, O. (2025). Replication Data for: Improving Data-based Trajectory Generation by Quadratic Programming for Redundant Mobile Manipulators [DaRUS]. https://doi.org/10.18419/DARUS-5029

    

To the top of the page