AI-Supported Collaborative Control of Mobile Manipulators

Research Project 26-1 (RP 26-1)

IntCDC Reserach Project 26-1 – AI-supported Collaborative Control of Mobile Manipulators
AI-SUPPORTED COLLABORATIVE CONTROL AND TRAJECTORY GENERATION OF MOBILE MANIPULATORS FOR INDOOR CONSTRUCTION TASKS

For the automation of indoor construction tasks, especially in existing buildings, mobile robots offer great flexibility and provide digital support for time-consuming tasks such as the positioning of building elements. In this research project, we focus on collaborative path and trajectory generation for a heterogeneous group of collaborating mobile robots equipped with sensors and different manipulator arms. The mobile manipulators can perform positioning tasks autonomously and with high precision. In order to optimise the construction with respect to execution time and accuracy, we investigate optimisation-based and AI-supported methods for decentralised path and trajectory generation to enable real-time capability and to allow an adaptive replanning of trajectories based on sensor feedback and environmental maps. This will require the development of indoor positioning and environmental sensing algorithms using e.g. total stations, laser scanners, cameras, and IMUs. We will exploit the capabilities of Simultaneous Localisation and Mapping (SLAM) algorithms both for positioning of the mobile manipulators and for acquiring 3D information of the respective indoor environment. The main objective will be to modify and develop high-precision SLAM approaches for real-time positioning and data acquisition for a digital twin of the construction site. This digital twin consists of an up-to-date semantically enriched consistent 3D model as represented by a labelled 3D point cloud or 3D voxel grid. 

 

PRINCIPAL INVESTIGATORS

Prof. Dr.-Ing. habil. Dr. h.c. Oliver Sawodny
Institute for System Dynamics (ISYS), University of Stuttgart
Prof. Dr.-Ing. habil. Volker Schwieger
Institute of Engineering Geodesy (IIGS), University of Stuttgart
Prof. Dr.-Ing. Uwe Sörgel
Institute for Photogrammetry (IFP), University of Stuttgart

TEAM

Dr.-Ing. Li Zhang (IIGS)  
apl. Prof. Dr.-Ing. Norbert Haala (IFP)
Sahar Abolhasani (IIGS)
Alice Hierholz (ISYS)
Lena Joachim (IFP)
Vincent Reß (IFP)

 

PEER-REVIEWED PUBLICATIONS

  1. 2025

    1. Zhang, X., Lin, D., & Soergel, U. (2025). Target-aware attentional network for rare class segmentation in large-scale LiDAR point clouds. ISPRS Journal of Photogrammetry and Remote Sensing, 220, 32–50. https://doi.org/10.1016/j.isprsjprs.2024.11.012
    2. Ress, V., Meyer, J., Zhang, W., Skuddis, D., Soergel, U., & Haala, N. (2025). 3D Gaussian Splatting aided Localization for Large and Complex Indoor-Environments. https://arxiv.org/abs/2502.13803
    3. Hierholz, A., Reß, V., Hentschel, N., Gienger, A., Haala, N., & Sawodny, O. (2025). Active Visual SLAM using Potential Fields and Model Predictive Control.
    4. Friz, F., Hierholz, A., Sawodny, O., & Uchiyama, N. (2025). Function-Parameterized Optimal Trajectory Generation for Mobile Robots. IEEE/IFAC 9th International Conference on Control Automation and Diagnosis (ICCAD′25).

  2. 2024

    1. Zhang, W., Cheng, Q., Skuddis, D., Zeller, N., Cremers, D., & Haala, N. (2024). HI-SLAM2: Geometry-Aware Gaussian SLAM for Fast Monocular Scene Reconstruction. https://arxiv.org/abs/2411.17982
    2. Ress, V., Zhang, W., Skuddis, D., Haala, N., & Soergel, U. (2024). SLAM for Indoor Mapping of Wide Area Construction Environments. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, X-2-2024, 209–216. https://doi.org/10.5194/isprs-annals-X-2-2024-209-2024
    3. Ress, V., Brändle, M., & Haala, N. (2024). Analysis and Implementation of Rotation-invariant Neural Network Architectures for Feature Extraction. 44. Wissenschaftlich-Technische Jahrestagung Der DGPF in Remagen – Publikationen Der DGPF, 32, 12–19. https://www.dgpf.de/src/tagung/jt2024/proceedings/paper/02_KKNP_dgpf2024_Ress_et_al.pdf
    4. Hierholz, A., Gienger, A., & Sawodny, O. (2024). Improving Data-based Trajectory Generation by Quadratic Programming for Redundant Mobile Manipulators*. Proceedings of the 2024 IEEE International Conference on Advanced Intelligent Mechatronics (AIM), 771–776. https://doi.org/10.1109/AIM55361.2024.10636991
    5. Gienger, A., Stein, C., Lauer, A. P. R., Sawodny, O., & Tarín, C. (2024). Data-Based Reachability Analysis and Optimized Robot Positioning for Co-Design of Construction Processes. 2024 IEEE/SICE International Symposium on System Integration (SII).
    6. Collmar, D., Walter, V., & Soergel, U. (2024). Crowd Controls Crowd: Quality Improvement of Polygon Integration in Paid Crowdsourcing. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, X-4-2024, 83–90. https://doi.org/10.5194/isprs-annals-X-4-2024-83-2024

  3. 2023

    1. Zhang, X., Lin, D., Xue, R., & Soergel, U. (2023). TARGET-GUIDED LEARNING FOR RARE CLASS SEGMENTATION IN LARGE-SCALE URBAN POINT CLOUDS. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, XLVIII-1/W2-2023, 1693–1698. https://doi.org/10.5194/isprs-archives-XLVIII-1-W2-2023-1693-2023
    2. Parlapanis, C., Frontull, M., & Sawodny, O. (2023). Feed-Forward Control of a Construction Vehicle’s Hydro-Mechanical Powertrain to Prevent Engine Stalling. Proceedings of the 2023 IEEE Conference on Systems, Man, and Cybernetics (SMC).
    3. Koelle, M., Walter, V., Schmohl, S., & Soergel, U. (2023). LEARNING ON THE EDGE: BENCHMARKING ACTIVE LEARNING FOR THE SEMANTIC SEGMENTATION OF ALS POINT CLOUDS. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, X-1/W1-2023, 945–952. https://doi.org/10.5194/isprs-annals-X-1-W1-2023-945-2023
    4. Hierholz, A., Gienger, A., & Sawodny, O. (2023). Cooperative Time-Optimal Trajectory Generation for a Heterogeneous Group of Redundant Mobile Manipulators. Proceedings of the 2023 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM), 1296–1301. https://doi.org/10.1109/AIM46323.2023.10196138
    5. Haala, N., Zhang, W., Joachim, L., Skuddis, D., Abolhasani, S., Schwieger, V., & Soergel, U. (2023). Zum Potenzial von SLAM-Verfahren für geodätische Echtzeit-Messaufgaben. Allgemeine Vermessungsnachrichten (Avn), 163–172. https://gispoint.de/artikelarchiv/avn/2023/avn-ausgabe-052023/7879-zum-potenzial-von-slam-verfahren-fuer-geodaetische-echtzeit-messaufgaben.html
    6. Haala, N., Zhang, W., Joachim, L., & Skuddis, D. (2023). SLAM für die Echtzeit-Erfassung von Innenräumen. 22. Internationale Geodätische Woche Obergurgl 2022, 188–179.
    7. Collmar, D., Walter, V., Koelle, M., & Soergel, U. (2023). A two-step approach for the acquisition of individual tree outlines using paid crowdsourcing. 43. Wissenschaftlich-Technische Jahrestagung Der DGPF, 22.-23. März 2023 in München, 163–173. https://doi.org/10.24407/KXP:1841079707
    8. Collmar, D., Walter, V., Koelle, M., & Soergel, U. (2023). FROM MULTIPLE POLYGONS TO SINGLE GEOMETRY: OPTIMIZATION OF POLYGON INTEGRATION FOR CROWDSOURCED DATA. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, X-1/W1-2023, 159–166. https://doi.org/10.5194/isprs-annals-X-1-W1-2023-159-2023
    9. Christos Parlapanis, Matthias Frontull, Oliver Sawodny. (2023). Feed-Forward Control of a Construction Vehicle’s Hydro-Mechanical Powertrain to Prevent Engine Stalling. Proceedings of the 2023 IEEE Conference on Systems, Man, and Cybernetics (SMC).

  4. 2022

    1. Zhang, X., Xue, R., Kölle, M., & Sörgel, U. (2022). Point surfel transformer network for semantic segmentation of large-scale ALS point clouds. 42. Wissenschaftlich-Technische Jahrestagung Der DGPF, 5.-6. Oktober 2022 in Dresden, 290–296. https://doi.org/10.24407/KXP:1796046701
To the top of the page