Position within the page tree

Integrative Computational Design and Construction for Architecture (T3.0)
Research
Research Projects
RP 26-1 – AI-supported Collaborative Control and Trajectory Generation of Mobile Manipulators

AI-Supported Collaborative Control and Trajectory Generation of Mobile Manipulators for Indoor Construction Tasks

Research Project 26-1 (RP 26-1)

IntCDC Reserach Project 26-1 – AI-supported Collaborative Control of Mobile Manipulators

AI-SUPPORTED COLLABORATIVE CONTROL AND TRAJECTORY GENERATION OF MOBILE MANIPULATORS FOR INDOOR CONSTRUCTION TASKS

For the automation of indoor construction tasks specifically in existing buildings, mobile robots offer great flexibility and provide digital support for time-consuming tasks such as the positioning of building elements. In this research project, we focus on collaborative path and trajectory generation for a heterogeneous group of collaborating mobile robots equipped with sensors and different manipulator arms. The mobile manipulators can perform positioning tasks autonomously and with high precision. In order to optimize the construction with respect to execution time and accuracy, optimization-based and AI-supported methods for decentralized path and trajectory generation are investigated to enable real-time capability and to allow an adaptive replanning of trajectories based on sensor feedback and environmental maps. Therefore, indoor positioning and environmental sensing algorithms have to be developed using e.g. total stations, laser scanners, cameras, and IMUs. We will use the capability of simultaneous localization and mapping (SLAM) algorithms both for positioning of the mobile manipulators and the acquisition of 3D information of the respective indoor environment. The main goal will be the modification and development of highly precise SLAM approaches for real-time positioning and data collection for a digital twin of the construction site. This digital twin consists of an up-to-date semantically enriched consistent 3D model as represented by a labeled 3D point cloud or 3D voxel grid.

PRINCIPAL INVESTIGATORS

Prof. Dr.-Ing. Dr. h.c. Oliver Sawodny
Institute for System Dynamics (ISYS), University of Stuttgart
Prof. Dr.-Ing. habil. Volker Schwieger
Institute of Engineering Geodesy (IIGS), University of Stuttgart
Prof. Dr.-Ing. Uwe Sörgel
Institute for Photogrammetry (IFP), University of Stuttgart

TEAM

Dr.-Ing. Li Zhang (IIGS)
apl. Prof. Dr.-Ing. Norbert Haala (IFP)
Sahar Abolhasani (IIGS)
Alice Hierholz (ISYS)
Lena Joachim (IFP)
Vincent Reß (IFP)

PEER-REVIEWED PUBLICATIONS

2024
1. Ress, V., Brändle, M., & Haala, N. (2024). Analysis and Implementation of Rotation-invariant Neural Network Architectures for Feature Extraction. 44. Wissenschaftlich-Technische Jahrestagung Der DGPF in Remagen – Publikationen Der DGPF, 32, 12–19. https://www.dgpf.de/src/tagung/jt2024/proceedings/paper/02_KKNP_dgpf2024_Ress_et_al.pdf
  - BibTeX
  BibTeX
  @article{ress2024analysis, author = {Ress, V. and Brändle, M. and Haala, N.}, journal = {44. Wissenschaftlich-Technische Jahrestagung der DGPF in Remagen – Publikationen der DGPF}, pages = {12-19}, title = {Analysis and Implementation of Rotation-invariant Neural Network Architectures for Feature Extraction }, url = {https://www.dgpf.de/src/tagung/jt2024/proceedings/paper/02_KKNP_dgpf2024_Ress_et_al.pdf }, volume = 32, year = 2024 }
2. Gienger, A., Stein, C., Lauer, A. P. R., Sawodny, O., & Tarín, C. (2024). Data-Based Reachability Analysis and Optimized Robot Positioning for Co-Design of Construction Processes. 2024 IEEE/SICE International Symposium on System Integration (SII).
  - BibTeX
  BibTeX
  @conference{noauthororeditor, author = {Gienger, Andreas and Stein, Charlotte and Lauer, Anja Patricia Regina and Sawodny, Oliver and Tarín, Cristina}, booktitle = {2024 IEEE/SICE International Symposium on System Integration (SII)}, title = {Data-Based Reachability Analysis and Optimized Robot Positioning for Co-Design of Construction Processes}, year = 2024 }
2023
1. Koelle, M., Walter, V., Schmohl, S., & Soergel, U. (2023). LEARNING ON THE EDGE: BENCHMARKING ACTIVE LEARNING FOR THE SEMANTIC SEGMENTATION OF ALS POINT CLOUDS. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, X-1/W1-2023, 945--952. https://doi.org/10.5194/isprs-annals-X-1-W1-2023-945-2023
  - BibTeX
  BibTeX
  @article{isprs-annals-X-1-W1-2023-945-2023, author = {Koelle, M. and Walter, V. and Schmohl, S. and Soergel, U.}, doi = {10.5194/isprs-annals-X-1-W1-2023-945-2023}, journal = {ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences}, pages = {945--952}, title = {LEARNING ON THE EDGE: BENCHMARKING ACTIVE LEARNING FOR THE SEMANTIC SEGMENTATION OF ALS POINT CLOUDS}, url = {https://isprs-annals.copernicus.org/articles/X-1-W1-2023/945/2023/}, volume = {X-1/W1-2023}, year = 2023 }
2. Hierholz, A., Gienger, A., & Sawodny, O. (2023). Cooperative Time-Optimal Trajectory Generation for a Heterogeneous Group of Redundant Mobile Manipulators. Proceedings of the 2023 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM), 1296–1301. https://doi.org/10.1109/AIM46323.2023.10196138
  - BibTeX
  BibTeX
  @inproceedings{hierholz2023cooperative, abstract = {In order to increase productivity and resource efficiency in the construction sector, new approaches are being pursued to automate tasks in the interior fitting of existing buildings. For this purpose, a heterogeneous group of redundant mobile manipulators is used for the time-consuming exact positioning of workpieces. The focus lies hereby on cooperative time-optimal trajectory generation for mobile manipulators. To minimize the effort for modeling and formulating the optimal control problem for the different robots, a unified approach based on the Unified Robotics Description Format is developed. The optimal control problem considers the equations of motion of the robots and existing kinematic and dynamic constraints as well as collisions. To enable cooperative trajectories, additional kinematic constraints are imposed on the poses of the end effectors of each mobile manipulator. Different numerical methods, degrees of model abstraction and cost-functionals are investigated with respect to the generated trajectories and the required computation time towards real-time capability for future adaptive trajectory generation using sensor feedback.}, author = {Hierholz, Alice and Gienger, Andreas and Sawodny, Oliver}, booktitle = {Proceedings of the 2023 IEEE/ASME International Conference on Advanced Intelligent Mechatronics (AIM)}, doi = {10.1109/AIM46323.2023.10196138}, issn = {2159-6255}, month = {06}, pages = {1296-1301}, title = {Cooperative Time-Optimal Trajectory Generation for a Heterogeneous Group of Redundant Mobile Manipulators}, year = 2023 }
3. Haala, N., Zhang, W., Joachim, L., Skuddis, D., Abolhasani, S., Schwieger, V., & Soergel, U. (2023). Zum Potenzial von SLAM-Verfahren für geodätische Echtzeit-Messaufgaben. Allgemeine Vermessungsnachrichten (Avn), 163–172. https://gispoint.de/artikelarchiv/avn/2023/avn-ausgabe-052023/7879-zum-potenzial-von-slam-verfahren-fuer-geodaetische-echtzeit-messaufgaben.html
  - BibTeX
  BibTeX
  @article{haala2023potenzial, abstract = {Simultaneous Localization and Mapping (SLAM) zielt auf die Echtzeiterfassung einer Umgebungskarte mittels einer mobilen Roboterplattform, wobei gleichzeitig die Lokalisierung des Roboters bezüglich seiner Umgebung erfolgt. Zu diesem Zweck kombiniert der Roboter Navigationssensoren, wie IMU oder Odometer, mit Sensoren, wie Laserscanner und/oder monokulare, Stereo- oder RGB-D-Kameras, zur Erfassung der Umgebung. Analog zum bekannten Structure-from-Motion-Verfahren basiert die simultane Bestimmung der Aufnahmepose und der 3D-Karte der Umgebung auf einer photogrammetrischen Bündelblockausgleichung. Hierzu werden zugeordnete Bildpunkte und/oder Laserscans zwischen den jeweiligen Aufnahmen genutzt. SLAM-Verfahren haben mittlerweile einen beachtlichen Entwicklungsstand erreicht. Sie kommen in immer mehr praktischen Anwendungen zum Einsatz und können zunehmend auch für ingenieurgeodätische Messaufgaben genutzt werden. Von besonderem Interesse sind dabei Echtzeitanwendungen. In diesem Beitrag wird das Potenzial existierender SLAM-Verfahren für geodätische Aufgaben am Beispiel aktueller Projekte, wie der Erfassung von Baustellen mittels auf einem Kran montierten Kameras und der mobilen 3D-Kartierung von Innenräumen, demonstriert und diskutiert.}, author = {Haala, Norbert and Zhang, Wei and Joachim, Lena and Skuddis, David and Abolhasani, Sahar and Schwieger, Volker and Soergel, Uwe}, journal = {Allgemeine Vermessungsnachrichten (avn)}, month = {05}, pages = {163-172}, title = {Zum Potenzial von SLAM-Verfahren für geodätische Echtzeit-Messaufgaben}, url = {https://gispoint.de/artikelarchiv/avn/2023/avn-ausgabe-052023/7879-zum-potenzial-von-slam-verfahren-fuer-geodaetische-echtzeit-messaufgaben.html}, year = 2023 }
4. Haala, N., Zhang, W., Joachim, L., & Skuddis, D. (2023). SLAM für die Echtzeit-Erfassung von Innenräumen. 22. Internationale Geodätische Woche Obergurgl 2022, 188–179.
  - BibTeX
  BibTeX
  @article{haala2023echtzeiterfassung, author = {Haala, N. and Zhang, W. and Joachim, L. and Skuddis, D.}, editor = {Verlag, Wichmann}, isbn = {ISBN 978-3-87907-738-0}, journal = {22. Internationale Geodätische Woche Obergurgl 2022}, pages = {188-179}, title = {SLAM für die Echtzeit-Erfassung von Innenräumen }, year = 2023 }
5. Christos Parlapanis, Matthias Frontull, Oliver Sawodny. (2023). Feed-Forward Control of a Construction Vehicle’s Hydro-Mechanical Powertrain to Prevent Engine Stalling. Proceedings of the 2023 IEEE Conference on Systems, Man, and Cybernetics (SMC).
  - BibTeX
  BibTeX
  @inproceedings{christosparlapanis2023feedforward, author = {{Christos Parlapanis, Matthias Frontull, Oliver Sawodny}}, booktitle = {Proceedings of the 2023 IEEE Conference on Systems, Man, and Cybernetics (SMC)}, title = {Feed-Forward Control of a Construction Vehicle's Hydro-Mechanical Powertrain to Prevent Engine Stalling}, year = 2023 }

OTHER PUBLICATIONS

DATA SETS

Prof. Dr.-Ing. habil Volker Schwieger

Principal Investigator

Phone: +49 711 685 84040

E-Mail

Prof. Dr.-Ing. Uwe Sörgel

Principal Investigator

Phone: +49 711 685 83336

E-Mail

Prof. Dr.-Ing. habil. Dr. h.c. Oliver Sawodny

Board of Directors

Phone: +49 711 685 66302

E-Mail