HOBA
Homogeneous soils assistant for the automatic, construction site-specific recording of soil classes according to the new VOB 2016 HOBA
Motivation
The digitalization of construction processes and the implementation of Building Information Modeling construction ("BIM" for short) are current challenges in civil engineering. The aim of the R&D project HOBA is the development of the hardware and software of a compact sensor and a computing system unit, mounted on the excavator with data interfaces to the excavator IT and with connection to external sensors -briefly called "HKA HOBA-Box".
Research and developments also include multi-sensor (GNSS/MEMS/Optics) 3D geo-referencing of the excavation in the global ITRF reference in conjunction with sensor-based acquisition and computation of a 3D voxel volume model of the excavation, voxel-wise classification of soil types via image-based AI/ML algorithms, and back-calculation of the classified and geo-referenced 2D individual images to the 3D voxel model.
Overall project goal
The R&D project "Homogeneous soils assistant for the automatic, construction site-specific recording of soil classes according to the new VOB 2016 HOBA”, shortly HOBA, deals with the development of a system for an automatic classification, detection & segmentation and a georeferenced voxel-based 3D-volume model generation for excavation site-specific soil types according to the new VOB 2016.
In the project, a homogeneous soils assistant with a graphical user interface is developed, which enables an automatic assignment of soil types to homogeneous areas with quantity determination. The hardware and software development of the above-mentioned HKA HOBA-Box are an innovative contribution to the BIM-compliant digital real-time documentation of soil type-classified excavation work in civil engineering.
Project goals of the R&D
The aim of the R&D at HKA is the development of hardware and software for a compact sensor and computer system unit attached to the excavation machine (example excavator, Fig. 1)) with data interfaces to the excavator IT and for the connection of external sensors - hereinafter referred to as "HKA HOBA-Box". The hardware and software development "HKA HOBA-Box" is an innovative contribution to BIM (Building Information Modeling) compliant digital real-time documentation of excavation work in civil engineering.
The "HKA HOBA-Box" to be developed (Fig. 1, Fig. 2) enables a multisensory 3D geo-referencing of the excavation in the ETRF89/ITRF in combination with the sensor-based acquisition and computation of a 3D model of the excavation based on so-called "voxels" (cube elements as 3D volume units), the classification of the soil types of the excavation via image-based AI/ML algorithms, and finally the back-calculation of the classified and georeferenced 2D individual images into a georeferenced 3D voxel model classified by soil types.
The essential component or the basic element of the complete georeferencing in all above mentioned calculation steps is the 3D navigation status vector to be calculated under algorithmic fusion of all internal sensor data of the HKA HOBA Box (IMU, magnetometer, barometer, inclinometer), the external HOBA Box sensors connected to it (GNSS, ToF (Time-of-Flight) camera and an RGB-optical camera) as well as external navigation state data.
The self-sufficient box variant 1 of the HKA HOBA-Box (Fig. 1, Fig. 2) uses only the data from the HKA HOBA-Box for the sensor fusion in respect to the estimation of y(t), or the SLAM parameters y(t)SLAM=(y(t),m(t)), respectively. In the case of box variant 2, y(t) and vector y(t)SLAM=(y(t),m(t)) are calculated on the HKA HOBA-Box again from all sensors, except the GNSS. The reason is, the unfavorable placement of GNSS at the location of the box in the vicinity of the bucket with regard to signal shading, multipath and cycle slips. Instead, the sensor fusion and SLAM on the box variant 2 are making use in addition of the MTS navigation part solution y(t)'=(xe ye ze )T provided via the local machine server (see local machine server, fig. 2) at the localization of the body (b) and box origin, respectively
The HKA HOBA-Box - in both variants - enables a multi-sensory 3D geo-referencing and voxel-based classified 3D model of the excavation volume in the ETRF89 / ITRF based on (GNSS/MEMS/RGB/ToF-3D-camera Optics) sensor data. The classification of soil types on the site is based on image-related AI/ML algorithms, and finally the re-calculation of the classified and geo-referenced 2D images into the above-mentioned geo-referenced 3D voxel model according to the soil types.
Hardware Developments
The concept of the HKA HOBA-Box hardware design as shown in fig. 2 as central processor unit a NVIDIA Jetson TX2 with a 256-core NVIDIA Pascal GPU, a hex-core ARMv8 64-bit CPU complex and an 8 GB LPDDR4 memory with 128-bit interface is used on the box. The CPU combines one dual-core NVIDIA Denver and 2 processors with a quad-core Cortex-A57 arm.
The HKA HOBA-Box internal sensors include ZED F9 GNSS, compact ICM 20948 sensors (3-axis gyroscope, 3-axis magnetometer and 3-axis accelerometers), and MS5611 barometer and an SCL3300 inclinometer (tilt meter). This box is the above box-variant 1, while variant 2 uses a part of the navigation state y(t)’ - namely the position y(t)’= (xe ye ze) - calculated by MTS from the two GNSS and tiltmeters on the excavator machine (fig. 1) by the use of the Denavit-Hartenberg transformation (DH).
The used backup drive is a WD Red SA500 NAS SATA SSD 1 TB and is used for data backup via defined threshold values and/or a defined backup rate (e.g. hourly).
The LUCID Helios-2 TOF with integrated RBG IP67 kit (Triton 3.2MP) is used as the optical component of the HKA HOBA-Box. The additional digital camera of the LU-CID Helios-2 TOF could is used mainly for the ML/AI-based classification, as well as for texturing the TOF generated point clouds.
The prototype box (fig.3) is set up as full system for the running navigation and SLAM, image processing, voxel generation and geo-referencing algorithms and software developments, while the final smart box is under development as well.
Algorithms and Software Developments
The previous hardware design and the prototype box are used as a base for the algorithms and software implementations as described in fig. 4.
All the calculations are processed centrally on the HKA HOBA-Box, i.e. the NVIDIA Jetson TX2 computer (NVIDIA Pascal Architecture GPU, 8 GB L 128 bit DDR4 memory, 32 GB eMMC 5.1 flash memory and 1.0 TB external memory) there.
The operating system of the HKA HOBA-Box is Linux 18.04 LTS "Bionic Beaver" with ROS Distribution (Robot Operating System-Melodic). The system is already prepared for deep learning based on Python and C/C++ with a lot of necessary dependencies installed.
Initial experimental tests using TF1.x and PyTorch have successfully been carried out on image classification, detection and segmentation using transfer learning with different pre-trained models e.g., ReNet, FCN- ReNet, SSD-Mobilenet, Inception V3, etc.
Status
Ongoing project 02/2020 — 01/2023
Contact
Project manager
Prof. Dr.-Ing. Reiner Jäger
Phone: +49 (0)721 925-2620
reiner.jaeger@h-ka.de
More information
Project partners
HOBA is located at the Institute of Applied Research (IAF) at Karlsruhe University of Applied Sciences (HKA). Research and development are carried out in the GNSS & Navigation Laboratory in collaboration with the main industry partner MTS Schrode AG and their partner VEMCON GmbH
Project funding
HOBA is financed as a so-called ZIM (Central Innovation Program for SMEs) research and development project by the Federal Ministry for Economic Affairs and Climate Action (BMWI)
