SyDaVis-AI
Synthetic Data for Vision Systems with AI
Context
In optical quality inspection systems for industrial assembly and production lines, a large number of images are required to train AI algorithms (especially deep neural networks, CNNs). However, since usually only a few product samples or prototypes are available in the development and implementation phase, the training can usually only take place at the beginning of series production. As a result, the AI-based inspection system generates not reliable or even incorrect inspection results at the start of production. Due to the lack of real training data, this also applies to production in "batch size one" and very small batches (< 200 pieces).
The lack of training data reduces the usage possibilities and practically prevents automated quality control from the first work piece. For this reason, users are forced to fall back on conventional image evaluation algorithms, which can only be used successfully with a lot of experience and product knowledge.
Project goals
The aim of the SyDaVis-AI project is to allow AI-based evaluation procedures of image processing systems to be trained with synthetic image data (transfer learning) before the start of production and to be successfully transferred to real applications. The required training data is to be generated from existing CAD models and associated textures in the simulation in order to be able to provide the customer with a validated inspection system even before the start of production. The SyDaVis-AI project pays special attention to the modeling of the camera and the optics, which have so far only been used in a highly simplified form (pinhole camera).
The goal is to integrate a new realistic camera model including optical effects such as distortion and blur into the simulation software. The impact of this improved camera model on the resulting AI systems will be investigated and evaluated. Another special feature of SyDaVis-AI is the attempt to make neural networks more interpretable using innovative visualization techniques. Overcoming the "black box" property can significantly increase the acceptance of AI algorithms in the industrial environment, since the lack of comprehensibility is often a major obstacle.
Methods
Within the project, a real quality inspection task of the automobile manufacturer Daimler AG is solved using a state-of-the-art AI-based image processing system. The simulated objects and environments are modeled and textured based on the real use case and integrated to a realistic scene. In order to achieve a high precision while maintaining robustness of the system, the combination of a realistic replication of the original scene („Scene Engineering“) and the selective randomization of suitable scene properties („Domain Randomization“) is examined in detail. For this purpose, a toolbox based on the rendering engine Unity3D, developed at Karlsruhe University of Applied Sciences, is used, and further developed. The focus is on the identification of suitable randomization parameters and the derivation of more generalizable findings which can also be transferred to other use cases and areas.
In addition, the transferability of neural networks with regard to statistical image and camera properties is examined in order to minimize the difference between synthetic and real images ("Domain Gap"). The main focus of the project is the realistic modeling of the camera lens and the associated defects and inaccuracies (including distortion, depth of field, shutter type, spectral transmission, and alias frequencies). Usually, these influences are neglected in transfer learning with synthetic image data and often a simplified unrealistic camera model (ideal pinhole camera) is used.
In the context of the project, these properties are integrated into the simulation or into a post-processing process and their influence on the neural network trained with them is investigated and comprehensively evaluated. The relevance of realistic camera and lens modeling for the industrial use of AI-based machine vision systems is determined.
In order to allow users to interpret the decisions of the learning system, existing visualization techniques (e.g., Grad-CAM) are being further developed and investigated for use in complex tasks such as object recognition. On the one hand, the visualizations increase the comprehensibility of the learned algorithms. On the other hand, they allow the systematic detection of structural errors (e.g. focus on non-representative object and scene properties). Furthermore, the project team investigates the influence of the developed camera and lens model on the visualization results and potential derives conclusions.
Status
Completed project 01/2021 — 12/2021
Project consortium
This project is funded by
The project SyDaVis-AI is funded by the Ministry of Science, Research and Art Baden-Württemberg (MWK) within the funding line "KI für den Mittelstand" and the Ministry of Economic Affairs, Labor and Housing Baden-Württemberg
