KI-Nachwuchs@FH 2021: Building a distributed IT infrastructure for latency-critical AI applications (ILKA)
Advances in artificial intelligence (AI) are fostering digitization in nearly all areas of society today. A wide variety of AI applications and systems are enabling the automation of increasingly demanding tasks. However, to exploit the potentials, skilled workers with specific expertise are required, which are currently not sufficiently available. This is also due to the fact that training such experts requires AI-capable IT infrastructures, which are only available to a limited extent at universities.
The field of latency-critical AI applications is particularly affected by this. Here, AI systems must react quickly to enable sophisticated applications such as autonomous robots or virtual assistance systems, where decisions must be made in fractions of a second. The special purpose processors, high-performance networks and mass data storage required for these tasks are currently not available at universities. This poses major problems both for AI research, including the training of young scientists, and for the teaching of students in these promising areas.
Missing infrastructures for research and teaching in the field of latency-critical AI applications to be created include powerful Graphics Processing Units (GPU) for fast learning and real-time recognition of important relationships in digitized processes. The GPUs are connected to application environments such as robotic cells or Virtual Reality (VR) installations via high-performance communication networks, resulting in minimal transmission times of sensor and control data.
The provision of such an infrastructure opens up completely new opportunities for scientists to pursue their AI research and for students to learn cutting-edge AI skills. This also significantly strengthens the university region as a business location, where advanced AI innovations can be created by research and transferred into products and services by skilled workers. The proliferation of resulting AI applications is an important step towards the digital knowledge society.
Based on the expertise available at Karlsruhe University of Applied Sciences (HKA) and the strategic orientation in the research area of Intelligent Systems, the goal is to enable research and teaching of software architectures, technologies and development methods for latency-critical AI applications.
For this purpose, a fundamental AI-capable IT infrastructure is to be established within the project. This infrastructure is designed to be applicable for university-wide use in research and for the training of young scientists.
The ILKA IT infrastructure for low-latency AI applications will be initially provided for three main application areas of robotics in production, intelligent perception and assistance systems, and intelligent IT infrastructures within the lifetime of the project.
After project completion, the established infrastructure and the experience gained from the three pilot projects will be carried over to other application areas and new user groups.
The main goal is the provision of two high-performance servers with special purpose processors (Graphics Processing Units, GPU), which meets the fact that modern planning- and AI processes require a significant amount of specific compute resources. The servers will be used simultaneously for training of known situations and their independent inference. Especially the inference shall be enabled for applications with real-time requirements and mass data throughput. This means that an upper time limit for the recognition of special situations in complex environments must be guaranteed.
The main characteristic of the planned infrastructure is given by the fact that it will be distributed over distant sites. Still, it should enable cross-site operation for latency-critical AI applications with high bandwidth. High-performance networking and a modular IT infrastructure will be utilized to achieve this. Thus, multiple user groups and main application areas will be flexibly connected to the infrastructure.
Innovation and perspectives
The ILKA IT infrastructure will enable research and teaching in the area of latency-critical AI applications that were previously not feasible. Indeed, access to specialized processors for accelerating machine learning tasks was previously only available in a very limited manner. Moreover, the interconnection of such resources with experimental environments of different use cases was almost non-existent.
After the realization of the project, AI applications with extremely complex real-time requirements can be researched for the first time for various use cases in robotics, participatory assistance systems, and dynamic IT infrastructures.
ILKA's unique infrastructure design, in which specialized processors are distributed across two decentralized sites in the urban area using a high-performance network, enables research on innovative approaches to facilitate the feasibility of latency-critical AI applications. In particular, it is possible to investigate how AI tasks of training and situation inference can be divided among several resources that have different response times due to spatial distribution, still meeting real-time requirements.
The results of this study can be used to realize latency-critical AI applications under difficult conditions even if special purpose processors are only limitedly available at the application's site. As a result, digitization of complex processes could be realized in much more cases.
The ILKA IT infrastructure for latency-critical AI applications is intended not only to strengthen research of the related key technologies in general, but also in particular to enable the promotion of young scientists who need suitable high-performance environments in these areas.
In addition, the planned infrastructure within the ILKA project significantly improves the possibilities of teaching AI technologies, methods, and applications on advanced level. Students will benefit from this, for example, in the context of practical application projects in various master's degree programs.
Ongoing project 10/2021 — 03/2023
Institute of Applied Research Project manager
Prof. Dr. rer. nat. Christian Zirpins
Phone: +49 (0)721 925-1528
The project is funded by
The ILKA project is sponsored by the German Federal Ministry of Education and Research (BMBF) within the funding program “Forschung an Fachhochschulen” under contract number 13FH009KI1.