Hochschule Karlsruhe Hochschule Karlsruhe - University of Applied Sciences
Navigation öffnen Navigation schließen
Hochschule Karlsruhe Hochschule Karlsruhe - University of Applied Sciences

Intelligent Systems Research Group

3D Morphable Models

Morphable human shape models are a prerequisite for a wide variety of tasks such as 3D virtual avatar generation from incomplete 3D data. Important applications are virtual try-on, gaming, and markerless motion capture.

We have developed a differentiable multi-person articulated human shape model that can be trained using joint optimization without any 3D supervision. One of the strength of our approach is that it works with comparatively few input scans. No manually selected landmarks or pretained models are required.

Overview

Besides the input data, the model is based on an user defined skeleton and an attached template base mesh with associated body parts. Based on these inputs, the usage of subdivision surfaces enables us to use joint, continuous end-to-end optimization of a single, self-contained objective. The resulting model then follows known concepts such as linear blend skinning and factorizes shape and pose parameters.

Results

Human Shape Models

We have trained three human shape models - male, female and unisex. Quantitative results and a detailed description of the method is given in our ECCV 2020 paper. The corresponding source code is free for non-commercial use and published in our GitHub-repository JOMS

Externer Inhalt

Um diesen Inhalt zu verwenden (Quelle: www.xyz.de), klicken Sie bitte auf Akzeptieren. Wir möchten Sie darauf hinweisen, dass durch die Annahme dieser IFrames Daten an Dritte übertragen oder Cookies gespeichert werden könnten.

Weitere Informationen finden Sie in unserer Datenschutzerklärung.

Some technical details: Each of our human models was trained on approx. 1000 scans from the Caesar and D-FAUST data set. We used approximately 20k points per scan which lead to 3.6 •10^7 latent variables. The training took three days on a RTX 2080 TI with 11GB memory.

Infant Head Model

We applied the same method to scans of infant heads with the goal of measuring baby head deformations due to plagiacephalus, see our project InferMod3D

Related Code & Publications

Zeitvogel S., Dornheim J., Laubenheimer A.
Joint Optimization for Multi-Person Shape Models from Markerless 3D-Scans
16th European Conference on Computer Vision, ECCV, August 23 – 28, 2020. [pdf] [source] [submat]

Zeitvogel S., Laubenheimer A.
An Open-Source Multi-Person Shape Model Training and Inference Pipeline
11th 3DBODY.TECH Conference & Expo, 21-22 Oct. 2020, Lugano, Switzerland. 2020. [pdf]

Zeitvogel S., Laubenheimer A.
Towards End-to-End 3D Human Avatar Shape Reconstruction from 4D Data
Proceedings, 13th IEEE International Symposium on Electronics and Telecommunication ISETC'18, Timisoara, 2018.

Partners & Related Projects

The research results were developed in the projects WISO3D and InferMod3D in collaboration with our partners: