Intelligent Systems Research Group

3D Morphable Models

Morphable human shape models are a prerequisite for a wide variety of tasks such as 3D virtual avatar generation from incomplete 3D data. Important applications are virtual try-on, gaming, and markerless motion capture.

We have developed a differentiable multi-person articulated human shape model that can be trained using joint optimization without any 3D supervision. One of the strength of our approach is that it works with comparatively few input scans. No manually selected landmarks or pretained models are required.

Overview

Besides the input data, the model is based on an user defined skeleton and an attached template base mesh with associated body parts. Based on these inputs, the usage of subdivision surfaces enables us to use joint, continuous end-to-end optimization of a single, self-contained objective. The resulting model then follows known concepts such as linear blend skinning and factorizes shape and pose parameters.

Results

Human Shape Models

We have trained three human shape models - male, female and unisex. Quantitative results and a detailed description of the method is given in our ECCV 2020 paper. The corresponding source code is free for non-commercial use and published in our GitHub-repository JOMS

To use this content (Source: www.xyz.de), please click to Accept. We would like to point out that by accepting this iframes data to third parties transmitted or cookies could be stored.

You can find further information in our Privacy policy.

Some technical details: Each of our human models was trained on approx. 1000 scans from the Caesar and D-FAUST data set. We used approximately 20k points per scan which lead to 3.6 •10^7 latent variables. The training took three days on a RTX 2080 TI with 11GB memory.

Infant Head Model

We applied the same method to scans of infant heads with the goal of measuring baby head deformations due to plagiacephalus, see our project InferMod3D

Related Code & Publications

Zeitvogel S., Dornheim J., Laubenheimer A.
Joint Optimization for Multi-Person Shape Models from Markerless 3D-Scans
16th European Conference on Computer Vision, ECCV, August 23 – 28, 2020. [pdf] [source] [submat]

Zeitvogel S., Laubenheimer A.
An Open-Source Multi-Person Shape Model Training and Inference Pipeline
11th 3DBODY.TECH Conference & Expo, 21-22 Oct. 2020, Lugano, Switzerland. 2020. [pdf]

Zeitvogel S., Laubenheimer A.
Towards End-to-End 3D Human Avatar Shape Reconstruction from 4D Data
Proceedings, 13th IEEE International Symposium on Electronics and Telecommunication ISETC'18, Timisoara, 2018.