Machine Learning
Applying machine learning to digital-media creation and user experience in image and motion domains
LEGO Digital Play
Exploring the future of play
Machine Learning for Graphics
DigitalFish delivers solutions for digital storytelling, and within that we are applying machine learning (ML) to challenges within 3D character animation, digital production and augmented reality. Our focus areas include ML applied both in the image (including video) and 3D-motion domains. Our recent work includes:
Image Domain
- Face tracking
- 2D and 3D pose tracking
- Segmentation and classification
Motion Domain
- Denoising for motion-capture cleanup
- Retargeting from motion capture and pose tracking to rigged characters
- Motion style transfer
We apply varied deep neural network (DNN) architectures to these problems, both in offline and real-time contexts, using multiple deep-learning frameworks including TensorFlow, Caffe and PyTorch.
For training data, we use public databases, we synthesize data, and we have access through partnerships to very large proprietary motion databases for the exclusive purpose of ML training.
CartoonGAN: Chen, Lai and Lui; CVPR 2018
We anticipate solutions enabled via motion ML will represent a large market opportunity in 2020 and beyond, with applications in games, narrative media and non-entertainment fields. Within these application areas, motion ML will both create new efficiencies for existing creators and will make 3D content creation accessible to a much larger audience of new creators for whom existing tools and workflows are too complex.
Relevant Recent Projects
