Using AI to turn animated characters into realistic portraits

October 22, 2020

Motion graphic artist Nathan Shipley has been using a StyleGAN encoder to turn works of art into realistic-looking portraits. Above are the Mona Lisa and Miles Morales from Into The Spider-Verse, but his latest focus has been on Pixar characters. So far he's done The Incredibles, Russell from Up, and Miguel from Coco.

Today's reverse toonification experiments with art from @Pixar for Incredibles 2, Up, & Coco.

This framework from @EladRichardson and @yuvalalaluf quickly finds a "real" human face in the #StyleGAN FFHQ latent space. Adding some style randomness too.

More Pixar in thread! ⬇️ pic.twitter.com/uI2v4Tn96m
— Nathan Shipley (@CitizenPlain) October 15, 2020

Keep going for more of Shipley's work. It turns out computers are way better at turning cartoons into realistic portraits than humans are. The results actually look like people as opposed to horrific nightmare monsters. Case in point: these Simpsons monstrosities.

¿Como sería si Miguel fue un niño real? Here's "real" Miguel from @Pixar's Coco. ¿Lindo? ¿Raro? Si. Una prueba de #InteligenciaArtificial #StyleGAN #Coco pic.twitter.com/cmhSjS907C
— Nathan Shipley (@CitizenPlain) October 15, 2020

What if @lilmiquela was real? Using #MachineLearning to create three plausibly real versions of a CG version of a real girl. pic.twitter.com/pRpQ5Ff1Cv
— Nathan Shipley (@CitizenPlain) October 13, 2020

Here's another reverse toonification (realification?) of Rembrandt's self portrait from the @metmuseum. The dark hat gets processed as a crazy frock of hair but the face is totally plausible. #pixel2style2pixel #StyleGAN pic.twitter.com/4Xzoq8wpMT
— Nathan Shipley (@CitizenPlain) October 14, 2020