New Tech Can Manipulate Faces, And Mouths, In Real Time

I was going to save this until April Fool’s Day but it’s too good. Or bad. Or spooky.

Researchers from the University of Erlangen-Nuremberg, Max Planck Institute and Stanford University have made great, crazy and. . . uncomfortable strides with software called Face2Face  that allows them to alter what someone online appears to be  saying, and make that fake-take look really, really believable.

I mean, really believable.

To bring it into sharp focus, students and researchers chose to use real YouTube video of President Obama,  Vladimir Putin, George W. Bush and Donald Trump.  

After researchers essentially create a facial mask--and with a newly perfected ability to move their subjects' mouths just so--you end up watching fake video, without much work to have it happen. 

These videos don’t have fake audio, it should be made clear,  but that seems, really, like the easy part, doesn’t it?



According to Wired, the research is supposed to be presented at a conference in June. Face2Face basically (and I do mean basically)  tracks the real-time  facial gesture of the subjects (in this case, world leaders and one frightening masquerader) and matches them with a researcher-manipulator who treats those real faces like a mask on his own face. It’s fascinating to watch.

“We present a novel facial recognition method that works with any commodity Webcam,” says the research project’s narrator on their YouTube video. It could be (and was, in this experiment)  used to mess with  other YouTube videos. “Since our method only uses RGB [camera]  data for both the source and the target actor, we are able to manipulate YouTube videos in real time.”

The manipulator makes faces, which instantly are the ones the source is making. When the puppeteer raises Trump’s eyebrow, so does Trump in the video. It’s freaky. Want The Donald to pucker up? The worker puckers up, and so does Trump.

In a very benign imagination of what the world will be able to do with this new-found ability to put words into people’s mouths, says,  “Hopefully the app will have become sufficiently mainstream by the time Trump takes office that his advisors can ‘clean up’ his answers to policy questions in real time.”  
Taking the dim and more depressingly realistic view is Keith Darnay, the tech writer for the Bismarck Tribune, who wrote, “Someone can alter a 15-second clip of, say, a presidential candidate, make him or her say something offensive and share it online as the real thing. The rapid, viral powers of the Internet will do the rest. Since truth always seems to play catch up to rumor and innuendo online, the damage will be done and almost impossible to clean up after the fact.”

Somewhere, that scenario sounds like a love poem to some political operative.

3 comments about " New Tech Can Manipulate Faces, And Mouths, In Real Time".
Check to receive email when comments are posted.
  1. Chuck Lantz from, network, March 28, 2016 at 4:28 p.m.

    "Somewhere, that scenario sounds like a love poem to some political operative" 

    ... as well as to defense attorneys who have been confronted in court with seemingly damning video evidence against their clients, and who can now claim "manipulation."  

    Note: Any Firesign Theater fans out there?  Remember the Nick Danger radio play that was interrupted by FDR's famous radio speech after the Pearl Harbor attack, where FDR "unconditionally surrendered" to Japan?   There really is nothing new under the Sun.

  2. Paula Lynn from Who Else Unlimited, March 28, 2016 at 4:42 p.m.

    True about nothing new under the sun except the Firesign Theatre (pass the amunition) didn't have the tech that is available. Nor were there aliens landing. We can just send the people who have issues, especially violent tendacies to actions, to mental health professionals we get out of gumball machines. Really, how many people believe people they know are (DT) lying now and don't care ? This is not cool.

  3. pj bednarski from Media business freelancer, March 28, 2016 at 5:19 p.m.

    My, my, look at the time. I'l be late for my bridge club!

Next story loading loading..