Commentary

Familiar With Prisma?

Of course you are. It’s the Russian app that all your cool friends have been using to “filter” their social media in the style of artistic schools like Expressionism and Cubism.

For Facebook, Prisma is also a great way to get people more excited about its video services. As such, the app’s latest update gives users to ability to stream similarly filtered videos on Facebook Live. Per the change, Prisma users can augment video “broadcasts” using eight available filters.

Facebook’s filtering ambitions go way beyond Prisma, however. Indeed, the social giant has recently been hard at work on “style transfer”: a technology that can learn the artistic style of a painting and then apply that style to every frame of a video.

“This is a technically difficult trick to pull off, normally requiring that the video content be sent to data centers for the pixels to be analyzed and processed by AI running on big-compute servers,” Mike Schroepfer, Facebook's chief technology officer, notes in a new blog post.

The time required for data transfer and processing made for a slower experience, according to Schroepfer. So, about three months back, he and his team set out to do something unique: ship AI-based style transfer running live, in real time, on mobile devices.

“This was a major engineering challenge, as we needed to design software that could run high-powered computing operations on a device with unique resource constraints in areas like power, memory and compute capability,” Schroepfer explained.

The result is Caffe2Go, a new deep learning platform that can capture, analyze and process pixels in real time on a mobile device.

“We found that by condensing the size of the AI model used to process images and videos by 100x, we’re able to run deep neural networks with high efficiency on both iOS and Android,” according to Schroepfer. “This is all happening in the palm of your hand, so you can apply styles to videos as you’re taking them.”

All backed by continued investment in AI, Caffe2Go and similar platforms are part of a broader effort to get more people streaming and sharing content on Facebook’s platform -- and less time using rivals apps like Snapchat.

Next story loading loading..