Facebook tool converts smartphone photos into 3D images in a jiffy

Topics Facebook | 3D imaging | smartphones

Photo: Shutterstock

A team of Facebook researchers has created an end-to-end system that helps users turn photos they take on their smartphones into 3D images within seconds.

The new framework provides users with a more practical approach to 3D photography, addressing several design objectives.

Users can access the new technology via their own mobile device and the real-time conversion from a 2D input image to 3D is seamless, requiring no sophisticated photographic skills.

It only takes a few seconds to process and the method is robust enough to work on almost any photo -- new or one previously taken.

Over the last century, photography has gone through several tech 'upgrades' that increased the level of immersion.

"Initially, all photos were black and white and grainy, then came colour photography, and then digital photography brought us higher quality and better-resolution images," said Johannes Kopf, lead author of the work and research scientist at Facebook.

"Finally, these days we have 3D photography, which makes photos feel a lot more alive and real".

The 2D-to-3D photo technique has been available as a "photos feature" on Facebook since late 2018.

To take advantage of this feature, originally Facebook users were required to capture photos with a phone equipped with a dual-lens camera.

Now, the Facebook team has added an algorithm that automates depth estimation from the 2D input image, and the technique can be utilized directly on any mobile device.

To refine the new system, the researchers trained a convolutional neural network (CNN) on millions of pairs of public 3D images and their accompanying depth maps and leveraged mobile-optimization techniques developed by Facebook AI.

The framework also incorporates texture inpainting and geometry capture of the 2D input image to convert it into 3D, resulting in images that are more active and lively.

Each automated step that converts a user's 2D photo, directly from their mobile device, is optimized to run on a variety of makes and models and is able to work with a device's limited memory and data-transfer capabilities.

"Users get instant gratification, as the 3D results are literally generated in a matter of seconds," said the team.

Researchers at Facebook have been working toward new and inventive ways to create high quality, immersive 3D experiences, pushing the envelope in computer vision, graphics, and machine learning.

In future work, the team is investigating machine-learning methods that enable high-quality depth estimation for videos taken with mobile devices.

The Facebook researchers are set to demonstrate their system at SIGGRAPH 2020 conference in August.



(Only the headline and picture of this report may have been reworked by the Business Standard staff; the rest of the content is auto-generated from a syndicated feed.)

Dear Reader,

Business Standard has always strived hard to provide up-to-date information and commentary on developments that are of interest to you and have wider political and economic implications for the country and the world. Your encouragement and constant feedback on how to improve our offering have only made our resolve and commitment to these ideals stronger. Even during these difficult times arising out of Covid-19, we continue to remain committed to keeping you informed and updated with credible news, authoritative views and incisive commentary on topical issues of relevance.

We, however, have a request.

As we battle the economic impact of the pandemic, we need your support even more, so that we can continue to offer you more quality content. Our subscription model has seen an encouraging response from many of you, who have subscribed to our online content. More subscription to our online content can only help us achieve the goals of offering you even better and more relevant content. We believe in free, fair and credible journalism. Your support through more subscriptions can help us practise the journalism to which we are committed.

Support quality journalism and subscribe to Business Standard.

Digital Editor

Business Standard is now on Telegram.
For insightful reports and views on business, markets, politics and other issues, subscribe to our official Telegram channel