What are lightfields and why are they important? This is a question that VRARA SF got to tackle last week by interviewing Lytro at our Fall chapter event (video below).
The practice of lightfields is split between capture (cameras and lens arrays) and display (Holographic panels and VR headsets). Lytro is the industry leader in capture, with a range consumer to professional/cinematic cameras. It's latest work is Hallelujia produced with Within.
In fact, our discussion with Lytro came one week before Hallelujia's public release. You can see that here, making this week's Friday video a double feature. The full Hallelujia production involves 6DOF positional tracking, but this more portable version offers 3DOF head tracking.
As for our discussion with Lytro, the company's insights on lightfield capture have been formed over a long tenure in the field. Starting with refocusing cameras, it evolved into cinematic rigs, and on to today's flagship Immerge camera, which was used in the Hallelujia production.
One thing evident from the discussion is the degree that lightfield capture blends optical and digital technologies, making the field quite advanced. Some of the challenges on the digital side will recede as Moore's Law improves processing and compression of lightfield data.
But at the same time, Moore's Law doesn't govern optical technologies. Notice how camera lenses haven't changed much while other consumer tech like smartphones and flat screen TVs have streamlined. Optical equipment is generally the size it needs to be to do its job.
So while the digital barriers of lightfield capture will get smaller with Moore's law, some of the camera rigs and lens arrays required to capture lightfields will remain large (not a bad thing). For example, the degree of parallax you can achieve from a lens array is a function of its diameter.
Lightfields also aren't a silver bullet for AR & VR. Polygon approaches (graphics) are prevalent today in volumetric VR, and will continue to be advantageous for interactivity (games, etc.). But lightfields will excel for playback of photorealistic or cinematic content. They can work together.
"We absolutely see them coexisting," said Lytro Senior Architect Colvin Pitts. "It's a little bit tricky to manipulate lightfield data in real time, but combining the two is absolutely on the table."
See the full interview below with Pitts and Lytro Director of Engineering for VR, Alex Song. And stay tuned for more VRARA SF Chapter events in the coming months.
