Post originally appearing on fastcompany.com by Mark Sullivan.
The metaverse—and by extension the mixed-reality headgear we might use to access it—had a moment in the sun in 2021, in part because of the social isolation of the pandemic, and in part because Mark Zuckerberg seemed to bet the future of Meta (née Facebook) on it. But much of the hype around the idea of an immersive virtual public space has vaporized as it has become clear that the hardware, software, and standards needed to create the experience are simply not ready.
As the public’s gaze moved on, consumer interest in augmented and virtual reality seemed to flag. In the U.S., for example, sales of VR headsets sagged 2% year-over-year (as of early December 2022), after doubling year-over-year the year prior, according to research from NPD Group. Yet tech companies big and small continue investing big money and top-tier talent in spatial computing devices and experiences.
Despite losing $10 billion on its Reality Labs business in 2021, Meta has said it expects to continue spending at that same rate to advance its VR, AR, and metaverse ambitions. The company defined the state of the art in VR headsets with its new Quest Pro, which featured much-improved pass-through imaging, new eye-tracking technology, and better hand controllers.
Other companies on our list tackled specific aspects of the AR/VR ecosystem. In content, Archer’s Mark, Innersloth, PatchXR, and Rendever leveraged spatial computing to breathe new life into formerly 2D experiences. Unity and Varjo found new ways of making AR/VR work for enterprises. Niantic reached impressive scale with its AR mapping layer, which will allow developers to anchor virtual objects at fixed points in the real world. And Coca-Cola found a mix of social and AR that helped it connect with tech-forward customers.
The wildcard in mixed reality’s journey toward the mainstream is Apple, which reportedly has hundreds of engineers working in secret on its own mixed-reality platform. It may be years before all the moving parts needed for a mainstream metaverse are ready to go. In the meantime, we can look forward to steady improvement in both the hardware and software needed for cool—if smaller-scale—spatial computing experiences.
1. UNITY
For integrating data with digital twins
Unity, known for its dominant 3D gaming engine, has been working with organizations that manage several large airports to develop digital twins of their facilities. One of them is the Vancouver Airport Authority, which launched its digital twin in March 2022. The technology, which intakes data from sensors placed around the airport as well as historical data, allows airport personnel to visualize many aspects of real-time operations, and can be used for training, optimization, future planning, simulations, and testing. This allows administrators and planners to make data-driven decisions to respond to situations that could affect passenger experience or safety. For instance, the airport might anticipate an increase in auto traffic around the airport, model its likely effects (such as increased security wait times), then plan accordingly. In September 2022, Unity unveiled its most ambitious digital twin effort, working with the Orlando Economic Partnership to produce the virtual 800-square-mile region to help the metropolis do a wide array of urban planning work, including climate change, construction and utility projects, and transportation. Unity doesn’t break out revenue from its digital twin work but the company grew 25% year over year in 2022, with revenue of $1.4 billion.
2. COCA-COLA
For adding fizz to the cultural conversation with mysterious new flavors
Plenty of companies leveraged social media in an effort to stand out from the competition in 2022, but few (if any) kept people talking like Coke did with Coca-Cola Creations. Beginning in February 2022, the company released mysterious new flavors with elaborate (but comparatively low-cost) campaigns designed to generate social conversation. The first soda to roll out was Starlight, which the company claimed was space-flavored. It included an augmented-reality concert by singer-songwriter Ava Max that could be accessed only by scanning a QR code on a Starlight can. When a user did so, they could see the performance, which appeared to be on a translucent stage on a space station.
In April, Coke followed up with Sugar Byte, which it swore tasted like pixels. This flavor targeted gamers, launching in Fortnite with a game accessible within its realm. Like Starlight, the actual soda was sold in very limited quantities, first in Latin America and later in the United States. The exclusive nature of the product further amped up chatter on social media.
An exclusive collaboration with the DJ Marshmello followed in June 2022. Coca-Cola did a Twitch takeover in July, meaning that the first ad Twitch users saw was for this promotion. Scanning the QR code on these cans took viewers to a colorful, morphing video that resembled something between a mood ring and a lava lamp while a new Marshmello song played. Another new flavor, Dreamworld, arrived in August, with a digital clothing collab for users’ avatars and a shareable AR mural.
Although Coke has not released data on the impact of the campaign, CEO James Quincey told investors several times during the year that Creations exceeded expectations and had “tremendous traction” and engagement. Overall, Coca-Cola increased global case volume 5% during 2022.
3. META
For making a “real” mixed-reality headset
At $1,499, Meta’s new Quest Pro VR headset is far more expensive—$1,100 more—than its popular Quest 2 VR headset. But from a technical point of view, the Pro advances some of the technologies that will be needed in a true mixed-reality device and that were simply missing in the Quest 2. The Pro’s vastly improved “pass-through” image, for example, lets wearers see the real world in front of them far more clearly (and in color), so that graphics can be integrated into that real world more believably. The Quest Pro also adds eye-tracking, which, among other things, lets the wearer’s avatar look around and make eye contact with the avatars of others as they’re speaking. The Meta Quest Pro is hardly perfect: It’s a bit ungainly, could be more comfortable, and its battery life is only two hours. That said, it’s a harbinger of what’s to come in the world of mixed-reality headsets. Meta has been criticized for its spending to create a new platform where it could control both the hardware and software; it spent $5 billion on this effort in the fourth quarter of 2022 and an estimated $49 billion since 2012. Although the company has recently said that “efficiency” is a key focus for 2023, it already has what may be the most advanced standalone VR and mixed-reality device on the market today—and is still devoting vast resources to further development.
4. GOOGLE
For using 3D imaging to bring people together
Project Starline is Google’s response to a post-pandemic world where remote work has changed the way we communicate. Since remote meetings such as 2D Zoom calls have their obvious drawbacks, Google sought to build an advanced 3D teleconferencing booth that makes remote meetings seem a lot more like real, in-person communication. The company says that the experience is made possible by a convergence of breakthroughs in 3D imaging of people, compression of the 3D video signal for efficient transmission, and the 3D displays needed to render people in a life-size and lifelike way.
In 2022, Google began providing some large companies—including WeWork and Salesforce—with Project Starline booths so that workers could begin testing and providing feedback on the experience. So far Google has spoken only about the performance of the technology, not about its actual hardware components and how much it all might cost, assuming it decides to commercialize it. It’s possible that Google sees the Starline tech as a new, headset-free approach to mixed reality that it can develop as a foundational technology to be used in future AR/VR products: Both headset-based AR/VR efforts and Project Starline are part of a group called Google Labs.
5. INNERSLOTH
For bringing Among Us to VR
Few games have benefited as much as being ported to VR as Innersloth’s Among Us. As in the earlier 2D versions of the game, which remain among the best-selling titles according to the NPD data, players (represented as little armless cartoon astronauts) work together to repair a spaceship—knowing all the while that one or more of the “crew members” is secretly an “imposter” bent on sabotaging the ship and killing everybody on board. Among Us VR, which Innersloth created with help from Schell Games and released in November 2022, amps up the tension by putting game play in a first-person point of view within the immersive 3D environs of the ship. Players can see what’s in front of them, but danger may lurk behind a wall or around the next corner. Among Us VR makes compelling use of spatial audio: Players might hear the squish and thunk of a crew member being killed in some other part of the ship, or footsteps ominously approaching, or the close voices of other crew members frantically trying to deduce the identity of the imposters before it’s too late. Among Us was already a suspenseful experience in 2D, but the $10 VR version takes it to another level—which some players have termed, in the best possible way, “horrifying.”
6. NIANTIC
For growing an AR map of the world
At its developer conference in May 2022, Niantic formally launched its “Lightship Virtual Positioning System,” a virtual map of the world that allows AR developers to anchor 3D graphics to physical places. For example, a developer might hide a digital prize near a well-known statue as part of a scavenger hunt game. These objects are persistent—that is, users can find them tethered to the same real-world place when they leave and return. Niantic’s map is important because developers need it to create Pokémon Go–style games instead of closing themselves inside a VR headset. The map is growing and spreading rapidly. When it launched last spring, there were 30,000 VPS-activated public locations, mostly in San Francisco, London, Tokyo, Los Angeles, New York City, and Seattle. As of December 2022, the map has more than 140,000 VPS-activated public locations in 125 cities around the world.
7. RENDEVER
For using VR to help seniors
Rendever is using VR to promote engagement and mental fitness among senior citizens. The Somerville, Massachusetts–based VR content company operates a platform that delivers customized 3D immersive experiences to nursing homes and other senior living facilities. The content lets people relive moments from the past (weddings, for example) and virtually visit bucket-list destinations. Rendever’s experiences are designed to fight off feelings of loneliness and isolation, which research shows are common among its target audience, and to stave off dementia by challenging users’ minds. In 2022, the company launched RendeverFit, a VR program that combines physical fitness with cognitive stimulation and socialization. It consists of three different modules—Cycle, Paddle, and Paint—each designed to let seniors “gain the benefits of physical activity without feeling like they’re working out.” The five-year-old company says that it has now delivered more than one million VR experiences, and it picked up 3,000 new users in 2022. As touching videos of old folks immersed in 3D memories for the first time attest, it’s a deft use of the new technology for good.
8. ARCHER’S MARK
For reliving history through VR
Archer’s Mark‘s “On the Morning You Wake (to the End of the World): Take Cover” debuted at the Sundance Film Festival’s New Frontiers program in January 2022. The production studio’s narrative VR work lets participants relive the horrifying morning in Hawaii on January 13, 2018, when residents began getting text messages saying that a nuclear attack on the island was underway. The five-minute VR story may be the first to incorporate real-life emergency alerts within an immersive 3D environment to preserve to memory the terror and chaos of a real historical event. “Take Cover” leaves behind it a powerful question: Why does nuclear weaponry’s Sword of Damocles still swing so close overhead?
9. VARJO
For easing VR cloud streaming for enterprises
Varjo started out making high-end mixed-reality (XR) headsets for enterprises, but it has more recently expanded to help those businesses overcome the challenge of administering XR content to their teams (such as designers). Doing this one seat at a time is complicated, data-intensive, expensive, and requires hardware with lots of processing power, so in April 2022, the Finnish company introduced what it calls the “Varjo Reality Cloud” to stream high-resolution, mixed-reality content down to less powerful PCs and headsets across an organization. The onboarding and security of new users happens in the cloud. The end result is that joining an XR collaboration session is more like joining a Zoom call, and it’s attracted a number of automotive industry customers such as Kia, Rivian, and Volvo. Varjo announced in November 2022 that its cloud service can now stream high-quality XR content powered by Unity’s gaming engine and Epic Games’s Unreal Engine.
10. PATCHXR
For bringing virtual instruments to VR
Playing around with virtual musical instruments within a digital audio app such as Apple’s Garageband is lots of fun, but it’s a 2D experience that can be visually confusing, especially as more instruments come into play. PatchXR‘s Patchworld experience for Meta Quest 2, which was released in July 2022, surrounds the player with virtual musical instruments in VR. It’s surprisingly robust as a creation tool—if only for electronic music. Users can select (or make their own) musical instruments and play them within bizarre virtual spaces, or jam in the same space with friends. They can also record the performance, sing, and add cool effects. Best of all, the expansive 3D space makes it all simple. It’s possible to jump right into one of the ready-to-play worlds and start creating and remixing. The $30 app has earned five-star ratings from 87% of reviewers in the Quest app store.
