Today we’re featuring a guest post from Evan Helda, the Principal Specialist for Spatial Computing at AWS, where he does business development and strategy for all things immersive tech: real-time 3D, AR, and VR.
Come see Amazon AWS at our IGS at Metacenter Global Week!
Evan has been at the forefront of the immersive technology industry for the last 7 years. His experience spans numerous layers of the AR/VR/3D tech stack; from AR headsets and apps (the OG Meta), to simulation and game engines at Improbable, to cloud and edge computing/5G (AWS).
Evan also writes a newsletter called Medium Energy, where he explores the impact of exponential technology on the human experience.
We recently came across Evan's writing and thought you might enjoy his perspective on the Apple Vision Pro. If you do like this piece, we encourage you to check out more of his content over at MediumEnergy.io!
####
Today was the big day.
The fateful day our tired and battle-worn industry has waited for; for a long, long time. So many troughs of disillusionment, so many clunky demos, so many shattered startup dreams...
We all sat bated breath, leaning forward with anticipation.
The backdrop was out of a movie: dozens of industry experts, leaders, investors, and entrepreneurs, sprawled across rows of couches on the beach-side deck of a Malibu mansion.
To our right, waves crashed rhythmically, bringing sea foam right up to our feet. To our left, a sprawling spread of breakfast delicacies and of course, champagne. Copious amounts of champagne. The extent to which it would be popped & consumed? TBD... Directly ahead was a massive flat screen TV unfolding what we all hoped would be our industry's 'big bang'.
And then, the moment finally arrived. Apple CEO, Tim Cook, re-appeared and said those historic words, "But wait... there's just one... more... thing".
Our small crowd erupted with hoots, hollers, and applause. My skin erupted with goose bumps.
As the Apple Vision Pro faded onto the screen, it felt like a dream. And for a split second I did dream, flashing back to another fateful day.... five years prior.
The Office of the Future (Spring 2018)
Today was the big day.
The fateful day our augmented reality startup, Meta (the original Meta…), would finally fulfill the promise our CEO had made to the world; to throw away our computer monitors and replace them with a more natural human-computer interface: an AR headset that would blend the physical world with the digital.
Meta 2 AR Headset
We called it 'spatial computing'.
Our CEO made this promise on the grand stage that is TED (worththe 10 mins to watch here). And in about a month, Bloomberg was set to visit our office. Their tech reporters wanted to see this bold exclamation for themselves and write an article on the outcome.
CEO, Meron Gribetz, on the TED stage
We were being held accountable. The boats were burned. There was nowhere to hide.
Today was the dress rehearsal for that Bloomberg visit. All 100 hundred employees would finally taste the fruits of our labor; three years of blood, sweat, and tears towards building a fully vertical AR stack; our own display system, our own sensor array for positional tracking, our own SLAM algorithms, our own hand tracking algorithms, our own SDK, and most importantly... our own 'spatial operating system'.
This was no ordinary OS. It was meant to be the 'OS of the Mind': one that would conform to how our brains have naturally evolved, guided by what we called 'the principles of spatial design'.
(If you watched the Vision Pro announcement... sound familiar? It's no coincidence. Apple seriously considered buying Meta back in 2017. Our 'spatial design principles' and vision for a SpatialOS were a big reason why. Oh, what could have been…)
We would place virtual monitors all around us at limitless scale. We would free 3D models from the confines of 2D screens, visualizing and interacting with them as they were always intended: spatially.
Gone were the days of the mouse & keyboard. Thanks to computer vision, we would use our hands to more naturally & directly interact with the digital world, just as we do the physical.
This same computer vision tech would turn the world into our desktop background, understanding the environment and anchoring actualized figments of imagination all around us.
Meta, the OG Meta... was going to build the true iteration of Steve Job's 'bicycle of the mind': a computer that grandma or a child could pick up and intuitively know how to use, with zero learning curve.
Oh, how beautiful the vision was...
But oh... how naive we were.
The Revenge of the Meta 2
The office that day of the ‘Bloomberg rehearsal’ was buzzing with anticipation.
For the first time, we each received our own headset. The packaging was a work of art; a beautiful piece of engineering and design, accompanied by a thoughtful developer guide and a document outlining our 'spatial design principles'.
The first step: plugging the tethered device into a computer and then into the wall for power (yes, the sequencing mattered...).
It was a collective stumble right out of the gates. Our computers failed to recognize the device. For the next hour, we twiddled our thumbs as the engineers scrambled to fix a bug and re-distribute the SDK (software development kit)
Hot start.
Once the 'Spatial OS' finally launched, the user was tasked with calibrating the sensors and mapping the world.
A graphical UI instructed you to look up, look down, look left, look right.
The next 5-10 minutes was a comical display of 100+ people in a state of manic indecision; stuck between vigorous yes's and no's; shaking our heads this way and that; waiting, hoping, yearning for the cameras to lock-on to our physical surroundings.
Some devices registered the real world within a few minutes. Other poor souls were left doing neck exercises for the next 5-10 minutes.
If you were lucky enough to create your environment map, then the OS would finally launch. The OS interface looked like a holographic book shelf. Each shelf with floating orbs representing a variety of spatial apps.
But upon launch, exactly where this holographic shelf appeared in space was anyone's guess.
For some, it was down on the floor. For others, it was off in the distant horizon or behind them. The next 10 minutes we collectively embarked on a holographic treasure hunt at our desks; searching up, down, and all around for our 'app launcher'.
My holographic shelf was above me, anchored to the ceiling.
Now the primary way to interact with these holograms was with your hands. You had to reach out and grab them. But doing so was quite the art... it required your hand being in the perfect spot, at just the right proximity to the hologram. When you found that magic zone, a circle would appear.
Then, and only then, could you close your hand and 'grab' the hologram. The camera on the headset needed to see a very distinct gesture: a wide-open hand and then a distinctively closed fist. When the cameras saw this movement, the circle UI would become a dot, confirming the hologram was secured.
This led to yet another comical sight; an entire office of people, waving their hands in the air, trying to materialize that circle. Everyone was flailing about, groping the air and repeatedly trying to turn that circle into a dot. We became a horde of perverts molesting invisible objects of desire.
I stood up and reached longingly into the air for my holographic shelf, only to be immediately yanked back into my chair by the tether.
Screw it. I resorted to using the mouse we so vehemently vowed to replace. It was a fallback form of input, controlling a 'spatial cursor' that allowed me to click on the 3D shelf and pull it closer.
Finally, I could start pulling out little apps & experiences, placing them all around me at my desk. For a split second I was living in the future.
There were virtual monitors showcasing the future of productivity, with PowerPoint, web browsing, and spreadsheets. But I could barely read the text. It was blurry and the eye strain was very real. There was a beating heart for the future of education. There was a virtual jukebox to show case our (attempts at) spatial audio. There was a 3D model of a Tesla, hinting at the future of immersive design or e-commerce.
And my personal favorite... a box with an image of a butterfly. When you touched it, the box exploded into a cloud of 3D butterflies, fluttering vigorously this way and that. When you held out your hand, they would come land and gently rest.
For many, the mind would play tricks. You could feel the tickle of the butterfly's little legs on your hand...
This… this to me is the magic of spatial computing; mind merging with machine, tapping into the mystery of how the brain has naturally evolved to interact with and understand the real world.
Imagine the impact of this for communication, education, collaboration, and creation. We were passionately driven by this potential. We were mission obsessed, and mission bound.
But that moment in the future was short lived. After a few minutes, naseua set in from the motion-to-photon latency (aka: the time between head movement and the display’s output/reaction ). Then, the virtual shelf suddenly started to jitter and float away, carrying with it our collective hopes & dreams.
Alas, my headset lost world tracking entirely and holographic chaos ensued.
Before I knew it, holograms were flying all over the place. The virtual heart shot past my head, the virtual monitor turned upside down and shot through my desk, and the 3D shelf/OS UI zoomed right back up to its original home; the ceiling.
Next to me sat my sales colleague and dear friend, Connor McGill. We looked at each other, let out massive sighs, and just laughed. What else could we do?
We had spent the last 18 months traveling the world; from LA to NYC, Shanghai to London, Amsterdam to Rome giving thousands of demos and convincing the world’s largest companies that spatial computing was the future, and that it was imminent with the Meta 2: Nike and Adidas, Lockheed Martin and Boeing, Exxon and Shell, Ford and Tesla, Disney and Universal, Dell and Lenovo. The list goes on.
This was going to make for some awkward conversations.
Welp... at least we had good packaging.
Dell Technologies President, Jeff Clarke, celebrating the deal to become a Meta 2 reseller.
Meta 2 @ The Pantheon in Rome
Kate Middleton & Prince William
Even Bert wanted in on the action
When the Apple Vision Pro presentation ended, I was in awe. They seemed to have absolutely nailed it.
The display quality— near perfection; making it seem like you’re viewing the world through a pane of glass. The pixel density— mind blowing; making text perfectly legible, at last. The innovation with the R1 chip— a sci-fi feat; processing data from 12 cameras to produce zero perceived latency and making nausea a thing of the past. The world tracking— immediate and flawless, anchoring holograms perfectly and elevating them to first class citizens in the real world. The input/interaction- pure magic, creating the illusion of mind conrol with the perfect tandem of eye and hands tracking.
The list goes on... they seemed to think of every little detail, and thoughtfully addressed the majority of paper cuts that have plagued AR/VR for decades.
When I left the viewing party that day, I half-expected there to be a ‘spatial computing parade’ in the streets.
The tech we’ve all been waiting for was finally here! A cause worth celebrating, for sure. Heck, I was ready to take the day off and paint the town red! (And that is exactly what a few of us did…)
Spatial Squad
But when I integrated back into the real world the next day, my enthusiasm wasn’t quite the norm.
The first friend I talked to about the announcement said “it made me want to put my feet in the grass and hide in the woods”.
Okay, considering some of the concept videos, I get it… (we’ll address those later).
And then there were the mainstream media pundits, spewing all kinds of nonsense: ‘Apple has lost its way’, ‘this product will never sell’, ‘no one needs this’, etc. etc.
As the weeks went by, the criticism kept pouring in…
It's too expensive!
What is this good for?
People won’t wear something on their face
It's too isolating
The digital eyes are creepy
Only a two-hour battery life?!
When I first heard these critiques, my blood boiled. I couldn’t help but think… What the hell is wrong with these people? How can they not see what I see? Do they not get the magnitude of these technical feats and this product’s potential impact?
With my emotions at the helm, I realized I needed to take a step back, think objectively, and question my beliefs… turns out inherent career bias is a helluva a drug.
Why was I so triggered? Am I the crazy one here? Or is everyone else missing it, and my bullishness is indeed warranted?
Over the last month, I’ve done the inner work to remove my XR fanboy cap and think more deeply about Apple’s strategy, along with the importance of this moment.
With my bias officially on the shelf, I remain convinced-- the pundits are wildly wrong, consumers don’t know what they don’t know, and this moment truly does matter, indeed.
And turns out, I’m not alone… while I’ve not tried the device myself, I have listened & talked to those who have. Upon hearing their feedback, I feel (a bit) less crazy. My initial perceptions & instincts seem to hold true.
The best part? Their favorite moment of the Vision Pro demo is a holographic butterfly that lands on your hand. They too could feel the tickle of its legs…
(A coincidence? I think not… Apple hired many of Meta’s top engineers upon failure, including the ones who built that original ‘office rehearsal’ demo)
Now, before exploring the impact of this moment, why Apple’s strategy is the right one, and why you should care… Come with me back in time once more, to a moment chalk full of lessons & predictions of what’s to come.
General Magic
They say history doesn't repeat, but it certainly rhymes.
The Meta journey is a reflection of a similar story, with a similar outcome for an eerily similar company: General Magic.
If you like documentaries, this doc about General Magic is a must watch, even if tech & business is not your thing. It's just a compelling story; of ambition and courage; of how to blaze new trails; and of how to cope with heartbreak and shattered dreams.
If you're unfamiliar: General Magic attempted to create the iPhone back in 1989/1990. The vision and the use cases were exactly the same: a personal computer in your pocket acting as a remote control for your life.
General Magic Design
The team was perfect, the vision was prophetic, and much of the technology existed. But the timing was wrong and the tech couldn’t yet merge to make the whole greater than the sum of its parts.
While the individual pieces were there, they weren’t mature enough to yield a compelling user experience. A lot of technology still needed to be invented, and there were numerous UI/UX rough edges to be smoothed over. Very similar to the Meta 2 headset.
There also wasn’t a fertile ecosystem. The internet wasn't ubiquitous, telco connectivity wasn't mature, and 'mobile developers' didn't really exist. There were very few builders and businesses with properly primed imaginations or business models.
Perhaps most important... consumer behavior wasn't properly evolved. They didn't see the point. The use cases didn't quite click and people weren't quite sure what this thing was good for. The whole thing just seemed… silly.
Sadly, the General Magic dream came to an end in 2002.
Fast to June 29th, 2007 (five years after General Magic shuts down). Apple launches the iPhone and changes the world.
Apple was watching, studying, and learning from General Magic all along. They even hired some of their best/most talented employees (e.g. Tony Fadell). They had blueprints and prototypes all along the way. But it took 17 years to get the technology just right; polishing, testing, debating.
And boy, did they nail it.
In hindsight, it's easy to say the iPhone's future impact was obvious when it launched.
But was it?
Sure, it launched with some killer apps: calls, email/messaging, web browsing, and music. But these things weren't entirely new. It was things we were already doing, just better on multiple vectors. Very few people, if any, saw the app store coming and all the innovation that would follow…
Fast forward to 2023 (also five years after Meta closed its doors), and Apple uses the exact same playbook.
For 10-15 years… They were watching, learning, iterating, polishing, debating, and polishing some more.
Apple then launches the Vision Pro and change… well, depends who you ask. To most, its impact is far from obvious.
And so, the stage is set for perhaps a similar story, albeit with obvious differences. This is a bit more dramatic than going from a flip phone to a touch screen.
We’re now breaking through the screen and entering the machine. Of course, the tradeoffs, roll out plan, and adoption cycle must be radically different.
In Part II, we’ll explore those differences and analyze Apple’s strategy, diving deeper into the tradeoffs, why the skeptics are wrong, and how this adoption curve might unfold over the next 3-5 years.
Until then…
Thanks for taking the time to read Evan's essay. Let us know what you think about this perspective, and if you want to check out some more of Evan’s writing, here are some of our personal favorites:
