How to add spatial tracking to 3D objects and deliver attention data insights

Coffee, like spatial computing, is a universal passion for most XR developers so let’s look at three coffee machines as example 3D objects and apply spatial tracking to deliver attention insights. The objects could be product demos (retail, automotive etc), virtual try-ons (fashion, furniture etc), digital twins (manufacturing, construction etc) or training, advertising and game experiences. For these coffee machines, we want to know which one performs best, how we can optimise the user experience and what actions will drive business objectives. Spatial tracking and the AR/VR attention data it generates can be applied to all 3D objects whether it’s individual AR models or multiple objects in a VR scene. So, now we have your attention, let’s jump in, show you how it works and the type of attention data insights you get.

Add CORTEXR spatial tracking plug-in to 3D engine

Our plug-in for 3D engines (Unity, A-frame etc) adds spatial tracking on 3D objects without needing code or data analytics experience. Setting up spatial tracking and configuring 3D objects takes less than 10 minutes for developers with intermediate experience of your preferred XR platform. The plug-in places a cube around the 3D object which, in this case, is scaled according to the dimensions of the coffee machine with each face of the object generating attention data based on where users looked. Multiple 3D objects can be tracked in a scene (e.g. kitchen environment with various objects) but today we’re focused on individual 3D models and the insights our XR data analytics platform generates. Attention data outputs are similar to eye tracking, but infinitely more scalable, with spatial tracking revealing deeper insights about spatial behaviour in spatial environments compared to standard web data analytics tools like Google and Unity. 

Identify best performing object with highest attention levels

Almost 20% more attention is given to the red coffee machine so people found this 3D object more appealing as it held their attention more than the yellow and white models. Attention Total is the amount of attention users give to the 3D model with data split by each face visible to users (front, right, back, left, top) and we’ve added the brand logo as it’s important to this project’s objectives. Attention Distribution shows the percentage split to analyse attention for each face and how this differs for each 3D object. This extra data insight layer of spatial tracking on 3D objects on top of standard data analytics (dwell time, interactions etc) gives you a complete picture on user behaviour to improve the performance of your AR/VR project, whether it’s pre-testing, prototyping, A/B testing, live monitoring or post analysis.

Insights and actions:

Identify red as winning object so placement/position early in the experience will increase overall user engagement and e-commerce ‘buy now’ funnel metrics.

Optimise attention across all sides of 3D objects to improve interactions with areas (like the right side) which get attention but have no user actions or events.

Analyse successful elements and dial up these features e.g. increase performance of red model by adding coffee cups on the top as this works well on the yellow model.

Analyse user behaviour around 3D objects with attention heatmaps

Over 70% of user attention (on average) is focused on the front face of the 3D objects as people spend time understanding functional controls and ergonomic features of the coffee machines. Our Attention Areas heatmap shows spatial tracking on 3D objects in an unboxed cube to examine attention on each face of the 3D model and – importantly – if user attention drifts outside the 3D object. Heatmaps in our prebuilt dashboards are more than pretty pictures as hovering over each cell shows data values to let you dig into specific elements, export data and analyse the pattern of user attention and its impact on project objectives.

Insights and actions:

Identify high visual attention areas on each face of the object (e.g. functional controls on front) to ensure UX design and UI interactions meet objectives.

Optimise low attention areas with nascent appeal (e.g. coffee cups and water tank) to drive exploration around 3D objects to increase dwell time and engagement.

Analyse hierarchy of attention against commercial goals (e.g. brand logo receives only 5% of attention) to improve brand recall, call-to-action buttons etc.

Optimise order of events with Attention Priority maps

User attention is mainly on the front of these objects as they see the front first and aren’t really motivated to explore all sides of the object. This could be an issue if important information is being missed on the right/back/left/top faces but it’s a successful result for this project as aesthetics on a kitchen counter and ergonomics of functional elements are the main business goals. Attention Priority sequence maps show the order of user attention around 3D objects to reveal additional insights (white is first view whilst purple is last view). Here we dig into the results on the front face to understand what’s going on. First view is often most important but last view is sometimes more interesting as it adds another layer of diagnosis to improve project performance.

Insights and actions:

Analyse intuitive user behaviour against intended content experience (e.g. control panel ranks first as ergonomics is primary objective) so this is a successful result.

Optimise narrative and user flow around objects (e.g. users moved around to the right because nozzle position on the right led their attention in that direction).

Identify unexpected orders of attention (e.g. logo gets attention early in the experience despite having low overall attention) so brand validation is important.

The metrics of human behaviour with spatial data analytics

Analysing human behaviour with 3D spatial content doesn’t always follow the UX/UI design when an AR/VR experience is released into the wild. User behaviour in 6DOF spatial environments can be unpredictable and surprising so spatial tracking on 3D objects help you build, test, monitor, analyse and grow your XR projects. Are people viewing the areas you expect them to, which areas do they look at first (versus last) and does this support your business objectives? CORTEXR data analytics platform gives you the answers so – combined with standard data analytics (Google, Unity etc) – you have the full picture to grow your business.

Find out more 

Schedule a demo 

Banuba Helped sMedio Deliver a Video Calling App for a Fortune 500 Company

A major software development company from Japan sMedio integrated Banuba AR Face Filters SDK in the video conferencing app it made for a major electronics manufacturer. By the end of 2023, this app will have been installed on 500,000 devices, intended for business use.

sMedio took charge of developing the core functionality, while Banuba contributed their cutting-edge technologies for face touch-up and background replacement. These essential features are considered indispensable in a contemporary video calling application:

1. Professional Appearance: Virtual backgrounds play a vital role in avoiding awkward situations such as interruptions by children or pets during a video call. With this feature, only the person in front of the camera remains visible, while everything else can be replaced with static pictures, GIFs, videos, or even immersive 3D environments.

1. Alleviating Anxiety: Many individuals feel self-conscious about their appearance during video calls. Thanks to the discreet face touch-up feature, users can effortlessly remove skin imperfections, brighten their teeth, correct camera distortions, and enhance their overall appearance, helping them look their best during the call.

Creating these features from the ground up would demand specialized expertise and entail additional time, substantially inflating the final product's expenses. Opting to use the Banuba Face AR SDK proved to be a more cost-effective choice.

"Banuba provided high-quality products, proven technology, and solid support, leading us to make our OEM customer’s products competitive in a limited time and cost," Sadanori Iwamoto, CEO of sMedio, Inc. said.

About sMedio

sMedio is a Japanese company that specializes in software for consumer electronics and portable devices. Its main focus lies in multimedia and wireless connectivity technologies. sMedio is a respected player in the market, having secured contracts with several Fortune 500 companies.

About Banuba

Banuba is an augmented reality company with over 7 years on the market, pioneering face tracking, virtual try-on, and virtual background technologies. Besides online try-on solutions for makeup, headwear, jewelry, glasses, and more, it offers a Face filters SDK and Video Editor SDK – ready-made modules to apply effects and edit videos.

Sony investing $2B into XR (metaverse) research and development

Get to see the latest from Sony at our Immerse Global Summit at Metacenter Global Week in Orlando.

Sony, a member of the VR/AR Association (VRARA) dives into 'extended reality' with $2bn R&D war chest.

(Back in 2022, Sony already announced plans to expand into the Metaverse to boost its presence, by using its knowledge of gaming and music to build the Metaverse, allowing people to socialise, collaborate, and join gaming communities, among others. Also, Sony has invested roughly $1 B USD in real-time 3D software firm Epic Games Inc in a bid to develop its own Metaverse immersive worlds)

July 13, 2023 TOKYO -- Sony Group will ramp up R&D spending in its game business, as it seeks to hone technology in live service games and the emerging field of "extended reality" that blends physical and digital worlds. 

The company will pour 300 billion yen (about $2.13 billion) into research and development for its game segment for the fiscal year ending in March 2024. The amount, which will account for around 40% of its total R&D spending, now surpasses spending in electronics and semiconductors.

The figure is more than the 271.1 billion yen spent on game R&D last fiscal year, and more than double the 144.5 billion yen in fiscal 2020. Sony Group plans to spend 760 billion yen in company-wide R&D this fiscal year, up 3% on the year. 

Games are a major pillar driving Sony's business. Earnings before interest, taxes, depreciation and amortization -- or EBITDA -- for the company's game business was about 337 billion yen last fiscal year, up more than 60% from five years ago.

Sony's pivot from a focus on hardware to a subscription-based business model is spurring the company to step up R&D. Having long relied on sales of its PlayStation game console, the company will now focus on live service games that let customers purchase add-ons for titles streamed online.  

Sony projects that the market for add-on style games will reach $19 billion in 2026, a figure that would surpass the market for hardware like the PlayStation for the first time.

Yet Sony itself has yet to make a splash in live service games, relying mostly on sequels to traditional games for software sales. Meanwhile, Microsoft -- which already has a leg up in cloud-based streaming games -- is moving forward with a bid to acquire game publishing giant Activision Blizzard.

To chart a better growth strategy in the field, Sony in 2022 acquired major U.S. game studio Bungie for $3.7 billion, aiming to use its live game know-how to boost development.

Sony plans to have 12 live service game titles in its portfolio by the fiscal year ending March 2026, up from just one in fiscal 2021. 

It plans to allocate 55% of PlayStation 5 game development spending to live service games in the year ending March 2024, and 60% for the year ending March 2026.

The company also has its eyes on the metaverse.

Extended reality -- technologies that combine the physical and digital worlds -- is inching closer to the mainstream, with Apple's Vision Pro headset scheduled to hit the market next year. The technology promises immersive gaming experiences involving all five senses.

Sony aims to combine the resources of the nine overseas game studios it has acquired or invested in during the last two years to aid extended reality development. It has invested in U.S.-based Epic Games, whose "Unreal Engine" is one of the industry's leading computer graphics game engines.

"We will continue to make strategic investments going forward, prioritizing intellectual property," said Sony Group president Hiroki Totoki.

The company is also looking to bring the technology it cultivates in its game business into other fields, such as in-vehicle content for the electric car it is developing with Honda.

Sony's pivot comes amid a wider R&D push by other Japanese electronics companies, particularly in growth fields. The total R&D expenses of Japan's top eight electronics companies for the year ending March 2024 will increase 5% on the year to 2.28 trillion yen, reaching a record high for the third consecutive year.

Hitachi will increase R&D expenses in the environmental field by 27%, and Mitsubishi Electric will boost power-saving semiconductor development.

Source

Roblox is coming to Meta Quest VR headsets! A dream come true for many millions of users

Meet us at Metacenter Global Week in Orlando on Oct 17-19!

Today, more than 66M users join Roblox every day across a wide range of mobile devices (iOS and Android), desktops, and Xbox. Roblox, one of the largest 3D immersive platforms, will soon be available on Meta Quest, the leading VR headset.

This opens up another opportunity for developers to create and share their experiences with millions of people instantly on Meta Quest. Roblox developers will be able to bring their existing experiences to Meta Quest or create unique new experiences with VR in mind.

Traditionally, developing a VR experience required a team that had VR-specific experience and skills. With Roblox, developers will soon be able to publish their existing experiences to support VR with little to no additional coding effort required. This is because Roblox is designed to be universal, allowing the same experiences to run everywhere.

Meta says the open beta provides an opportunity for the Roblox developer community to optimize their existing games for Quest and build new ones for VR while receiving feedback from the Quest community. Developers will be able to experiment and iterate on VR experiences before Roblox’s full release on the Meta Quest Store.

Roblox is automatically publishing some experiences that use default player scripts to support VR devices,” Meta writes. “They’ve found that those experiences typically run well in VR without modifications, so they’re seeding the Roblox VR library with great content from day one.” There should be a lot of experiences to choose from; the platform has more than 15 million “active experiences” right now, Roblox says. And this Quest version of Roblox will support cross-platform play, so you’ll be able to jump into experiences with players on Xbox, mobile, and desktop.

Virtual Tour Company Outer Realm Wins Award from SiliconIndia Magazine!

Virtual Real estate is growing and growing, and Outer Realm is leading the way and is honored to be named SiliconIndia’s 2023 Most Promising Technology Company Founded or Managed by Indians in U.S.! They stand out from other virtual tour companies by tailoring our approach to each customer’s needs and working closely with industry professionals such as architects, builders, and interior designers.

What is Outer Realm?
Virtual tour company Outer Realm aims to help you sell your property faster. In fact, we can sell your property before it’s even built! Our services include:

VR tours
3D tours
Flythrough videos
2D images
Interactive walkthroughs.
Each of these services helps showcase the property on a different platform, such as through email campaigns or on social media. This “all-in-one package” lets you create a variety of content with the help of only one company.

The Challenges of Selling an Unbuilt Property
The biggest challenge that Outer Realm faces is selling an unbuilt property. Since potential buyers cannot yet visit the property, sales are entirely dependent on VR showings. Outer Realm turns this setback into a superpower by using VR to show features that can’t be seen with an in-person walkthrough. Our interactive tours allow people to zoom in on every detail of the property and to view labeled information throughout the tour. Whether you are trying to sell a house to potential homeowners or show a commercial property to stakeholders, we will help you create the right tour for your clients.

How Do We Do It?
How do we make an accurate tour of a property that doesn’t yet exist? Dhara Patel, CEO of Outer Realm, tells SiliconIndia the secret:

“We follow a customized approach tailored to each client’s specific needs. We provide as many revisions as necessary and ensure a fast turnaround based on the client’s requirements.”

By working with architects and designers, our tours become exactly what our clients envision. We create everything according to client requests, from furniture placement to lighting. Additionally, Patel personally involves herself in every project, which ensures that no communication goes unheard. Outer Realm combines the team’s VR technology expertise with our clients’ design expertise to create a tour that pleases everyone.

Empowering Women in the Industry
virtual-tour-company-stemSiliconIndia recognizes Outer Realm not only for their outstanding use of technology in virtual tours, but also for empowering women in the industry.

“This exemplary accomplishment exemplifies the transformative potential of perseverance and enthusiasm, while also upholding the empowerment of women within the industry,” the magazine reads.

Women only make up 23% of the STEM labor force in the United States. By starting her own technology company, Patel is leading the way for other women to follow in her footsteps and decrease the gap in women in technology. Outer Realm understands the importance of having a diverse team that uses everyone’s unique talents and skills to create an exemplary product.

Sell Your Property Earlier with the Help of Outer Realm!
As a trailblazer virtual tour company that is also leading the way for women and immigrants in the industry, Outer Realm can bring your vision to life. No matter what kind of property you’re selling or where you want to show it off, we can create the tour that will sell fast! Call us at (212) 655-4626 or contact us online to start creating your tour today!

More info: http://www.outerrealmvr.com

Source

VR/AR Association annual Retail Forum will be held on Aug 31. See industry highlights!

Join our annual Retail Forum on Aug 31 as we bring together the best minds in VR/AR, shopping, and the metaverse. We expect 1000+ attendees, 20+ speakers from Walmart, Meta, Google, Microsoft, Bridgestone, Marxent, Lenovo, VR Direct, Magic Leap, Sony, among others.

Our event will be 4hrs long. In addition to keynotes, rapid presentations, sessions, demos, expert panels, we will have the best-in-class 1-on-1 online networking where you can build meaningful connections! ​See highlights from our forum last year here​.

Some interesting highlights about the industry:

• More than 90% of Americans currently use, or would consider using augmented reality (AR) for shopping. This is an increase from 66% of people who expressed interest in using AR to make purchasing decisions in 2019 (Google research).

• Shopify's research shows a 94% higher conversion rate for products with AR content than products without AR.

• 69% of online shoppers in the US interested in using AR scanning app to try on apparel and footwear purchases. 62% of shoppers revealed their willingness to use smartphone scanning to determine the correct sizes for their apparel and footwear purchases Harris Poll/Footwear News (March 2023).

• 300% increase in conversion rates when shoppers utilize the “View in Room” AR feature while shopping for furniture (Marxent)

• Increased product engagement: 5 times the number of products viewed when AR is deployed on PDP. This resulted in a 25% add-to-cart increase (Jerome’s Furniture WebAR)

• Increased cart conversion: Shopify consumers of a retail brand were “65% more likely to place an order after interacting with a product in AR.” (Rebecca Minkoff)

• Personalized user-generated content: The modern consumer loves personalization, and 76% of consumers prefer brands that showcase UGC. (ChargeAR.com)

• Increased loyalty AR creates positive brand impressions, builds consumer confidence, and creates a trusted loyalty loop. That, in turn, reduces returns (up to a 22% decrease from Build.com) while keeping the brand top of mind. (Warby Parker Virtual Try On)

• 83% of consumer goods executives surveyed expect their company will have a stake in the Metaverse within the next 2 years. (Deloitte)

• Metaverse consumerism has the potential to drive $5 trillion in value creation by 2030 (McKinsey & Company, 2023).

Join us on Aug 31 to learn and connect!

Thank you Sponsors:

Also, join our ​Immerse Global Summit​ at ​Metacenter Global Week​ in Orlando on Oct 17-19. We expect 3000-5000 people and have Amazon AWS, Meta, Bridgestone, many others confirmed!

VRARA Retailer's Playbook for the Metaverse: e-commerce in Virtual Space using Web3, VR/AR, NFT (download)

Download publication

VRARA Retail Metaverse Playbook

(and participate in our annual Retail Forum 2023 on Aug 31!)

VR/AR Association White Paper

Retailer's Playbook for the Metaverse

e-commerce in Virtual Space using Web3, VR/AR, NFT

By the VRARA Retail Committee

Many retailers tell us that the metaverse feels more futuristic than practical, yet there are retailers already using metaverse technologies and early iterations of virtual worlds, like gaming platforms. We intend this guide to explain present-day retail possibilities, plus where we may be heading in the coming years. This is a companion guide to our previous ebook, The Retailer’s Playbook for AR and VR in 2023.

Foreword: Harnessing the Metaverse for Retail

Beck Besecker, Founder & CEO, 3D Cloud by Marxent

Sarah Ennis, Intergalactic Agency Inc

Table of contents

Harnessing the Metaverse for retail

Introduction

Why we made this guide

Who should read this guide?

Understanding what the metaverse is and could become

Benefits of the metaverse for retailers

8 major benefits from using metaverse for e-commerce and retail brands Web3 placemaking best practices

How shopping works with Web3 and the metaverse

Web3 placemaking best practices

The role of NFTs in metaverse retail

Example: NFT as a tokenized loyalty program

More examples of NFT Campaigns

Tips to discover your metaverse possibilities

How to start a metaverse project

Final thoughts

Calling all AR/VR content creators: take the new State of the AR/VR Survey (and win up to $1,000 USD in cash, Unity subscription plan, and more!)

New State of AR/VR survey is live now!

Our mission is simple - help the world understand what AR and VR creators need to take their creations to the next level.

-What tools are you using to build your reality-bending apps and games?

-Which technologies work best for you?

Answer the survey questions, shape the ecosystem and win cool prizes!

Here’s what we have prepared for you:

• NEW virtual goody bag: Everyone who completes the survey gets instant access to resources, data, discounts and more surprises. It’s free and it’s yours to help you plan your next move.

• Premium prize draws every week: Take the survey for a chance to wi

• Cash rewards if you spread the word: Join our referral program to share the survey with other creators and win up to $1,000 USD in cash.

• Make an impact: Every participation counts! We will donate up to $1,700 to the donation program of your choice.

Don’t miss out on your chance to impact on the world of AR/VR tools!

From Gamification to the Metaverse: 7 New Ways to Engage Your Audiences

“Work hard, play hard,” the age-old adage goes. But what if you could do both at the same time? That’s precisely what businesses are doing in 2023. Recent years have forever changed the world of games and the role of game makers. Today, the video game industry is larger than ever before, and gaming talent is in demand across all industries.

From healthcare to hospitality and beyond, companies are leveraging the power of games to engage their audiences. Gamification. Motivational design. Edutainment. You might not have heard these words before, but the odds are you’ve experienced them in action. Each is defined by the act of applying gameplay elements to other activities, such as marketing, ecommerce or education.

Before anyone could ever experience virtual reality, gamification strategies were rising in popularity. Today, gaming expertise has gone from ‘rare niche’ to ‘common need.’ Here are seven ways the most future-focused brands are tapping into gaming to engage their audience:

1. Market within the metaverse.

There is no metaverse without game designers, developers, and engineers. This digital replica of our world is being built by many of the same minds behind our favorite AAA video games.

From avatar character artists to blockchain engineers, the demand for gaming talent has skyrocketed since the start of the race to create the best extended reality (XR) space. And the metaverse is evolving quicker than companies can hire.

Over the next few years, this revolutionary innovation is predicted to change the way our world works, and market leaders, including Apple, Nike and Coca Cola, are investing in it. According to Bloomberg Analysts, the metaverse will be worth nearly $800 billion by 2024’s end.

2. Immerse audiences in experiences.

The metaverse may be under construction, but immersive shopping experiences, virtual real estate tours, and XR training simulations are already here.

Blurring the lines between the physical and the digital, XR technologies aren’t just for metaverse and game development. From try-on experiences to social media filters, virtual reality (VR), augmented reality (AR), and mixed reality (MR) advancements have endless applications.

Wal-Mart’s Training Academy leverages VR innovations for employee training simulations. The Ford Motor Company’s designers use MR headsets to visually render 3D car models. The Ikea Place App utilizes AR technology to allow users to visualize Ikea furniture in their own homes. And the video game industry’s finest software and hardware experts make it all possible.

3. Make Web3 your secret weapon.

Today’s gamers don’t just play to win; they play to earn. Play-to-earn games allow players to buy, sell, and trade in-game items, using blockchain technologies like crypto-currencies and non-fungible tokens (NFTs).

Blockchain is a system for recording information that makes it difficult to change or hack the data. Game studios adopted the technology early, but it’s good for far more than games alone. The system is at the center of the metaverse and the internet of the future, Web3 or Web 3.0.

Companies like Microsoft, IBM and JP Morgan Chase leverage the technology for digital identity verification, secure data sharing, supply chain management and more. Brands everywhere are exploring possibilities, and blockchain engineers are now among the most in-demand gaming talent within and outside of the video game industry.

4. Turn buying into playing.

Rewards programs are nothing new, but gameful thinking has changed the game. By gamifying customer loyalty programs and promotions, businesses turn purchasing into playing.

The details of every game-like deal are different, but often, the more customers buy, the more they win or the better their chance. Yet, purchases aren’t the only way to earn the badges, stars, points, awards and the free products that so many customers enjoy today.

Programs like Starbucks Rewards include bonus challenges and games that take brand engagement to higher heights, and Starbucks is just one example of many. Spin-to-win games, social media challenges, and iconic competitions like McDonald’s Monopoly have paved the way for a new age of advertising and a renewed need for gaming expertise.

5. Make games instead of ads.

Gone are the days of passive advertising and billboards. In 2023, experience is king, interactive marketing is everywhere, and branded games or ‘advergames’ are winning customers over.

Experiential marketing installations like Pepsi’s viral interactive soda machine have boosted brand awareness for years, but gaming experts have taken experiences to another level. ‘Advergaming’ isn’t gamifying existing customer activities; it’s building entirely new games for the sole purpose of brand promotion.

Lego mobile apps and KFC’s virtual Bucket Hunt aren’t just gamification; they’re legit games, and they’re driving real results for top brands. 60% of consumers are more likely to buy from a brand if they’ve enjoyed playing a game it created.

6. Design products people can play.

Customers aren’t the only ones you can engage through gamification. Gameful thinking can make learning fun and transform performing into winning.

Innovative educators are re-evaluating how teaching is done, and the benefits of their models extend beyond the school system and into the workplace. Brands like EY, Salesforce and IBM are putting gameful approaches to onboarding, team training and employee incentives into action.

New apps and platforms emerge each year to help employees expand their skill sets, track their progress and earn rewards for achieving their goals. According to numerous studies, the results are increased employee satisfaction, engagement, and performance.

7. Gamify work to incentivize wins.

Some games aren’t for marketing, education or entertainment. Businesses everywhere are incorporating game mechanics into products and services that you’d never imagine.

Wearable fitness trackers. Financial savings apps. Smart home devices that gamify household tasks. Imagine a dishwasher that turns loading and unloading dishes into a game that awards you points for efficiency. By introducing rewards, levels and challenges, these products turn everyday tasks into engaging experiences.

Brands like FitBit or DishMaster certainly aren’t video game studios, but their products are games in their own respect. And not unlike AAA video games, gameful products are the work of game designers.

Today’s game design is tomorrow’s design.

The meaning of ‘games’ has changed; gameplay is no longer reserved for entertainment alone, and the need for gaming talent has grown far beyond the gaming industry.

As the metaverse evolves, we can expect to see the demand for gaming experts continue to expand. Whether it’s through interactive promotions, immersive experiences, or innovative products, games have an undeniable ability to engage and motivate audiences in new ways.

Today, most businesses have only begun to scratch the surface of the possibilities. Tomorrow, people will no longer need to decide between ‘work’ and ‘play’ because the two will be viewed as the same. Until then, ask yourself: What could game makers do for you?

About Onward Play:

Onward Play connects companies and game studios with the talent to take gaming to the next level, push the limits of web3 technologies, and bring immersive experiences to life.  

Onward Play is a division of Onward Search, a specialized workforce solutions company that empowers creative excellence, brand engagement and digital innovation.

Contact:

Kim Shatzer
Senior Vice President, Delivery
Kim@OnwardPlay.com

The Role of Frontier Technologies in the Future of AR

Alexander Belugin, Chief Product Officer, at XPANCEO (builds a smart contract lens with seamless AR experience) delves into barriers for AR headsets  to become mainstream devices and how technology and science those limitations.

The Role of Frontier Technologies in the Future of AR

There was a remarkable interview with Display expert Karl Guttag about technical hurdles in developing AR-glasses suitable for the masses.  

We'd like to delve further into this topic and examine it from a different perspective. Suppose AR glasses are merely an intermediate product. How might advancements in engineering and science  aid in the miniaturization of AR devices? Such progress could potentially transform these devices into everyday gadgets, serving as indispensable assistants much like today's smartphones

In this article we will examine the several  major technological  barriers of AR glasses and explore possible pathways for solutions.

Display Technology:

One of the main challenges is the development of a compact display that can superimpose digital images onto the real world without reducing the quality of what the user is seeing. Current AR technologies often struggle to find a balance between creating high-resolution images and keeping the device small and lightweight. Another considerable challenge is a limited FOV. 

Solution

Many of these issues could be resolved simply by changing the form factor. For AR contact lenses, field of view (FOV) is not a problem. In fact, they can always provide a 100% FOV, as the display is located on the central axis of the eye and follows it wherever we look. 

In large headsets, a see-through system based on cameras is implemented, which in itself is bulky, or regular glasses cannot handle bright light, resulting in issues with brightness and contrast. On smart lenses, the image is positioned "in front of" the external light and, with a touch of polarization effect that protects an image from discoloring.  

If the screen is placed on a contact lens, a high pixel density is needed only in the very center, in a zone about 2x2 degrees, because our eyes distinguish details only in this area. Traditional headsets attempt to leverage this aspect of our vision by reducing the rendering quality in areas of the screen where the user isn't looking. In the case of a contact lens, there simply won't be any superfluous pixels, which will also decrease power consumption and computational load. 

Interestingly, it's possible to eliminate approximately 90% of pixels, given how poorly our eyes distinguish details and shades outside the main field of view.  Hiding  90% of pixels will drive significant improvements in data usage and energy consumption.  

Another important aspect is the waveguide. Improvements in waveguide technologies can significantly enhance the display and overall performance of AR devices, playing a critical role in form factor, miniaturization, power efficiency, and battery life. Frontier physics has made it possible to create waveguides from graphene-like 2D and low-dimensional materials. This advancement allows for the construction of a waveguide that is up to 100 times smaller than traditional waveguides (1.7mm) used in current AR devices

Power Consumption & Battery Life: 

AR glasses require a significant amount of power to run, particularly if they include features such as 3D tracking, gesture recognition, or complex visual overlays. Reducing the size of smart glasses will make it even harder to equip it with a suitable battery.

Solution

It is possible to reduce power consumption by a factor of 100 by positioning the screen directly in front of the user's eye. This cuts the pixel brightness requirements several times over, and it's only necessary to maintain pixel density at the center, as mentioned earlier. The entire smart lens consumes less than 0.1 Watt in active mode, while traditional glasses consume between 3 to 10 Watts.

The combination of low power consumption and the need for fast connectivity brings us closer to fulfilling the dream of creating a device that is entirely powered by 5G and 6G networks. Research projects already exist that conceptualize a wireless power grid running on 5G's mm-wave frequencies. 

Another extremely prospective solution is equipping an AR- device with a tiny solar battery made from a new class of materials – perovskites. Perovskites offer significant potential for solar energy harvesting and outperform silicon in terms of production cost. Moreover, perovskite solar cells can be made flexible, which opens up a multitude of possibilities for their use. This includes incorporation into wearable technology or deployment on uneven surfaces.

Connectivity

Let me begin with latency, as it's a critical aspect of all AR and VR systems. Since the human visual system is extremely sensitive to lag, 5 milliseconds of latency or less is required in optical see-through  systems in order to remain unnoticed by the user.

Achieving 2-5 millisecond lag for a seamless AR experience is extremely challenging. To transmit two images with 4K resolution, a stable and wide communication channel is necessary. Traditional connectivity protocols like Bluetooth, Wi-Fi, and UWB, as well as their appropriate chipsets, are not suitable for further miniaturization of glasses. This is because these protocols were not designed for low lag, small data sizes, and ultra-low power consumption. However, using these protocols can result in lag and other issues that diminish the AR experience. On the other hand, standalone AR glasses require more power and larger processors, which makes miniaturization more challenging.

Solution

To address the challenge of miniaturization, reducing power consumption, and achieving a lag of 2-5 milliseconds required for augmented reality operation, the industry will need to adopt a new protocol. 

Another challenge is the size of the antenna, which is quite significant in size. A potential solution is to develop transparent antennas made of atomically thin gold or copper, with sizes around 5-20 nanometers. These materials maintain high conductivity, while the antenna becomes invisible to the human eye. Additionally, the reduced pixel count allows for a 10-fold reduction in data transmission channel bandwidth requirements. 

In a nutshell, we observe how the progress of AR devices simultaneously depends on and drives the most advanced engineering and scientific directions. Yes, it will be a long journey, but that makes it all the more interesting. Right before our eyes, a completely new class of devices is emerging, which will change the usual ways we live and work. It's wonderful to witness this development

If you are interested in learning more about breakthrough technologies that drive XR industry, please contact: dragon@xpanceo.com

EducationXR is an innovative platform that combines VR/AR technologies with educational content

EducationXR is an innovative platform that combines virtual reality (VR) and augmented reality (AR) technologies with educational content. It aims to enhance the learning experience by providing students with immersive and interactive educational environments.

Key features of EducationXR include:

• Virtual field trips: Students can explore different places and environments from the comfort of their classrooms, allowing for a more engaging learning experience.

• Interactive simulations: EducationXR offers simulations that allow students to practice real-life scenarios, improving their understanding and problem-solving skills.

• 3D models and visualizations: Complex concepts can be visualized in three dimensions, making them easier to understand and grasp.

• Collaborative learning: The platform facilitates collaboration among students, allowing them to learn from each other and work together in shared virtual spaces.

EducationXR aims to make learning more exciting and accessible, and its immersive technology has the potential to revolutionize education by offering new ways of engaging with educational content.

For more information, you can visit educationxr.com or email cory@heizenrader.com

Uvisan offers UV-C disinfection services for VR/AR equipment and rooms

Uvisan uvisan.com offers UV-C disinfection services for rooms and equipment, using the power of UV-C light to effectively kill 99.99% of all viruses and bacteria.

No chemicals, heat, or liquid are needed for this process, making it a safer option for protecting customers, staff, and stakeholders. Uvisan offers a range of UV-C cabinets for versatile and rapid disinfection in just two minutes, as well as a Cleanroom solution that enables rapid surface and air disinfection in just 10 minutes.

For more info, email david.cartwright@uvisan.com

VRARA Retailer’s Playbook for AR and VR in 2023 (download)

Download publication

VRARA Retail Retailers Playbook 2023

(and participate in our annual Retail Forum 2023 on Aug 31!)

VR/AR Association White Paper

The Retailer’s Playbook for AR and VR in 2023

Stores, e-commerce, Web3, AI/ML, and beyond

By the VRARA Retail Committee

VRARA Retail Co-Chairs

Sonia Schechter, 3D Cloud by Marxent

Sarah Ennis, Intergalactic Agency

VRARA Retail Working Group Contributors

Ben Feld, Niantic

Lucky Gobindram, CemtrexXR

Vanessa Mullin, Agora

Aditya Mani, YOLOgram Style

Hassaan Iftikhar, Aequilibrium

Navjeet Chhina, Genius Ventures Inc

Jessi Sparks, Magnetic Mobile

Delz Erinle, Astraverse

Michael Potts, Polycount

Rachel Fischbein, Fashion Incubator San Francisco

Oktay Kesebi

Foreword: Unlocking the potential of AR, VR, and Web3 in retail

Beck Besecker, Founder & CEO, 3D Cloud by Marxent

It used to be relatively easy to test new retail tech concepts. Back in 1999, I worked for a datadriven couponing company. We conducted a simple experiment by having my team stand next to products in convenience stores and hand out coupons as shoppers passed by. Looking back, that experiment seems ultra-simple when compared to testing the current class of retail technologies.

But the good news is that testing new ideas doesn’t have to be complicated. The potential of today’s cutting-edge technologies—Augmented Reality (AR), Virtual Reality (VR), and Web3—is crystal clear, and there are accessible methods for quickly testing ideas. This is good news because consumers want to shop with these technologies.

According to a recent Harris Poll/Footwear News survey conducted in March 2023, 69 percent of online shoppers in the United States expressed their interest in using an FDA-cleared AR scanning app to try on apparel and footwear purchases.

Furthermore, approximately 62 percent of shoppers revealed their willingness to use smartphone scanning to determine the correct sizes for their apparel and footwear purchases. But the applications of AR continue beyond virtual try-on.

Our own data shows a staggering 300 percent increase in conversion rates when shoppers utilize the “View in Room” AR feature while shopping for furniture. Retailers across various categories are testing other applications, ranging from Snap lenses to in-store virtual mirrors.

While testing AR, VR, and Web3 may seem more complex than my in-store couponing test, they are well within the reach of retailers eager to evolve their strategies and embrace technologies that resonate with millennials and future generations. These emerging tools offer the ability to enhance the customer experience, drive operational efficiencies, and unlock previously unimaginable levels of engagement.

Retailers today have only begun to scratch the surface of what these transformative technologies can offer. They possess the power to redefine how businesses interact with customers, presenting exciting possibilities for innovation and growth. Now is the time to seize this opportunity, educate yourselves, build confidence, and test your ideas for utilizing AR and VR.

In this ebook, a group of early developers of retail technologies will serve as your guides on this journey. Our goal is to demystify these technologies, making them more accessible and easier to understand. We aim to empower retailers to explore and harness the immense potential of AR, VR, and Web3 in their own unique ways.

Drawing from our collective experiences and insights gained from numerous retail deployments, we have witnessed firsthand the immense impact these technologies can have. We have seen customers transported to virtual showrooms, engaging with products in ways previously only possible in dreams. We have witnessed shoppers trying on virtual garments, enabling them to visualize and personalize their style like never before. The power of AR in enhancing the in-store experience is undeniable, seamlessly blending the physical and digital worlds to create captivating environments that capture shoppers’ imagination.

Consider this ebook your trusted resource and compass as you embark on your exploration of AR, VR, and Web3. We will delve into the fundamentals, provide practical implementation advice, and showcase inspiring examples that illustrate the transformative power of these technologies. Our aim is to ignite your imagination and spark ideas for your own retail revolution.

It’s time to break free from the constraints of the past and embrace the boundless potential of the future. Together, let’s unlock the door to a new era of retail where AR, VR, and Web3 revolutionize the industry and make the shopping experience better for all.

Also, download our VRARA Retailer's Playbook for the Metaverse: e-commerce in Virtual Space using Web3, VR/AR, NFT

Participate in our annual Retail Forum 2023 on Aug 31!

How to Use VR for Healthcare, Education, Storytelling, New Workspaces

VRARA Member WarpVR shared with us how their customers use VR training and what impact they are getting from it. Follow the links below to see the newly published stories from CERONAV, OWNIT, and VR Bieb:

• How CERONAV improves maritime training with virtual reality

• How OWNIT. improves soft skills with immersive training on demand

• How VR Bieb improves healthcare education with immersive learning

Plutomen Technologies offers a platform for frontline operations & smartglasses

Plutomen Technologies pluto-men.com is a company that offers a frontline operations platform. This platform aims to improve the knowledge and capabilities of frontline workers by providing them with immediate assistance, work instructions, and training from top experts.

Plutomen's frontline operations platform allows for remote procedures, including communication, troubleshooting, inspection, commissioning, training, and reporting. By leveraging this platform, industrial enterprises can eliminate unplanned downtime, bridge the skill gap, and enhance workers' productivity.

The company also emphasizes the importance of digital connectivity and the potential of Industry 4.0 technologies. Plutomen offers solutions that can be easily integrated and deployed across service value chains, from industry 4.0 to customer service. These solutions can be used with industry-graded smart AR/VR wearables for seamless operations.

The goal of Plutomen is to empower frontline workers and make them job-ready. Their suite consists of three different solutions that aim to bridge the knowledge gap, especially for workers who are dispersed or lack access to proper training and support.

If you have any specific questions or require further information email keyur@pluto-men.com

Third Academy (a Web3 educational collective and career launchpad) joins VR/AR Association

Third Academy is a Web3 educational collective and career launchpad. It offers courses that are designed to onboard the next generation of professionals in Web3. These courses are open to all, as Third Academy believes that everyone's input and experience are needed to fix the mistakes of the past and build a new iteration of the Web. Their instructors are pioneers and early adopters in the field of Web3, who develop technologies, business models, and creative applications that are reshaping the future.

If you're interested in exploring the courses offered by Third Academy, you can visit their website at https://www.third.academy/courses2 or contact mike@third.academy

1249 people from 61 countries joined our Healthcare Forum 2023. See the top speakers and sessions.

Get ready for more healthcare innovations at our Immerse Global Summit in Orlando!

The VRARA Healthcare Forum Provided a Platform for Revolutionizing Healthcare Through Virtual and Augmented Reality

The VR/AR Association (VRARA) hosted it's annual Healthcare Forum, a virtual event comprised of health tech pioneers, industry experts, and healthcare professionals that are dedicated to advancing healthcare through the use of Virtual and Augmented Reality technologies, digital health, medical, and AI.

Our Forum 2023 had 45+ speakers and 1240+ attendees from Microsoft, Bloomberg, Magic Leap, Tel Aviv Medical Center, Merck, Thermo Fisher, Harvard, Edwards, MIT, NYU, CAE, Sony, Intel, Nebraska Medicine, Global Center for Health Security, NYC Health + Hospitals, Meta, Elsevier, Pfizer, National Institutes of Health (NIH), Mayo Clinic, SSM Health, PICO.

Top Speakers, Sessions included:

• VR/AR & AI Technology in Healthcare - With Super Powers come greater Responsibilities - Rafael Grossmann, Surgeon, MD

• How VR/AR and Generative AI will Impact the Future of Healthcare (panel) with Doug Hohulin, Harvey Castro Author of ChatGPT and Healthcare, and Denise Meade, Microsoft

• VR/AR in Healthcare: How to Scale Successful VR/AR Training Programs by David Burch, ArborXR

• TAMP Doctor Innovators (panel) with David Hall, CEO, TAMP and TAMP doctors from around the world

VRARA Healthcare Forum provided the healthcare industry with a unique opportunity to explore the benefits and applications of VR/AR technologies. These technologies offer innovative and immersive experiences that promote patient engagement, accelerate clinical trials, and enhance healthcare outcomes.

How VR and AR Technologies are Revolutionizing Healthcare

VR/AR technologies have brought about a myriad of changes in the healthcare industry. Some of the exciting ways the VR and AR technologies have revolutionized healthcare include:

1. Training Healthcare Professionals

VR and AR technologies offer a safe and immersive environment for training healthcare professionals. These technologies provide them the opportunity to hone their skills and practice managing complex medical scenarios in a realistic environment.

2. Enhanced Patient Experience

The immersive experience that VR provides can promote relaxation, reduce anxiety, and aid the rehabilitation of patients. AR technology can help provide patients with interactive and visual aids for better communication and understanding of their medical conditions.

3. Virtual Consultations and Checkups

VR and AR technologies are already being used for virtual consultations and checkups by healthcare providers. This technology enables physicians to conduct remote consultations and checkups with patients in different locations, thus reducing the time and cost associated with travel.

4. Improved Clinical Trials

Clinical trials are an integral part of developing new drugs and treatments for various medical conditions. VR and AR technologies have revolutionized this process by making it possible to conduct realistic and safe simulations using virtual patients, significantly reducing the time and costs of clinical trials.

The VRARA Healthcare Forum: A Platform for Revolutionizing Healthcare

The VRARA Healthcare Forum provided healthcare professionals, researchers, and industry experts with a platform to share their knowledge and insights on the use of VR and AR technologies in healthcare.

This forum brought together healthcare professionals, technology experts, and researchers from around the world to explore the vast potential of virtual and augmented reality technologies in healthcare.

By participating in the VRARA Healthcare Forum, individuals and organizations were able to:

• Connect with like-minded individuals and organizations committed to advancing healthcare through VR and AR technologies.

• Learn and share insights into the latest trends and developments in VR and AR technologies and how they apply to healthcare.

• Access informative and actionable resources that support the application of VR and AR technologies in healthcare.

In conclusion, the VRARA Healthcare Forum was an invaluable opportunity for healthcare professionals, researchers, and industry experts to explore the potential of VR and AR technologies in healthcare. The forum offered a platform for individuals to share their knowledge, connect with like-minded individuals and organizations, and advance healthcare.

Thank you to our forum Sponsors!

Get ready for more healthcare innovations at our Immerse Global Summit in Orlando!

Banuba and Agora Forge a Deeper Partnership

Banuba, a leading player in the augmented reality industry, is strengthening its collaboration with Agora, a global frontrunner in Real-Time Engagement Platform-as-a-Service (PaaS). The partnership is reaching new heights with the launch of updated extensions for iOS, Android, and Web platforms, enabling both companies to advance their cooperation. Agora will oversee the seamless integration of the plugins, enhancing convenience for their shared clients.

The offerings from Banuba and Agora naturally complement each other.

Synergistic features: Agora SDKs provide robust video calling capabilities, while Banuba's Face AR SDK enriches the experience with virtual backgrounds, AI face touch-up, AR filters, and other essential functionalities.

Accelerated development: The primary advantage of SDKs lies in their ability to significantly reduce time-to-market. By combining the power of these two SDKs, developers can incorporate even more valuable features in a fraction of the time it would take to build them from scratch.

Seamless integration: Integrating two SDKs within the same product necessitates smooth interoperability between them and the core application. Achieving such harmony often requires extensive time and effort. However, Banuba's extensions are designed to seamlessly integrate with Agora's products, requiring just a few lines of code and minimal testing.

Banuba and Agora, collaborating since 2019, have formed a mutually beneficial partnership that has greatly served our shared clientele. We have expanded the coverage of our SDK platform in conjunction with Agora, enabling our customers to deliver captivating augmented reality (AR) features to their end users across an extensive range of platforms and devices.

About Banuba

With a successful track record spanning over 7 years, Banuba is a leading augmented reality company that has pioneered technologies such as face tracking, virtual try-on, and virtual background solutions. Their offerings include online try-on solutions for various products like makeup, headwear, jewelry, glasses, and more. Additionally, we provide a comprehensive Face filters SDK and Video Editor SDK, offering pre-built modules for applying effects and editing videos.

About Agora

Headquartered in Santa Clara, California, Agora stands as a trailblazing global leader in Real-Time Engagement Platform-as-a-Service (PaaS). Their core focus is to empower developers by offering simple, flexible, and powerful application programming interfaces (APIs). These APIs enable the seamless integration of real-time voice, video, interactive live-streaming, chat, whiteboard, and artificial intelligence capabilities into applications.