The Line Between Mobile and Console Immersive Experiences Is Blurring

We’re closer than ever before to closing the gap in immersive gaming experiences between mobile and console. With the exponential rise in device performance, the right implementation approach, tooling, and optimization techniques, developers are now able to deliver mobile games at unprecedented levels of quality.

And it’s now becoming a whole lot easier to develop incredible immersive experiences thanks to the coming to market of standalone VR devices: lightweight virtual reality headsets capable of operating entirely independently of a console, smartphone or any other tether.

We recently featured Fast Travel Games, a virtual reality (VR) focused games developer, in our ‘Made Possible’ series of partner innovation stories – watch it below. The Fast Travel Games story is fascinating as it exemplifies how the line between quality of content deployable on mobile and console game platforms is blurring. Indeed, many games are now increasingly offering cross-platform play, with mobile devices at their epicenter. As a result, games are capable of reaching a far broader audience, hence the growing need to offer even more comparable experiences.

Fast Travel Games’ expertise in developing high-quality VR content, combined with the balanced performance and power-efficiency of Arm-based processors really showcase the ability to push the boundaries of immersive experiences and bring them to millions of players.

Standalone VR devices offer incredible experiences

Over the past decade or more, mobile compute – processor (CPU), graphics processor (GPU), interconnect, memory – performance has gone up exponentially while the form factor of devices have got thinner, all while staying within a tight thermal budget of about 2.5W. Arm and its ecosystem has been at the forefront of creating innovative solutions that have directly lead to this increase in mobile compute performance and efficiency. Standalone VR devices – also known as standalone VR headsets – are taking those same innovative mobile solutions, adding some additional VR focused goodness and putting it into a larger form factor that enables a much higher thermal budget (>4W) and increases the peak sustainable performance. Couple this with a slimmer, console style operating system and you have freed up resources capable of the heavy lifting required to give you an Oculus Rift-level VR experience on a truly untethered device.

However, Arm solutions aren’t just about the hardware: software and tools play a huge part in enabling a robust ecosystem to simplify the developer experience. There are a wealth of resources and powerful tools freeing the developers to focus on creating their vision without compromise by allowing easy detection of application bottlenecks through the visualization of all performance data right across the compute solution.

Shared multiplayer battles

As a gamer and long-time VR aficionado, I have been lucky enough to try many VR experiences – from early efforts such as The Lab VR Archery on HTC Vive through to my first VR scare in The Deep on PSVR. I’ve also been a huge Star Wars fan ever since I saw the original movie in 1977 – and thanks to VR games like Star Wars: Jedi Challenges and Vader Immortal, I can finally live out my lightsaber-wielding dreams.

However, it’s one thing playing against an AI opponent or ‘bot’. The next big thing for me is multiplayer in the same physical space – challenging your friends to an epic lightsaber duel. Being an old-school Nintendo fan, multiplayer gaming to me is synonymous with sitting together with my friends playing Mario Kart, Super Smash Bros and Goldeneye. We’re starting to see truly awesome multiplayer VR and these shared experiences would not be possible without the new wave of standalone VR devices with inside-out tracking capabilities.

High-end hardware for your head

Creating standalone VR devices is about trade-offs. Trading things like resolution and field of view against weight, cost & battery life. Eliminating or reducing the impact of these compromises is where Arm shines.

Through developing new or improved products like Arm’s recently launched Cortex-A77 CPU, Mali-G77 GPU & Mali-D77 DPU, future VR devices will enable more and better pixels, for longer, deeper experiences.

Arm Mali-D77, in particular, is a truly interesting VR innovation that simultaneously enables better quality VR and frees rendering resources to either deliver deeper or longer experiences. VR visuals need to be delivered matched to the user’s head movement; essentially what is rendered needs to be displayed and matched to the head position in less than 20ms – otherwise there is a risk of motion sickness. To do this most VR devices do something called ‘Asynchronous Timewarp’ or ATW. Just before images are sent to the display, ATW looks at the user’s head pose and slightly moves the image to match. At the same time, it can pre-correct for distortions the devices lens will introduce. Today the GPU handles this workload but it is quite inefficient and takes time away from the GPU’s more important job of rendering the immersive graphics. Mali-D77 performs ATW and lens correction in hardware, in real-time as the images are sent to the display. This saves power and gives resource back to the GPU while performing better and more consistent lens correction.

Arm and our ecosystem continue to innovate across all areas of processing. 5G infrastructure is another innovation that has the potential to push higher quality VR experiences, through distributed processing and rendering, and more immediate shared experiences through lower latency communications.

Untethering the VR experience

Standalone VR devices remove the friction that I think hinders tethered VR; I’m expecting to see more and deeper experiences from developers with even more interactive storylines. The platforms and tools are out there now, they just need developers to choose VR as the place to realize their vision.

Eye tracking and foveated rendering are two very exciting technologies coming to standalone VR devices as they can further enhance experiences with higher, denser visuals without increasing requirements on the rendering hardware and provide new ways of interacting within experiences. For example, I recently tried Varjo’s amazing VR-1 headset and was blown away by how realistic VR can be.

For developers, having lots of platforms to target for their VR projects is a blessing; more platforms, more users, raises the general market size. Yet it also can be a curse, as to address the widest audience you have to build for many platforms with potentially differing capabilities. This fragmentation in the industry makes life very difficult. In the games industry much of the fragmentation is mitigated through open standards like OpenGL-ES and thankfully a widely supported AR/VR open standard, OpenXR, was launched this year. Hopefully, as this rolls out it will remove some of the pain points developers have in supporting multiple platforms.

Moving onto the broader xR front, machine learning (ML) will become an essential aspect of providing you with truly immersive experiences. A device’s ability to understand the space you are in through identifying everyday objects such as a table, floor or person, comprehend what people are saying, interpret complex gestures and actions and apply context based on a person’s location or where they are looking will eventually deliver a more humanized machine interface.

There’s already some amazing content available on an ever-increasing number of great standalone VR devices. This new device category is removing some of the shortcomings to mass-adoption that have been the issue for tethered devices. Standalone devices coupled with other technology innovation will free consumers to experience VR whenever, wherever and with whomever they want.

Learn more about Arm AR/VR solutions and how Arm is pushing the boundaries of what’s possible with ever-smaller devices.