visionOS: Apple’s new Vision Pro OS – What is it? What to expect?

Apple’s milestone operating system updates can sometimes mean its software is getting anything from a fresh coat of paint to a complete renovation. However, the unveiling of Apple’s augmented and virtual reality-based visionOS platform is a new home for Apple users entirely.

The Apple Vision Pro may be the piece of hardware stealing the show right now, but its success as a product will be down to more than just 4K micro-OLED displays and dual M2 and R1 SoCs – visionOS is the heart and soul of Apple’s AR/VR vision, one that will grow and adapt over future generations of headsets.

There’s a lot going on here, and plenty to explore. Some features of visionOS will be relatively familiar to those in the Apple ecosystem already while some things are entirely new to us all. Not only do we now have a completely new Apple device to get to grips with, but there’s also an entirely new operating system to do it on. Let’s dive into the details on what we know so far and what those itching to step into the virtual Apple-verse can expect.

This story is currently developing as WWDC 2023 continues…

Apple visionOS highlights

• Native support for all iPad and iOS apps at launch
• Familiar developer-friendly frameworks for app creation
• visionOS includes facial scanning for real-time, true-to-life avatars
• Backed by M2 performance, and R1 SoCs
• Spatial ray-tracing audio engine for supreme soundscapes in 3D
• 3D video playback and recording
• IR EyeSight eye-tracking, and sense of presence functions 
• Hand tracking and voice control
• Familiar yet all-new interface
• Interfaces cast shadows to add to sense of scale
• Movable interfaces in 3D space
• Multitask across numerous AR windows with the Multi-App 3D engine
• Sync with MacBook to host 4K display in AR anywhere
• Virtual keyboard interaction, or audio dictation for input
• 100+ Apple Arcade titles ready to play at launch

What is visionOS?

Apple’s latest operating system has been designed with two key features in mind, augmented reality and virtual reality. The operating system will be the backbone for the new Apple Vision Pro headset, offering unique features, apps, and functions to users while exploring this new mixed-reality frontier.

We first became aware of visionOS under the name xrOS after some digital sleuthing unveiled a cleverly disguised trademark filed by an Apple shell corporation to the New Zealand Intellectual Property Office. The information coincided with a report from Bloomberg’s Mark Gurman, who revealed that insider sources also confirmed the name xrOS to be legit.

The “xr” of xrOS stands for “extended reality,” the clear focus and vision of the Apple Vision Pro. As it turns out, Apple’s final name for the software is visionOS. Apple’s goal to bridge the gap between our digital and physical worlds is a lofty one, but its new visionOS platform could be vital in ensuring Apple’s next steps into mixed reality are a success.

visionOS features

visionOS is the first operating system built from the ground up for spatial computing. 

visionOS makes use of the Vision Pro’s wide array of cameras to constantly blend the virtual and real world. Delivering a stable and constant picture with floating UI elements that you can interact with. The operating system also makes use of audio ray-tracing to ensure the soundscapes from these augmented reality vistas are as accurate as possible.

Infinite screen real estate is now possible, without any of that physical clutter. Shape and size your workstation at will, tailor your environment to suit your any given need, and more. 

visionOS supports all your favorite iOS and iPadOS apps

The Vision Pro is Apple’s first step into a post-iPhone landscape, but that doesn’t mean it’s leaving everything behind to start anew. While there’ll be a little less physical tapping, the visionOS will feel fresh and familiar to those clued into the Apple ecosystem. Expect the same design ethos, the same fluidity, and a range of familiar apps to find a new home on the visionOS platform.

At launch, visionOS will support hundreds of thousands of iPad and iOS. Even intensive apps like Adobe Lightroom will be available at launch, all easily controlled with subtle hand gestures and your eyes.

You can even play your favorite Apple Arcade titles to play on Vision Pro, with 100_ games available to play on day one alone. You can even use third-party controllers to control things!

visionOS – 3D content for a 3D you

A great number of these apps will be tailored directly to the Vision Pro’s unique features, allowing for eye-tracking and gesture recognition to play a key part in how you interact with Apple’s new operating system. There’s even a dial to fine tune just how much you want the visionOS software to phase out the world around you.

The home screen you’ve previously been used to, truly is a screen now. Expanding into an enormous virtual panel from which you can control how you interact with subtle gestures.

EyeSight tracking allows others to know when you can and cannot see them. It also allows the Vision Pro to display your eyes accurately when you can. Tracking your gaze to show others you’re clued into the conversation and paying attention.

Eyesight can be programmed to scan your face, capturing your likeness. This likeness can then be tracked and puppeteered by visionOS in various apps, acting as a highly responsive avatar. Allowing non Apple Vision Pro users to look into the device from outside, and see a clear and accurate representation of your own eyes looking back.

FaceTime on Vision Pro will display other callers in full scale, floating around the room, giving a real sense of presence. If a user is using the Vision Pro, they’ll be represented by one of the aforementioned avatars, bringing a lifelike and realistically animated avatar that mirrors expressions and mouth movements. You’ll also be able to share other apps through FaceTime, including the ability to watch movies together.

With the Vision Pro, you can capture and experience photos and videos in ways never seen before, soak in panoramas in full vision, or play back 3D spacial photos and video snapped by the Vision Pro’s 3D camera. Enjoy cinema level playback of films and TV with visionOS able to identify the room around you, dimming it from vision to help you focus fully on the moment.

visionOS x Disney

Disney CEO, Bob Igor, was welcomed to the stage to showcase visionOS’ AR flourish being introduced into Disney+ services. With simple, hand gestures able to interact with content. Including that of an interactive Marvel What If…? experience, alongside various other Disney properties.

Apple Vision Pro for developers

The same frameworks for apps across the Apple ecosystem can be found on visionOS, allowing the same fantastic and familiar options developers have grown used to over the years. This gives developers the tools needed to re-imagine their existing apps easily into complete spatial experiences. 

Developers can use familiar tools and frameworks like xCode, SwiftUI, Reality Kit, and AR kit to build apps on visionOS. Reality Composer Pro is another developer-friendly app capable of quickly and easily building complex scenes with realistic objects. There’s even a partnership with Unity3D to bring native support to the development tool.

Apple Vision Pro for productivity

visionOS will also be fantastic for productivity users. Microsoft apps like Excel, Word, and teams and conferencing tools like Zoom and Webex will all feature. With conferencing tools able to use your EyeSight-scanned avatar to communicate with others naturally.

visionOS security: Optic ID

We’ve already seen biometric data like fingerprints being used to keep devices and information secure. With visionOS, Apple Vision Pro is able to scan and use retina biometrics for increased security.

This new system encrypts your data, never to leave your device, and is only accessible with the secure enclave processor. This allows visionOS users to use apps like Apple Pay, App Store purchases and Password Auto-fill with nothing more than a glance. No more passwords required.

visionOS release date

visionOS’ one and only home will be on the Apple Vision Pro headset, so it should arrive alongside the Vision Pro headset in early 2024.