Unity and Vision Pro: It’s Complicated

I was originally planning on taking the Unity path to development for Vision Pro. But as I dug deeper, I ran into some complications.

First a few definitions:

A Shared Space app shares the space with other apps’ windows and volumes, with the room visible via pass-through video. A Full Space app has the entire space to itself, and can create unbounded content (unlike the strictly bounded volumes in Shared Space). Full Space apps can also access additional information, like polygon meshes for objects in the room, and user head and hand position.

Full Space apps can be either Mixed Immersion, where the real world is visible and can interact with virtual objects, or Full Immersion, which is essentially traditional VR — the outside world is not visible, and the app draws everything the user sees. (There’s also a Progressive Immersion mode where the user can switch between Mixed and Full.

With those definitions out of the way, we can discuss the roughly 3 ways to develop for Reality Pro with Unity:

Develop a conventional Unity VR app. This lets you use the full power of Unity, including custom shaders to create a full-space virtual reality app. But you can’t access the pass through video to display the real world. This is because Metal-based apps can’t access pass through, and Unity is using Metal to render. For more details, see the WWDC video “Bring your Unity VR app to a fully immersive space”.

Shared Space apps using Unity’s PolySpatial technology. This technology lets you build a Unity app on top of Apple’s RealityKit instead of Metal. This lets Unity apps participate in the Shared Space at the cost of some restrictions. Materials have to be mapped onto RealityKit materials, and no custom shaders are permitted. 3D is shown inside of Volumes, bounded areas of preset size. You specify what part of your Unity scene appears inside the Volume with a new camera in Unity called the Volume Camera. For more details, see the WWDC video “Create immersive Unity apps”.

Unbounded Volumes with PolySpatial? At 9:27 into the “Create immersive Unity apps” video there’s a tease of something fascinating. An unbounded volume that displays in Full Space, with the ability to use pass through along with other Full Space features like hand and head pose tracking. See the “input” section of the video for a demo where a user paints a wall with flowers. I haven’t been able to find out much about this functionality so far, but it sounds promising. Note that the PolySpatial restrictions still apply.

Other Considerations

There is a limited beta for Unity on the Vision Pro. I signed up for it almost immediately after it was announced, but have yet to be accepted. Meanwhile, the current versions of Unity 2022.3 (2022.3.9f1 as of this writing) let you download Vision Pro support alongside Mac, Windows, iOS and Android. However I haven’t successfully built for Vision Pro in Unity, possibly because Unity’s Vision Pro Support appears to require Xcode 15 beta 2 from June, whereas the latest version of Xcode 15 is the release candidate.

As I write this, Unity just announced that their licensing terms will be changing in 2024. Developers will be charged based on both revenue and total number of installs of an app, rather than just revenue. I’ve heard from several upset developers who are considering switching to another game engine as a result.

That’s pretty much all I know about developing for Vision Pro with Unity. Based on what I learned, I’ve switched to developing for Vision Pro in Swift using Apple’s native RealityKit API. I hope the information in this article will help you decide which development path is right for you.