VisionOS Portal

I think a occlusion material could work. So instead of having a positive shape with the portal material you would put the user in a box with an occlusion material applied to the inside and with an opening in the front. The ground assets close to the portal can be cliped in the 3D content creation software so that they dont pertrude the mask. If you want you could combine it with a flip/flop to change the shader behaviour when you are looking at the backface of the mask...

Whats also really good with "Encounter Dinosaurs", is how the lighting on the 3D meshes changes as they come out of the portal. It blends between the IBL of the scene and the IBL of your current environment. You can see a similar effect happening when playing around in a progressive space with the digital crown, but the effect in the experience is much more localized and precise.

Any idea on how to create such an Occlusion Material in RealityKit? I guess one would need to write it in Metal. I took a look a the custom nodes in RealityComposerPro but could not find anything.

Thanks for your reply.

OcclusionMaterial might be the way to go, but the edge cases will have to be addressed.

As far as how to create one, OcclusionMaterial is provided by RealityKit and is available on VisionOS, so all you have to do is initialize one like this: OcclusionMaterial() And pass it to the array of materials on a ModelComponent.

let myEntity = Entity()
let myModelComponent = ModelComponent(mesh: myMesh, materials: [OcclusionMaterial()])
myEntity.components.set(myModelComponent)

OR

let myModelEntity = ModelEntity(mesh: myMesh, materials: [OcclusionMaterial()])

There is a good example on Medium.com titled "Creating an iOS AR Portal App using Occlusion Materials" but it uses the older version of RealityKit for handheld devices.

Thanks! Yes I found an old session from 2019 where they talk exactly about this for iOS: https://developer.apple.com/videos/play/wwdc2019/605/?time=1012 The OcclusionMaterial() can also be created on VisionPro the but the material will stay black, and the pass through video is not revealed

Furthermore: when experimenting with their "Portal" Solution we are facing more and more issues. For example it's super hard to work with any kind of window or view inside the portal. And there is no glass effect, the UI looks flat. Also there are some rendering issues when other UI layers appear, like the control center for example... Generally the Portal seams a bit half-baked, but without an Occlusion Shader it's very hard to make these kind of experiences.

When trying this app you will notice that they will switch your something around when your head gets close to the door. But it seams like they have an working Occlusion Material: https://moonvrhome.com/spatial-portal

Just a little update. So the image I showed was taken with a progressive immersion style. Probably because of how the soft edge fade in progressive works. the passthrough is not available with OcclusionMaterial() but if you switch to a mixed immersion style the OcclusionMaterial() works as expected! So in this mode it should be possible to build a box with an opening and having stuff come out of the "portal".

Hey, OcclusionMaterial is indeed the way to go! For a working VisionOS example, take a look at this project: https://github.com/qvik/helsinkios-visionos-portals