Best Practice to Add Objects at Eye Level in Reality Kit

Question

Created Jun ’24

Replies 2

Boosts 0

Participants 2

I would think it would be common practice that when adding a new entity into your RealityView scene for them to appear in front of the user. And then the user places the entity in the scene. Image a puzzle piece appearing in front of you and you drag it to your puzzle board.

if you move around your puzzle board you’d expect that wherever you are the new piece should appear in front of you.

That seems applicable to a lot of applications.

I can add a new entity using the head anchor but as we all know that transform is the identity so reparenting the entity to something (eg puzzle board) won’t work.

I’ve been trying to use World positioning and query pose which helps but I’m stumped as to how to get the new entity to appear in front of me, no matter which way I turn.

Looking for suggestions and guidance on this.

Boost

Answer 1

keaura OP

Jun ’24

I worked out a solution. Maybe not the best, but here's what I did.

Needs to be in Immersive Space.
Enable world tracking via WorldTrackingProvider.
Create an entity (I parented mine to the main scene) and I called it "user".
Subscribe to SceneEvents.Update events.
When the update happens, use queryDeviceAnchor to get the position of the device (ie, the user).
Update the user entity position:

let transform = deviceAnchor.originFromAnchorTransform
user?.transform = Transform(matrix: transform)

Now when you place your new entity into the scene, just do this:

child.setPosition([0, -0.32, -0.53], relativeTo: user)
child.setOrientation(.init(.init(angle: .zero, axis: .y)), relativeTo: user)

First position the new child (of the scene) slightly ahead of the user and slightly down (or wherever you want to put it). The (0,0,0) position is inside the user's head, effectively.
Then change the orientation to 0º so it faces the user.

The new child is where I want it, but a little "wonky" in the X and Z axis - it has to do with the user's head tilt and I haven't figured out how to correct it.

0

Answer 2

Vision Pro Engineer OP

Apple

Jun ’24

Hi @keaura ,

nice job so far with the solution! I am linking a sample that I think might help you understand the layout a bit more and relate it to your own app.

https://developer.apple.com/documentation/realitykit/combining-2d-and-3d-views-in-an-immersive-app

It shows how to lay out 2D and 3D views in your app relative to one another, and although your question isn't focused on the 2D aspect, I think the explanation and sample code of how to lay them out could be helpful nevertheless.

0