Tracked object coordinates in program

Question

Created Aug ’24

Replies 1

Boosts 0

Participants 2

Hey, as a follow up to my earlier posts about object tracking on visionOS 2 - I'm doing some experimentation, and my use-case/requirements require me to track the coordinates of some digital entity that I attach (relative to my reference object) to my reference object.

Can something like this be done?

Right now, all I'm doing is putting my reference object in my scene, and then positioning the 3D content that I want to show at the corresponding locations on the reference object. I am then loading the scene in a RealityView block via my SwiftUI code.

I want to know now if I can also extract and use the coordinates of the digital entity that I have placed (post object-tracking), and then make some manipulations via code, for example, if the physical coordinates of the digital entity is in a certain x,y,z range -> trigger this function/bring up this alert message in a tile..

Is something like this possible, and if so, can you help me with understanding different aspects to this problem via code with some sample/reference code? So far I've only done most of the object tracking related tasks via the Reality Composer Pro, but this task that I'm trying to implement will require me to do quite a bit of programming as well, and I'm kinda lost as to how to start and go about this.

Thanks for any help that ya'll can give me!

Boost

Answer 1

Vision Pro Engineer OP

Apple

Aug ’24

Hi @adityach

It sounds like you want to track and respond to changes in an entity's transform (position, rotation, scale) over time. This is possible using the Entity Component System (ECS) architecture. Let's walk through this solution:

First review the documentation for ECS referenced above so you have the context to follow along.
Start a SpatialTrackingSession since you want to access the transform of an AnchorEntity or its children; for others following along this is covered in this post.
Create a component for the entity you wish to track.

struct TrackingComponent : Component {
}

Assign that component to the entity you wish to track.

entity.components.set(TrackingComponent())

Create a system, that runs every render cycle (many times per second), to query for all the entities with a TrackingComponent and act on them.

struct TrackingSystem: System {
    static let query = EntityQuery(where: .has(TrackingComponent.self))
    
    public init(scene: RealityKit.Scene) {
    }
    
    public func update(context: SceneUpdateContext) {
        let trackedEntities = context.entities(matching: Self.query,
                                          updatingSystemWhen: .rendering)
        
        for entity in trackedEntities {
            
            // print the position in world space
            print(entity.position(relativeTo: nil))
            
            // update the entity here
        }
    }
}

Important: register TriggerComponent and TriggerSystem in your app's initializer.

init() {
    TrackingSystem.registerSystem()
    TrackingComponent.registerComponent()
}

0