queryDeviceAnchor sometimes stops working in func update(context: SceneUpdateContext) (visionOS)

Question

XWDev OP

Created Aug ’24

Replies 1

Boosts 1

Participants 2

Hi,

I was wondering during developing for visionOS why when I try to use queryDeviceAnchor() with WorldTrackingProvider() after opening the immersive space in the update(context: SceneUpdateContext) function, it initially seems to provide the DeviceAnchor data every frame but stops at some point (about 5-10 seconds after pressing the Button which opens the immersive space) and then stops updating constantly and only updates somehow randomly if I move my head abruptly to the left, right, etc. Somehow, the tracking doesn't seem to work as it should directly on the AVP device.

Any help would be greatly appreciated!

See my code down below:

ContentView.swift

import SwiftUI

struct ContentView: View {
    @Environment(\.openImmersiveSpace) private var openImmersiveSpace
    @Environment(\.scenePhase) private var scenePhase
    
    var body: some View {
        VStack {
            Text("Head Tracking Prototype")
                .font(.largeTitle)
            Button("Start Head Tracking") {
                Task {
                    await openImmersiveSpace(id: "appSpace")
                }
            }
        }
        .onChange(of: scenePhase) {_, newScenePhase in
            switch newScenePhase {
            case .active:
                print("...")
            case .inactive:
                print("...")
            case .background:
                break
            @unknown default:
                print("...")
            }
        }
    }
}

HeadTrackingApp.swift

import SwiftUI


@main
struct HeadTrackingApp: App {
    
    init() {
        HeadTrackingSystem.registerSystem()
    }
    
    var body: some Scene {
        WindowGroup {
            ContentView()
        }
        
        ImmersiveSpace(id: "appSpace") {
          
        }
    }
}

HeadTrackingSystem.swift

import SwiftUI
import ARKit
import RealityKit

class HeadTrackingSystem: System {
    let arKitSession = ARKitSession()
    let worldTrackingProvider = WorldTrackingProvider()
    
    required public init(scene: RealityKit.Scene) {
            setUpSession()
    }
    
    
    func setUpSession() {
        Task {
            do {
                try await arKitSession.run([worldTrackingProvider])
                
            } catch {
                print("Error: \(error)")
            }
        }
    }
    
    public func update(context: SceneUpdateContext) {

        guard worldTrackingProvider.state == .running else { return }
        
        let avp = worldTrackingProvider.queryDeviceAnchor(atTimestamp:  CACurrentMediaTime())
        print(avp!)
    }

Answered by Vision Pro Engineer in 799352022

Hi @XWDev

I think the cause of the issue is HeadTrackingSystem.update not being invoked as often as you expect. Per the docs for System:

A system calls the update(context:) method as often as the updatingSystemWhen parameter of entities(matching:updatingSystemWhen:) defines.

One way around this is to create an empty AnchorEntity, add it to your scene then query for it in HeadTrackingSystem.update. For example:

In your ImmersiveView:

RealityView { content in
    let entity = AnchorEntity(.head)
    content.add(entity)
}

In HeadTrackingSystem:

struct HeadTrackingSystem: System {
    static let query = EntityQuery(where: .has(AnchoringComponent.self))

    // ...
    
    public func update(context: SceneUpdateContext) {
        let _ = context.entities(matching: Self.query,
                                         updatingSystemWhen: .rendering)
        //...
    }
}

As an aside (this may be something you are aware of), if your end goal is to position an entity relative to the user's head consider using an AnchorEntity with a head target. For example, this snippet will place a sphere in front of a person's head:

let sphere = ModelEntity(mesh: .generateSphere(radius: 0.02), materials: [SimpleMaterial(color: .green, isMetallic: false)])

sphere.position = [0, 0, -1.0]
let headAnchorEntity = AnchorEntity(.head)
headAnchorEntity.addChild(sphere)
content.add(headAnchorEntity)

Boost

Answer 1

Vision Pro Engineer OP

Apple

Aug ’24

Accepted Answer