Need to get The VisonOS Player's Camera forward vector on runtime

Question

Lhamed-ung OP

Created 6d

Replies 1

Boosts 0

Participants 2

How can I access the player’s camera vector in VisionOS, specifically using RealityKit?

In Unity and other game engines, there’s often an API like "Camera.main.transform.forward for this purpose. "

I’ve found the head anchor but haven’t identified a way to obtain the forward vector in RealityKit.

Is there a related API for this? Any guidance would be greatly appreciated.

Thanks!

Answered by Vision Pro Engineer in 814483022

Hi @Lhamed-ung

You can obtain the device's transform matrix using WorldTrackingProvider.queryDeviceAnchor. The 3rd entry in that matrix (x vector, y vector, z vector, translation) contains the forward vector. Here's a snippet that prints the device's forward and position to the console every frame.

struct ImmersiveView: View {
    @State var session = ARKitSession()
    @State var worldTrackingProvider = WorldTrackingProvider()
    
    var body: some View {
        RealityView { content in
            _ = content.subscribe(to: SceneEvents.Update.self) {_ in
                
                // Note: I'm using SceneEvents.Update to simplify this example.
                // Avoid manipulating entities in SceneEvents.Update.
                // To run code every frame, its best to use an ECS System.
                Task {
                    guard worldTrackingProvider.state == .running,
                          let deviceAnchor =  worldTrackingProvider.queryDeviceAnchor(atTimestamp: CACurrentMediaTime()) else {return}
                    
                    let cameraTransform = deviceAnchor.originFromAnchorTransform
                    let cameraForward = cameraTransform.columns.2
                    let cameraPosition = cameraTransform.columns.3
                    
                    print("Camera forward: \(cameraForward)")
                    print("Camera position: \(cameraPosition)")
                }
            }
        }
        .task {
            do {
                try await session.run([worldTrackingProvider])
            } catch {
                print("Error: \(error)")
            }
        }
    }
}

Boost

Answer 1

Vision Pro Engineer OP

Apple

5d

Accepted Answer

Recommended

Hi @Lhamed-ung

You can obtain the device's transform matrix using WorldTrackingProvider.queryDeviceAnchor. The 3rd entry in that matrix (x vector, y vector, z vector, translation) contains the forward vector. Here's a snippet that prints the device's forward and position to the console every frame.

struct ImmersiveView: View {
    @State var session = ARKitSession()
    @State var worldTrackingProvider = WorldTrackingProvider()
    
    var body: some View {
        RealityView { content in
            _ = content.subscribe(to: SceneEvents.Update.self) {_ in
                
                // Note: I'm using SceneEvents.Update to simplify this example.
                // Avoid manipulating entities in SceneEvents.Update.
                // To run code every frame, its best to use an ECS System.
                Task {
                    guard worldTrackingProvider.state == .running,
                          let deviceAnchor =  worldTrackingProvider.queryDeviceAnchor(atTimestamp: CACurrentMediaTime()) else {return}
                    
                    let cameraTransform = deviceAnchor.originFromAnchorTransform
                    let cameraForward = cameraTransform.columns.2
                    let cameraPosition = cameraTransform.columns.3
                    
                    print("Camera forward: \(cameraForward)")
                    print("Camera position: \(cameraPosition)")
                }
            }
        }
        .task {
            do {
                try await session.run([worldTrackingProvider])
            } catch {
                print("Error: \(error)")
            }
        }
    }
}

0