VisionOS Enterprise API: fail to get cameraFrame in cameraFrameUpdates{}

I am developing an app based on visionOS and need to utilize the main camera access provided by the Enterprise API. I have applied for an enterprise license and added the main camera access capability and the license file in Xcode. In my code, I used

await arKitSession.queryAuthorization(for: [.cameraAccess]) 

to request user permission for camera access. After obtaining the permission, I used arKitSession to run the cameraFrameProvider. However, when running

for await cameraFrame in cameraFrameUpdates{
    print("hello")
            
    guard let mainCameraSample = cameraFrame.sample(for: .left) else {
        continue
    }
    pixelBuffer = mainCameraSample.pixelBuffer
}

, I am unable to receive any frames from the camera, and even print("hello") within the braces do not execute. The app does not crash or throw any errors.

Here is my full code:

import SwiftUI
import ARKit

struct cameraTestView: View {
    
    @State var pixelBuffer: CVPixelBuffer?
    var body: some View {
        VStack{
            Button(action:{
                Task {
                    await loadCameraFeed()
                }
            }){
                Text("test")
            }
            if let pixelBuffer = pixelBuffer {
                let ciImage = CIImage(cvPixelBuffer: pixelBuffer)
                let context = CIContext(options: nil)
                if let cgImage = context.createCGImage(ciImage, from: ciImage.extent) {
                    Image(uiImage: UIImage(cgImage: cgImage))
                }
            }else{
                Image("exampleCase")
                    .resizable()
                    .scaledToFill()
                    .frame(width: 400,height: 400)
            }
        }
    }
    
    func loadCameraFeed() async {
        // Main Camera Feed Access Example
        let formats = CameraVideoFormat.supportedVideoFormats(for: .main, cameraPositions:[.left])
        let cameraFrameProvider = CameraFrameProvider()
        let arKitSession = ARKitSession()
        
        // main camera feed access example
        var cameraAuthorization = await arKitSession.queryAuthorization(for: [.cameraAccess])

        guard cameraAuthorization == [ARKitSession.AuthorizationType.cameraAccess:ARKitSession.AuthorizationStatus.allowed] else {
            return
        }
        
        do {
            try await arKitSession.run([cameraFrameProvider])
        } catch {
            return
        }
        
        let cameraFrameUpdates = cameraFrameProvider.cameraFrameUpdates(for: formats[0])
        
        if cameraFrameUpdates != nil  {
            print("identify cameraFrameUpdates")
        } else{
            print("fail to get cameraFrameUpdates")
            return
        }
        
        for await cameraFrame in cameraFrameUpdates! {
            
            print("hello")
            
            guard let mainCameraSample = cameraFrame.sample(for: .left) else {
                continue
            }
            pixelBuffer = mainCameraSample.pixelBuffer
        }
    }
}

#Preview(windowStyle: .automatic) {
    cameraTestView()
}

When I click the button, the console prints:

identify cameraFrameUpdates

It seems like it stuck in getting cameraFrame from cameraFrameUpdates. Occurring on VisionOS 2.0 Beta (just updated), Xcode 16 Beta 6 (just updated). Does anyone have a workaround for this? I would be grateful if anyone can help.

Answered by Vision Pro Engineer in 800261022

Hey @AnranChen,

The one thing that you are missing is that you need to be in an ImmersiveSpace. As noted by Setting up access to ARKit data:

ARKit data is available only when your app presents a Full Space and other apps are hidden

Let me know if you have additional questions,
Michael

Accepted Answer

Hey @AnranChen,

The one thing that you are missing is that you need to be in an ImmersiveSpace. As noted by Setting up access to ARKit data:

ARKit data is available only when your app presents a Full Space and other apps are hidden

Let me know if you have additional questions,
Michael

hi @Vision Pro Engineer, Thank you so much!! I didn't really notice this before

VisionOS Enterprise API: fail to get cameraFrame in cameraFrameUpdates{}
 
 
Q