Capture Video from my own app using enterprise APIs in visionOS

Hello,

I want to capture video from Vision Pro in the Vision OS app. I am referring to the (https://developer.apple.com/videos/play/wwdc2024/10139/) Apple video and their code. step like below

  1. import ARKit
  2. com.apple.developer.arkit.main-camera-access.allow = true in info.plist
  3. Do below code
 func loadCameraFeed() async {
        // Main Camera Feed Access Example
        let formats = CameraVideoFormat.supportedVideoFormats(for: .main, cameraPositions:[.left])
        let cameraFrameProvider = CameraFrameProvider()
        var arKitSession = ARKitSession()
        var pixelBuffer: CVPixelBuffer?
        await arKitSession.queryAuthorization(for: [.cameraAccess])
        do {
            try await arKitSession.run([cameraFrameProvider])
        } catch {
            return
        }
        guard let cameraFrameUpdates =
            cameraFrameProvider.cameraFrameUpdates(for: formats[0]) else {
            return
        }
        print(cameraFrameUpdates)
        for await cameraFrame in cameraFrameUpdates {
            
            print(cameraFrame)
            
            guard let mainCameraSample = cameraFrame.sample(for: .left) else {
                continue
            }
            pixelBuffer = mainCameraSample.pixelBuffer
        }
    }

I want to convert "pixelBuffer" into video streaming and show it in a frame like iOS.

Please guide me on how to achieve my next step. I am blank after this code.

Answered by Vision Pro Engineer in 792961022

Hey @Sandip_Aura,

If you haven't already, please make sure that you have requested the associated entitlements. Enterprise APIs are eligible for business use only, and you can only distribute apps that you develop with the Enterprise APIs privately as proprietary in-house apps or custom apps using Apple Business Manager.

Once you have the entitlement, use the init(cvPixelBuffer:) initializer create a CIImage. Once you have an CIImage, use the createCGImage(_:from:) method on a CIContext instance to create a CGImage. From there you can create a UIImage and then pass it to your SwiftUI Image view.

let ciImage = CIImage(cvPixelBuffer: pixelBuffer)
let context = CIContext(options: nil)
if let cgImage = context.createCGImage(ciImage, from: ciImage.extent) {
    Image(uiImage: UIImage(cgImage: cgImage))
}

Let me know if you have additional questions,

Michael

Hey @Sandip_Aura,

If you haven't already, please make sure that you have requested the associated entitlements. Enterprise APIs are eligible for business use only, and you can only distribute apps that you develop with the Enterprise APIs privately as proprietary in-house apps or custom apps using Apple Business Manager.

Once you have the entitlement, use the init(cvPixelBuffer:) initializer create a CIImage. Once you have an CIImage, use the createCGImage(_:from:) method on a CIContext instance to create a CGImage. From there you can create a UIImage and then pass it to your SwiftUI Image view.

let ciImage = CIImage(cvPixelBuffer: pixelBuffer)
let context = CIContext(options: nil)
if let cgImage = context.createCGImage(ciImage, from: ciImage.extent) {
    Image(uiImage: UIImage(cgImage: cgImage))
}

Let me know if you have additional questions,

Michael

Capture Video from my own app using enterprise APIs in visionOS
 
 
Q