How to insert modified video frames into the system camera to achieve AI erase effect?

Question

Created 2w

Replies 1

Boosts 0

Participants 2

By applying for the enterprise API, we can obtain the data of video frames collected by VisionPro glasses, and then we process the collected video frames to achieve the function of eliminating a certain object. But it was not found how to insert the processed video frames into the data source collected by the system camera. So I would like to ask if there is any API that can insert processed video frames into the original data and present them to the user? This effect is similar to the right side twist of VisionPro glasses, which allows the physical world and digital space to blend perfectly after rotation. So, I would like to ask if there is a related API that can solve this problem?

STEPS TO REPRODUCE

Obtain video frames,
Process the obtained video frames
Insert the processed video frames into the VisonOS system camera.

System: VisionOS 2.0

API used: Enterprise APIs Main camera access permissions

Boost

Answer 1

Vision Pro Engineer OP

Apple

2w

So I would like to ask if there is any API that can insert processed video frames into the original data and present them to the user?

Sorry, such an API does not exist on visionOS. Please note that for user comfort it is very important to keep the motion-to-photon latency as short as possible, so there is no time for apps to process the camera stream before it is drawn on screen.

0