Is there any action that can clone the entity in RealityView to the number I want? If there is, please let me know. Thank you!

I created a simple Timeline animation with only a "Play Audio" action in RCP. Also a Behaviors Component setting an "OnTap" trigger to fire this Timeline animation. In my code, I simply run Entity.applyTapForBehaviors() when something happened. The audio can be normally played on the simulator but cannot be played on the device. Any potential bug leads this behavior? Env below: Simulator Version: visionOS 2.0 (22N5286g) XCode Version: Version 16.0 beta 4 (16A5211f) Device Version: visionOS 2.0 beta (latest)

We appear to be experiencing a bug with the latest beta for visionOS, we are attempting to playback a video with a transparent background in the app. In the previous beta playback worked as expected and the transparent parts of the video were transparent. In the latest beta the background appears black. The view we are using in a SwiftUI wrapped version of AVPlayerViewController, we have narrowed the bug down to only occurring only when playback is being presented in the embedded experience mode, if playback is being done in the expanded experience then playback is as expected. This has only only been visible on an actual device, we have been unable to replicate the behaviour in the simulator using the latest Xcode 16.0 beta(beta 5 (16A5221g)) This is sample project that shows off the bug

When using the sample application of object tracking with apple Vision Pro, the left eye display shows the same position of the original object's target object, but the right eye is misaligned. Is there any way to resolve this issue?

I can execute an action by allowing Xcode to send a notification to Reality Composer Pro via NotificationCenter, or I can send notifications to Xcode through the Notification Action in Reality Composer Pro. However, I discovered that they were unable to accept notifications from both parties within my project. To ascertain whether there was an error in my code, I created a simple Demo project. I utilized the same code and determined that it functioned normally within the Demo project. It is perplexing that I am unable to resolve this issue. Do I require additional modifications?

It's all about notifications to trigger actions from RCP's new Timeline system. From Compose interactive 3D content in Reality Composer Pro I am actually starting to confuse why there was need to use Entity.applyTapForBehaviors in code to trigger content in Behaviors Component. Simply because in Behaviors Component, we have chosen OnTap to allow a "Tap Notification" to trigger our action (on a selected target object). Then I guess by selecting OnCollision this trigger, I should write something like CollisionEvent.entityA.applyCollisionForBehaviors, which we don't have. And ofc the collision on my object won't trigger this action (because I only did things in RCP not in code). Ignoring this post has pointed out we could use Behaviors Component's OnNotification to trigger something for now. I found that I could still use OnTap trigger but actually put my code Entity.applyTapForBehaviors under my subscribed collision's begin event. That actually works better than OnCollision So what is the design principles here? And how could I trigger a collision notification to let my Behaviors Component's OnCollision actually works?

Steps to Reproduce: Create a SwiftUI view that initializes an ARKit session and a camera frame provider. Attempt to run the ARKit session and retrieve camera frames. Extract the intrinsics and extrinsics matrices from the camera frame’s sample data. Attempt to project a 3D point from the world space onto the 2D screen using the retrieved camera parameters. Encounter issues due to lack of detailed documentation on the correct usage and structure of the intrinsics and extrinsics matrices. struct CodeLevelSupportView: View { @State private var vm = CodeLevelSupportViewModel() var body: some View { RealityView { realityViewContent in } .onAppear { vm.receiveCamera() } } } @MainActor @Observable class CodeLevelSupportViewModel { let cameraSession = CameraFrameProvider() let arSession = ARKitSession() init() { Task { await arSession.requestAuthorization(for: [.cameraAccess]) } } func receiveCamera() { Task { do { try await arSession.run([cameraSession]) guard let sequence = cameraSession.cameraFrameUpdates(for: .supportedVideoFormats(for: .main, cameraPositions: [.left])[0]) else { print("failed to get cameraAccess authorization") return } for try await frame in sequence { guard let sample = frame.sample(for: .left) else { print("failed to get camera sample") return } let leftEyeScreenImage:CVPixelBuffer = sample.pixelBuffer let leftEyeViewportWidth:Int = CVPixelBufferGetWidth(leftEyeScreenImage) let leftEyeViewportHeight:Int = CVPixelBufferGetHeight(leftEyeScreenImage) let intrinsics = sample.parameters.intrinsics let extrinsics = sample.parameters.extrinsics let oneMeterInFront:SIMD3<Float> = .init(x: 0, y: 0, z: -1) projectWorldLocationToLeftEyeScreen(worldLocation: oneMeterInFront, intrinsics: intrinsics, extrinsics: extrinsics, viewportSize: (leftEyeViewportWidth,leftEyeViewportHeight)) } } catch { } } } //After the function implementation is completed, it should return a CGPoint?, representing the point of this worldLocation in the LeftEyeViewport. If this worldLocation is not visible in the LeftEyeViewport (out of bounds), return nil. func projectWorldLocationToLeftEyeScreen(worldLocation:SIMD3<Float>,intrinsics:simd_float3x3,extrinsics:simd_float4x4,viewportSize:(width:Int,height:Int)) { //The API documentation does not provide the structure of intrinsics and extrinsics, making it hard to done this function. } }

We are experiencing a crash when attempting to capture a high resolution frame. The crash only happens on the A12 and A12X devices (iPhone11,8 or iPad8,7). The crash did occur on older versions of iOS and continues to happen with more recent versions, iOS 17.6 and iPadOS 17.5.1. Any ideas on how to get a high-resolution image & frame from an AR Session using these devices? To reproduce the crash: Use an A12 or A12X device. Other devices have not produced the same result. Setup an AR session that can capture high resolution static images: Create a class that conforms to UIViewController and ARSessionDelegate and connect it to an Interface with an ARView. @IBOutlet var arView: ARView! During view setup configure the AR session: let configuration = ARBodyTrackingConfiguration() if let hiResFormat = ARBodyTrackingConfiguration.recommendedVideoFormatForHighResolutionFrameCapturing { configuration.videoFormat = hiResFormat } self.arView.session.run(configuration) Trigger code to capture the high resolution image (we call this from a button @IBAction: Task { let hiResFrame = try? await self.arView.session.captureHighResolutionFrame() print("crash above. this print never occurs") } The crash occurs with both the sync & async versions of the captureHighResolutionFrame call. Code above crashes in Apple AltruisticBodyPoseKit : 0 libsystem_kernel.dylib 0xc2ec __pthread_kill + 8 1 libsystem_pthread.dylib 0x7c0c pthread_kill + 268 2 libsystem_c.dylib 0x75ba0 abort + 180 3 libsystem_c.dylib 0x74eac err + 282 4 AltruisticBodyPoseKit 0x4b2b0 cva::MatrixData<int, 0ul, 0ul, false>::allocate(unsigned long) (.cold.1) + 42 5 AltruisticBodyPoseKit 0x20dac std::__1::vector<std::__1::pair<cva::Matrix<double, 3u, 1u, false>, cva::Matrix<double, 2u, 1u, false> >, std::__1::allocator<std::__1::pair<cva::Matrix<double, 3u, 1u, false>, cva::Matrix<double, 2u, 1u, false> > > >::vector(unsigned long) + 1186 6 AltruisticBodyPoseKit 0x2058c btr::(anonymous namespace)::EstimatePoseFromCorrespondences(btr::CameraPoseInfo&, btr::Correspondences2d3d const&, bool) + 564 7 AltruisticBodyPoseKit 0x2018c btr::BodyRegistration::RegisterBody(float vector[2] const*, unsigned long, float vector[3] const*, unsigned long, simd_float4x4 const*, unsigned long, simd_float3x3 const*, simd_float4x4 const*) + 1228 8 AltruisticBodyPoseKit 0x4354c -[ABPKCameraRegistration estimateCameraPoseFromMatchingwithImageIntrinsics:imageResolution:joints2d:jointsLifted3D:jointsLifted3DCount:] + 1160 9 ARKitCore 0x131d30 -[AR3DSkeletonRegistrationTechnique _estimateCameraPoseFromMatchingImageData:to3DData:worldTrackingPose:pCameraFromBody:depthData:pScaleOut:] + 396 10 ARKitCore 0x131818 -[AR3DSkeletonRegistrationTechnique requestResultDataAtTimestamp:context:] + 388 11 ARKitCore 0x91fd8 -[ARParentTechnique technique:didOutputResultData:timestamp:context:onTechniques:] + 1400 12 ARKitCore 0x91a28 -[ARParentTechnique technique:didOutputResultData:timestamp:context:] + 112 13 ARKitCore 0x8150c -[ARExposureLightEstimationTechnique requestResultDataAtTimestamp:context:] + 352 14 ARKitCore 0x91fd8 -[ARParentTechnique technique:didOutputResultData:timestamp:context:onTechniques:] + 1400 15 ARKitCore 0x91a28 -[ARParentTechnique technique:didOutputResultData:timestamp:context:] + 112 16 ARKitCore 0xcd9c4 -[ARWorldAlignmentTechnique requestResultDataAtTimestamp:context:] + 1044 17 ARKitCore 0x91fd8 -[ARParentTechnique technique:didOutputResultData:timestamp:context:onTechniques:] + 1400 18 ARKitCore 0x91a28 -[ARParentTechnique technique:didOutputResultData:timestamp:context:] + 112 19 ARKitCore 0x92584 -[ARParentTechnique _submitResultsForTimestamp:context:] + 396 20 ARKitCore 0x90124 __71-[ARParentTechnique requestResultDataAtTimestamp:context:onTechniques:]_block_invoke_3 + 72

AFAIK there's no way to programmatically detect when an ImmersiveSpaceContent is dismissed by a user (i.e. by pressing the home button). By comparison, ImmersiveView has .onAppear() and .onDisappear(): ImmersiveSpace(id: appModel.immersiveSpaceID) { ImmersiveView() .environment(appModel) .onAppear { appModel.immersiveSpaceState = .open } .onDisappear { appModel.immersiveSpaceState = .closed } } In comparison: // No similar callbacks for here: struct MyImmersiveSpace: ImmersiveSpaceContent { var body: CompositorLayer { /* ... */ } }

In visionOS2, there exists a function that enables users to raise their hand to display the home button. However, this functionality conflicts with the interaction required for the mixed display space utilized within my application. Therefore, I seek a method to disable this functionality.

The entity in My RealityView contains tracking components and allows them to track different places of the hand. However, I found that except for the fingertip of the index finger, the fingertip of the thumb, the palm and the wrist, all other positions cannot be tracked normally (such as the fingertip of the middle finger). How can I solve it (I think it may be a beta version of the bug)

In my Volume, there is a RealityView that includes lighting effects. However, when the user drags the position of the window back and forth, the farther the distance between the volume and the user, the greater the brightness of the light effect. ( I believe this may be a Beta version of a bug.) Note: The volume windowGroup has the .defaultWorldScaling(.dynamic) property.

In the reality view, I found that the entity could not cast a shadow on the reality. What configuration should I add to achieve this function?

I am developing an immersive visionOS app based on RealityKit and SwiftUI. This app has ModelEntities that have a PerspectiveCamera entity as child. I want to display the camera view in a 2D window in visionOS. I am creating the camera, and add it to the entity with let cameraEntity = PerspectiveCamera() cameraEntity.camera.far = 10000 cameraEntity.camera.fieldOfViewInDegrees = 60 cameraEntity.camera.near = 0.01 entity.addChild(cameraEntity) My app is not AR. The immersive view is programmatically generated. In iOS, I could use an ARView with non AR camera mode. However, ARView is not available in visionOS. How can I show the camera view in a SwiftUI 2D window in the immersive space?

Hello, I'm trying to stream stereoscopic SBS video on the APV. I see that AVPlayerViewController supports MV-HEVC video playback but it's not clear how to play SBS video on the Apple Vision Pro. Are there any docs or examples you can share? For my use case, SBS is the only format I can support.

I am attempting to execute actions after clicking an entity in Reality View using the Behaviors component. I have added the Input Target component and the Tap gesture as follows: TapGesture().targetedToAnyEntity() .onEnded({ value in _ = value.entity.applyTapForBehaviors() }) ) However, during testing, I have observed that the entity does not appear to recognize the click gesture. Could you kindly provide any relevant documentation or guidance on this matter?

Hi everyone, I'm new to Swift and VisionOS development in general, so please go easy on me. Currently, I'm looking at a sample project from a WWDC23 session that uses RealityKit and ARKit to add a cube entity to a scene via tap gesture. The link to the sample project is here. Instead of adding a cube, I changed the code to adding a usdz model instead. Here is my code: func add3DModel(tapLocation: SIMD3<Float>) { let placementLocation = tapLocation + SIMD3<Float>(0, 0.1, 0) guard let entity = try? Entity.load(named: "cake-usdz", in: realityKitContentBundle) else { logger.error("failed to load 3D model") return } // calculate the collision box (the boundaries) let entitySize = entity.visualBounds(relativeTo: nil) let width = entitySize.max.x - entitySize.min.x let height = entitySize.max.y - entitySize.min.y let depth = entitySize.max.z - entitySize.min.z // logger.debug("width: \(width), height: \(height), depth: \(depth)") // set collision shape let collisionShape = ShapeResource.generateBox(size: SIMD3<Float>(width, height, depth)) entity.components.set(CollisionComponent(shapes: [collisionShape])) // set the position and input types to indirect entity.setPosition(placementLocation, relativeTo: nil) entity.components.set(InputTargetComponent(allowedInputTypes: .indirect)) let material = PhysicsMaterialResource.generate(friction: 0.8, restitution: 0.0) entity.components.set(PhysicsBodyComponent( shapes: [collisionShape], mass: 1.0, material: material, mode: .dynamic )) contentEntity.addChild(entity) } This works fine so far. But when I tried to add a Drag Gesture to drag the added entity around. There are weird glitches happening with the model. The model jumped up and down, and even rotating around it self sometimes. Below is my code for Drag Gesture. I placed it directly below the code for Spatial Tap Gesture in the sample project. .gesture(DragGesture().targetedToAnyEntity().onChanged({ value in let targetedEntity = value.entity targetedEntity.position = value.convert(value.location3D, from: .local, to: .scene) })) At first, I thought my code was wrong. But after looking around and removing the PhysicsBodyComponent for the added model, the entity was moving as intended while dragging. I can't figure out a solution to this. Could anyone help me? I'm currently on Xcode 16 beta 2, and visionOS 2.0. Because I'm on Beta, I'm unsure if this is a bug or if I just missed something. Thank you.

I'm trying to hit an API URL, however I am getting this error (501) Invalidation handler invoked, clearing connection (501) personaAttributesForPersonaType for type:0 failed with error Error Domain=NSCocoaErrorDomain Code=4099 "The connection to service named com.apple.mobile.usermanagerd.xpc was invalidated: failed at lookup with error 159 - Sandbox restriction." UserInfo={NSDebugDescription=The connection to service named com.apple.mobile.usermanagerd.xpc was invalidated: failed at lookup with error 159 - Sandbox restriction.} Received port for identifier response: <(null)> with error:Error Domain=RBSServiceErrorDomain Code=1 "Client not entitled" UserInfo={RBSEntitlement=com.apple.runningboard.process-state, NSLocalizedFailureReason=Client not entitled, RBSPermanent=false} elapsedCPUTimeForFrontBoard couldn't generate a task port This is what my info.plist looks like - This is the code I'm using to hit the URL and get a response func sendMessage() { guard let url = URL(string: "https://API_URL") else { return } var request = URLRequest(url: url) request.httpMethod = "POST" request.setValue("application/json", forHTTPHeaderField: "Content-Type") let body = ["query": message] //creates a dictionary with key:value pair request.httpBody = try? JSONSerialization.data(withJSONObject: body) //converts the dictionary to json data and sets as body of request isLoading = true response = "" URLSession.shared.dataTask(with: request) { data, _, error in //initiates async task for sending request DispatchQueue.main.async { //async update of UI on main thread isLoading = false } if let data = data, error == nil, //checks that data was received and that there is no error let json = try? JSONSerialization.jsonObject(with: data) as? [String: String] { //parsing json response DispatchQueue.main.async { response = json["response"] ?? "No response" } } }.resume() // starts the network request } Can anyone help me understand what the errors are and why I'm not able to get the response back?

We are encountering an issue with our app on Vision Pro with OS version 1.3. The app runs perfectly in the VisionOS Simulator, but when tested on the actual device, no content is displayed. Here’s the expected behavior: When the app launches, a video should play in a window. Once the video ends, another information window should open. After a series of these information windows, we will load to an immersive space to handle 3D elements. We've set the "Preferred Default Scene Session Role" to "Window Application Session Role" in info.plist, but the issue persists. Below is the code we're using. Any advice or suggestions would be greatly appreciated. import SwiftUI @main struct myApp: App {          @StateObject var sharedData = SharedDataModel()     @State private var isFactoryEnabled = false     var body: some Scene {                  WindowGroup(id: "LaunchScreen", content: {             LaunchScreen()                          })         .windowStyle(.plain)         .environmentObject(sharedData)                  WindowGroup(id: "LoginView", content: {             ZStack {                 let _ = UserDefaults.standard.set(false, forKey: "_UIConstraintBasedLayoutLogUnsatisfiable")                 let _ = print(FileManager.default.urls(for: .documentDirectory, in: .userDomainMask).first!.path)                                      LoginView()                 }         }).windowStyle(.plain)         .environmentObject(sharedData)                  WindowGroup(id: "TrainingSelection", content: {                              if !sharedData.showNavigationHintView{                     NavigationHintView()                         .glassBackgroundEffect()                         .cornerRadius(30)                                      }                 else {                     TrainingSelection()                 }         }).windowStyle(.plain)         .environmentObject(sharedData)                  WindowGroup(id: "Salutations", content: {             Salutations()         }).windowStyle(.plain)         .environmentObject(sharedData)                  WindowGroup {             ContentView()         }         .environmentObject(sharedData)         ImmersiveSpace(id: "myImmersiveSpace") {             ImmersiveView(viewModel: .init())         }         .environmentObject(sharedData)     } } import SwiftUI import AVFoundation import RealityKit import RealityKitContent struct LaunchScreen: View {     @State private var player: AVPlayer?     @State private var navigateToContentView = false     @EnvironmentObject var audioPlayer: AudioPlayer     var body: some View {         ZStack {             ZStack {                 if navigateToContentView {                     WarningView()                         .transition(.opacity)                         .glassBackgroundEffect()                         .cornerRadius(15)                 } else {                     if let player = player {                         AVPlayerView(player: player)                             .onAppear {                                 player.play()                                 addObserver()                             }                             .cornerRadius(30)                     } else {                         Text("Unable to Load the Video")                             .foregroundColor(.white)                             .onAppear {                                 loadVideo()                             }                     }                 }             }             .edgesIgnoringSafeArea(.all)             .animation(.easeIn, value: 1)         }         .glassBackgroundEffect()     }     private func loadVideo() {         if let videoUrl = Bundle.main.url(forResource: "launchScreen", withExtension: "mp4") {             player = AVPlayer(url: videoUrl)         } else {             print("Unable to Load the Video")         }     }     private func addObserver() {         NotificationCenter.default.addObserver(             forName: .AVPlayerItemDidPlayToEndTime,             object: player?.currentItem,             queue: .main         ) { _ in             self.navigateToContentView = true         }     } }

We’re looking to extend the capabilities of our Apple Vision Pro app to properly support the spatial playback of 180° 3D immersive videos. Currently, when these videos are played back, they are projected onto the entire 360° sphere, which results in a distorted and less-than-optimal experience for the user. Our goal is to ensure that the 180° video content is correctly displayed within the horizontal hemisphere only, rather than across the full sphere. We’re unsure of the best approach to achieve this and would greatly appreciate your guidance. Would it be possible for your team to review our code and provide us with the necessary steps or adjustments needed to achieve the desired playback results? Case-ID: 8729125 Thank you for your assistance.