I am a student at Utah Valley University doing a UX Research project involving spatial web browsing on Safari. I am trying to determine if spatial video and photos would be supported on a safari web page while using the AVP. I am not a developer, so my knowledge of that front is limited, but I am hoping to get any insight into if that feature would be able to be implemented into a web based experience. If so, what formats would need to be used? Is the MV-HEVC format able to be directly embedded? Or is there another format that needs to be explored? Any insight is appreciated!

In Mixed Reality Mode there is strange issues with indirect pinches on objects. If a user uses an indirect pinch to select an object and then walks around, or moves and re-orients their body while maintaining the pinch, the object moves as if there is some scalar being applied to it and it causes the object to behave in ways that are extremely counter-intuitive compared to other MR devices. If a user indirect pinches on an object and then walks forward the object flies away from the user, faster than they are walking. If a user indirect pinches on an object and then walks backward, the object flies towards and eventually past the user, faster than they are walking. If a user indirect pinches an object and then turns around, the object rotates around some unknown position and with some added scalar resulting in very strange behavior. Here are some examples of the issue in action. The first video is using Unity's Polyspatial SDK. The second video is using an entirely native stack of SwiftUI and RealityKit with NO Unity at all. For some reason I am not allowed to link videos here from Drive or Gyazo, so I am including it in plaintext for now. If someone could direct me how I can upload video examples of what I am describing directly to these forums, I would appreciate it. First Video Showing Issue in Unity with PolySpatial SDK: https://i.gyazo.com/95788cf9d4587c167b544db031fbf412.mp4 Second Video Showing Issue in native only stack with RealityKit and Swift UI: https://drive.google.com/file/d/1mgt8TXJiopbm6qdJw2rFG0geam0irnMn/view?usp=sharing Unity Forum Bug Discussion which, after Investigation, Confirmed this issue is on the Native Platform: https://discussions.unity.com/t/objects-do-not-behave-properly-when-manipulated-in-an-mr-space/1482439 For a Mixed Reality Environment, where a user may want to move around their space, while using Indirect Pinches to manipulate and "carry" objects with them this is a big issue. Thank you

Hi, I love VideoMaterial API that gives so much power to play video on any mesh. But I am trying to play a side-by-side 3D video usingVideoMaterial: RealityView { content in let mesh = MeshResource.generatePlane(width: 300.0, height: 300.0, cornerRadius: 0) //generate mesh let vidMaterial = VideoMaterial(avPlayer: AVPlayer(url: URL(string: "https://someurl/test/master.m3u8")!)) //VideoMaterial vidMaterial.controller.preferredViewingMode = .stereo //<-- no idea why it doesn't work for SBS video in simulator vidMaterial.avPlayer?.play() let planeEntity = Entity() //new entity planeEntity.components.set(ModelComponent(mesh: mesh, materials: [vidMaterial])) //set a new ModelComponent to the entity content.add(planeEntity) } this code works well for plain 2D video playback but how do I display a Side-by-Side or Top-Bottom 3D video? I found GeometrySwitchCameraIndex in custom ShaderGraphMaterial but if I use input node as a image texture then how do I pass the video frame as texture into my custom shader to achieve the 3D effect or maybe there is an even better way to deal with this? There seems to be additional API .preferredViewingMode on the VideoMaterial's controller that can be set to .stereo but it doesn't give any stereo effect. Perhaps it's only for MV-HEVC media playback?

Is it possible to play a stereoscopic video in MV-HEVC format using a player embedded in an HTML page?

In a listing program in WWDC24, it shows that users can control the robot to walk by pinching and sliding. However, I haven't found any documents or videos related to this function. If you know, please let me know. Thank you!

I am trying to display a 3D model in iOS app using RealityView. The same 3D model is displayed successfully in the visionOS app. Everything works perfectly only when I set my project’s minimum deployment target to iOS 18.0. However, my app’s minimum deployment target is iOS 15.0. When I use the RealityKitContent package to load the 3D model, it fails to compile and gives me the following error: Compiling for iOS 15.0, but module 'RealityKitContent' has a minimum deployment target of iOS 18.0: /Users/Library/Developer/Xcode/DerivedData/RealityViewForiOS-cbfkgimsqngtuegqwvezusvscllf/Index.noindex/Build/Products/Debug-iphonesimulator/RealityKitContent.swiftmodule/arm64-apple-ios-simulator.swiftmodule I have made the RealityKitContent package optional and tried importing using the following condition: #if canImport(RealityKitContent) import RealityKitContent #endif Despite this, it still fails to compile and produces the same error. I have not found a workaround for using the RealityKitContent package with app targets lower than iOS 18.0. Here is my package definition: let package = Package( name: "RealityKitContent", platforms: [ .visionOS(.v1), .macOS(.v15), .iOS(.v18) ], products: [ .library( name: "RealityKitContent", targets: ["RealityKitContent"]), ], dependencies: [], targets: [ .target( name: "RealityKitContent", dependencies: []), ] ) Here is the code I am using to load the 3D model with RealityView using the RealityKitContent package: import SwiftUI import RealityKit #if canImport(RealityKitContent) import RealityKitContent #endif struct ContentView: View { var body: some View { VStack { if #available(iOS 18.0, *) { RealityView { content in if let scene = try? await Entity(named: "Scene", in: realityKitContentBundle) { content.add(scene) } } update: { content in if let scene = content.entities.first { let uniformScale: Float = 3.0 scene.transform.scale = [uniformScale, uniformScale, uniformScale] } } } else { // Fallback for earlier versions } } } } #Preview { ContentView() } Any help or guidance on how to use the RealityKitContent package for app targets lower than iOS 18.0 would be greatly appreciated.

I am a student developer We are trying to implement an application that allows you to take photos in visionOS mr mode and access the photos you took. Can the contents of the link below be used on visionOS? https://developer.apple.com/tutorials/sample-apps/capturingphotos-captureandsave/ I would really appreciate your reply. For reference, we plan to package the methods in swift and import the framework into Unity to use them.

Hi I am using this function to create collisions in my scene from Apple Developer Video I found. func processReconstructionUpdates() async { for await update in sceneReconstruction.anchorUpdates { let meshAnchor = update.anchor guard let shape = try? await ShapeResource.generateStaticMesh(from: meshAnchor) else {continue} switch update.event { case .added: let entity = ModelEntity() entity.transform = Transform(matrix: meshAnchor.originFromAnchorTransform) entity.collision = CollisionComponent(shapes: [shape], isStatic: true) entity.physicsBody = PhysicsBodyComponent() entity.components.set(InputTargetComponent()) meshEntities[meshAnchor.id] = entity contentEntity.addChild(entity) case .updated: guard let entity = meshEntities[meshAnchor.id] else { fatalError("...") } entity.transform = Transform(matrix: meshAnchor.originFromAnchorTransform) entity.collision?.shapes = [shape] case .removed: meshEntities[meshAnchor.id]?.removeFromParent() meshEntities.removeValue(forKey: meshAnchor.id) } } } The code works great. In the same immersive space I am opening a window: var body: some View { RealityView { content in // some other code here openWindow(id: "mywindowidhere") // some other code here } } The window opens in front of me, but I am not able to click or even hover on the buttons. At first I did not know why that was happening. But then I turned on pointer control and found out that the pointer is actually colliding with the wall. (the window is kinda inside the wall). That is why the pointer never reaches the window and the button never gets clicked. I initially thought this was a layering issue, but I was not able to find any documentation related to this. Is this a known issue and is there any way to fix this? Or I am doing anything wrong on my side?

Hello, I was wondering how I can initialize an ImageAnchoringSource using https://developer.apple.com/documentation/realitykit/anchoringcomponent/imageanchoringsource/init(_:) When I construct one using a URL, it doesn't seem to be tracked and I see in the following when I debug print the component: ▿ 0 : AnchoringComponent ▿ target : Target ▿ referenceImage : 1 element ▿ from : ImageAnchoringSource ▿ url : Optional<URL> ▿ some : file:///var/mobile/Containers/Data/Application/D1126EA0-A1D7-468F-A40C-8578B7F5BDDF/Library/Caches/CodeCache/0E457AA7-2195-48B9-9DD4-58CEB9397F69.png - _url : file:///var/mobile/Containers/Data/Application/D1126EA0-A1D7-468F-A40C-8578B7F5BDDF/Library/Caches/CodeCache/0E457AA7-2195-48B9-9DD4-58CEB9397F69.png - _parseInfo : nil - _baseParseInfo : nil - name : nil - group : nil ▿ trackingMode : TrackingMode - trackingMode : 2 Is there a specific format for the parseInfo? When I use the same image to make an image anchoring source by group and name in AR Resources, it is tracked. Thank you!

Hi, I'm brainstorming ideas for getting dynamic content inside my visionOS app on the Vision Pro. I have some data coming out of a piece of equipment, and reaching a cloud hub (something like IoT Hub on Azure). I want to get that data inside a visionOS app, ideally inside an attachment that is attached to some 3D entity inside my RealityView. Is something like this possible? Can someone give me some starter points on how I can enable a pipeline like this, and if there are any resources that I could use for reference.

I am using the Xcode visionOS debugging tool to visualize the bounds of all the containers, but it shows my Entity is inside the Volume. Then why does it get clipped? Is there something wrong with the debugger, or am I missing something? import SwiftUI @main struct RealityViewAttachmentApp: App { var body: some Scene { WindowGroup { ContentView() } .windowStyle(.volumetric) .defaultSize(Size3D(width: 1, height: 1, depth: 1), in: .meters) } } import SwiftUI import RealityKit import RealityKitContent struct ContentView: View { var body: some View { RealityView { content, attachments in if let earth = try? await Entity(named: "Scene", in: realityKitContentBundle) { content.add(earth) if let earthAttachment = attachments.entity(for: "earth_label") { earthAttachment.position = [0, -0.15, 0] earth.addChild(earthAttachment) } if let textAttachment = attachments.entity(for: "text_label") { textAttachment.position = [-0.5, 0, 0] earth.addChild(textAttachment) } } } attachments: { Attachment(id: "earth_label") { Text("Earth") } Attachment(id: "text_label") { VStack { Text("This is just an example") .font(.title) .padding(.bottom, 20) Text("This is just some random content") .font(.caption) } .frame(minWidth: 100, maxWidth: 300, minHeight: 100, maxHeight: 300) .glassBackgroundEffect() } } } }

I installed Beta 4 of VisionOS 2 and now whenever I take the headset off nothing works after putting it back on except passthrough until the device is rebooted. This is somewhat inconvenient. I've raised it as a feedback but wondering if this is something others have noticed.

I can‘t Figure Out How to Get My Earth Entity to Rotate on its Axis. This is a follow up post from a previous Apple Developer forum post. How would I have the earth (parent) entity rotate CCW underneath the orbiting starship child? I tried adding the following code block to the RealityView but it is not working: if let rotatingEarth = starshipEntity.findEntity(named: "Earth") { rotatingEarth.transform.rotation = simd_quatf.init(angle: 360, axis: SIMD3(x: 0, y: 1, z: 0)) if let animation = try? AnimationResource.generate(with: rotatingEarth as! AnimationDefinition) { rotatingEarth.playAnimation(animation) } } Any advice on getting the earth to rotate? I tried reviewing the Hello World WWDC23 project code, but I was unable to understand the complexity and how that sample project got the earth to rotate. i want to do this for visionOS 1.2. I realize there are some new animation and possible other capabilities coming up in vision 2.0 but I want to try to address this issue in the current released visionOS version.

I'm working on a multi-platform app (macOS and visionOS for now). In these early stages it’s easier to target the Mac, but I started with a visionOS project. One of the things the template creates is a RealityKitContent package dependency. I can target macOS 14.5 in Xcode, but when it goes to build the RealiityKitContent, I get this error: error: Building for 'macosx', but '14.0' must be &gt;= '15.0' [macosx] info: realitytool ["/Applications/Xcode-beta.app/Contents/Developer/usr/bin/realitytool" "compile" "--platform" "macosx" "--deployment-target" "14.0" … Unfortunately, I'm unwilling to update this machine to macOS 15, as it's too risky. Running macOS 15 in a VM is not possible (Apple Silicon). This strikes me as a bug, or severe shortcoming, of realitytool. This was introduced with visionOS 1.0, and should be able to target macOS &lt; 15. It's not really reasonable to use Xcode 15, since soon enough Apple will require I build with Xcode 16 for submission to the App Store. Is this a bug, or intentional?

I am trying to add joints via code in my visionOS app. My scenario requires me to combine models from Reality Composer Pro with entities and components from code to generate the dynamic result. I am using the latest visionOS beta and Xcode versions and there is no documentation about joints. I tried to add them via the available API but regardless of how I combine pins, joints and various component, my entities will not get restricted or stay fixated like they are when they are in a child/parent relationship. I am using RealityKit and RealityView in mixed mode. I also searched the whole internet for related information without finding anything. Any insights or pointers appreciated!

Hi, I read online that to downgrade from Vision OS 2 developer beta back to Vision OS 1 - we need the developer strap. My request for the strap is still under review, is there a way to downgrade without the strap if the need arises?

So in the WWDC23 video on the Roomplan enhancement, it says that it is now possible to set a custom ARSession for the RoomCaptureSession. But how do you actually set the config for the custom ARSession? init() { let arConfig = ARWorldTrackingConfiguration() arConfig.worldAlignment = .gravityAndHeading arSession = ARSession() roomCaptureView = RoomCaptureView(frame: CGRect(x: 0, y: 0, width: 42, height: 42), arSession: arSession) sessionConfig = RoomCaptureSession.Configuration() roomCaptureView.captureSession.delegate = self roomCaptureView.delegate = self } However, I keep getting an issue that self is being used in the property access before being initialised. What can I do to fix it?

Hello! I have an iOS app where I am looking into support for visionOS. I have a whole bunch of gestures set up using UIGestureRecognizer and so far most of them work great in visionOS! But I do see something odd that I am not sure can be fixed on my end. I have a UITapGestureRecognizer which is set up with numberOfTouchesRequired = 2 which I am assuming translates in visionOS to when you tap your thumb and index finger on both hands. When I tap with both hands sometimes this tap gesture gets kicked off and other times it doesn't and it says it only received one touch when it should be two. Interestingly, I see this behavior in Apple Maps where tapping once with both hands should zoom out the map, which only works sometimes. Can anyone explain this or am I missing something?

Hello, I have an iOS app that is using SwiftUI but the gesture code is written using UIGestureRecognizer. When I run this app on visionOS using the "Designed for iPad" destination and try to use any of my gestures I see this warning in the console: Trying to convert coordinates between views that are in different UIWindows, which isn't supported. Use convertPoint:fromCoordinateSpace: instead. But I don't see any visible problems with the gestures. I see this warning printed out after the gesture takes place but before any of our gesture methods get kicked off. So now I am wondering if this is something we need to deal with or some internal work that needs to happen in UIKit. Does anyone have any thoughts on this?

Hi, it seems the accuracy of the true depth map is far worse when streaming (using iPhone 13) with similar artefacts as shown in this post: https://forums.developer.apple.com/forums/thread/694147. However when taking static photos, the quality is pretty good, despite resolutions being the same (480x640). This is for an object <1m distance. Does anyone know how I can improve the accuracy when streaming?