Hi! I think this should be a pretty normal usage of ARKit / RealityKit I have a static mesh for my environment, that I want to have static collision properties. My options for making this interact with dynamic bodies are: ShapeResource.generateConvex(...) -- which overshoots my shape dramatically. Entity.generateCollisionShapes(...) which also overshoots. I notice additional APIs around ShapeResource -- ShapeResource.generateStaticMesh(positions:faceIndices) seems to be exactly what I need. So far, I haven't been able to invoke this successfully to set my collision box. Questions: Is this not, a completely normal thing for developers to want to do? Why is there no support for this out-of-the-box in RealityKit/ARKit? To support this in my app, everywhere I've read has said I need to parse the .obj of my terrain manually, and find triangulated faces and pipe them into this function. That feels like a very standardized process -- and given that RealityKit is already forcing me to use .usdz, why should this not be a part of the SDK? Regardless- I triangulated my terrain mesh, and have been working on parsing code to get the positions and faceIndices for this set up (as an extension on Entity). Is this the right approach? Am I missing something more obvious? Thanks, Justin

Im working on the following problem: For a measurment application i want to take a picture of something laying on the ground, and given i will have the floor plane detecred, i plan to raycast 4 points from the corners of the screens, given the raycast land on this plane, i want to use those coordinates to do a perspective transform (warp) of the camera-image onto the new coordinates. This way i should be able to perform pixel-per-cm measurments. The prolem i have is the screen coordinates dont seem to reflect the camera-frame coordinates, and im not sure how to go from one to another.

Hi, I create an entity and add a bunch of attachments (code is based on the Diorama demo). I can rotate the entity with this: .gesture( DragGesture() .targetedToAnyEntity() .onChanged { value in let entity = value.entity let orientation = Rotation3D(entity.orientation(relativeTo: nil)) let newOrientation: Rotation3D if (value.location.x >= lastGestureValue) { newOrientation = orientation.rotated(by: .init(angle: .degrees(0.5), axis: .y)) } else { newOrientation = orientation.rotated(by: .init(angle: .degrees(-0.5), axis: .y)) } entity.setOrientation(.init(newOrientation), relativeTo: nil) lastGestureValue = value.location.x } ) But the attachments stay still. How can I rotate the entity AND the attachment at the same time?

I have a custom material in Reality Composer. When I attach it to a cube and try loading the scene in XCode, the material cannot be cast to a ShaderGraphMaterial because it has been changed to a PhysicallyBasedMaterial. The material was always a Custom material, I did not change the type in Reality Composer. Does anyone know how to fix?

We are porting a iOS Unity AR app to native visionOS. Ideally, we want to re-use our AR models in both applications. These AR models are rather simple. But still, converting them manually would be time-consuming, especially when it gets to the shaders. Is anyone aware of any attempts to write conversion tools for this? Maybe in other ecosystems like Godot or Unreal, where folks also want to convert the proprietary Unity format to something else? I've seen there's an FBX converter, but this would not care for shaders or particles. I am basically looking for something like the Polyspatial-internal conversion tools, but without the heavy weight of all the rest of Unity. Alternatively, is there a way to export a Unity project to visionOS and then just take the models out of the Xcode project?

I've got a couple 2D PNG assets that I want to add to a scene made of a couple other udsz files in RCP (picture adding a couple 2D videogame characters to a simple 3D diorama). When I try to drag the PNGs to the workspace or the file tree…nothing happens. I found a walkthrough on Medium (called "Importing and Exporting Personalized Objects for Augmented Reality: Reality Composer and SwiftUI" for those curious as I can't link to Medium posts here) that makes it look like users could do this with simple drag-and-drop. The Medium post is from June 2023, and in the screenshots RCP visually looks a lot more like Reality Composer on iPad, so I'm assuming it's changed a lot since then? Is there still a way to do this? I've tried adding the 2D elements to a scene with Blenders "import images as planes," but I'm getting weird halos around them and was hoping RCP could make the process a bit easier/cleaner.

I would like to add text to a Reality Composer Pro scene and set the actual text via code. How can I achieve this? I haven't seen any "Text" element in the editor.

I see example code converting the results of a SpatialTap to a SIMD3 location. For example, from WWDC session Meet ARKit for spatial computing: let location3D = value.convert(value.location3D, from: .global, to: .scene) What I really want is a simd_float4x4 that includes orientation of the surface that the tap gesture/cast collided with? My goal is to place an object with its Y-axis along the normal of the surface that was tapped. For example, in the referenced WWDC session, they create a CollisionComponent from the MeshAnchor data. If that mesh data is covering a curved couch cushion, I would like the normal from that curved cushion (i.e., the closest triangle approximating it). Is this possible? My planned fallback is to only use planes for collision surfaces for tap gestures, extract the tap gesture value's entity (which I am hoping is the plane), and grab its transform for the orientation information. I am hoping Apple has a simple function call that is more general than my fallback approach.

I'm trying to make a simple demo of using ShaderGraphMaterial in a USDZ file that I can preview on Mac and VisionOS but I'm having trouble. In Reality Composer, I make a sphere, then assign a ShaderGraphMaterial to the material, with a simple diffuse color (green) input. When I save the file as .usda, it displays as a gray sphere on mac rather than the green sphere shown in reality composer. If I then convert to usdz using Reality Converter, I get a warning on import: "Shader nodes must have “id” as the implementationSource, with id values that begin with “Usd”. Also, shader inputs with connections must each have a single, valid connection source." And the exported .usdz also shows as a gray sphere. Is there a simple demo of a .usda file using ShaderGraphMaterial that displays on Mac, iOS, and VisionOS that I can look at to see how it looks internally? My actual problem is creating usdz / usda files on visionOS for viewing on iOS / Mac / VisionOS.. but the first step is showing it's possible to even use ShaderGraphMaterial across all platforms. Thanks

I'm following the Meet Reality Composer Pro walkthrough and ran into something that didn't function as expected. When I got to the step where I add five "Bird_With_Audio.usda" references to the scene, I found they did not play audio. After some trial and error, I found that Preview > Resource in each of their Spatial Audio items was set to "None." If I click the dropdown menu, I see several "Bird_Calls" groups to pick from. I checked the original Bird_With_Audio.usda that I had created, and the "Bird_Calls" audio group was correctly assigned and worked. I tried dragging a sixth Bird_With_Audio into the scene and confirmed that the Spatial Audio item suddenly empties, rendering the bird silent. I was able to go through each of the five birds and set their Spatial Audio Resource to Bird_Calls, and the group worked like the video demonstrates. While this fixed the issue, as a beginner I'd like to know why this happened. It doesn't seem right that I would build and item and then have to re-attach any sounds to it when I place it in the main scene. So…where did I mess up?

Hello. I'm developing the app using ARKit and RealityKit. The purpose of the app is to scan the apartment and put furniture next to the walls. It works good, but if AR session takes more than 3 mins at some point app is crashed. According to crash report it's not something related to my code. I'm attaching crash report (company data is hidden). Any help is appreciated. Thanks in advance.

Hi, I'm trying to have an entity (and some attachments to it) to rotate. If I add the entity to content, add the attachment as a child entity, and set the entity as InputTargetComponent, then when I add a gesture ONLY the entity rotates and NOT the attachments (added as child entities). If I add a parent entity with let parentEntity = ModelEntity(), add my entity to the parentEntity, then add the attachments to an entity (which is now a child of the ModelEntity) and set the ModelEntity as InputTargetComponent then the whole thing rotates (including attachments) I'm sure there must be a bug, why would it work only with an added ModelEntity? Anyway, bug or not a bug, the problem I have now is that it rotates around the axes of the ModelEntity, not my primary entity, which is what I want. Is there a way to set the ModelEntity axes to be the axes of my primary child entity so it rotates like I want? What call should I use to move the axes where would I find the axes of the first child entity which should be the focus of my app? Here is my code: var body: some View { RealityView { content, attachments in // Add the initial RealityKit content if let specimenentity = try? await Entity(named: "Immersive", in: realityKitContentBundle) { let parentEntity = ModelEntity() parentEntity.addChild(specimenentity) content.add(parentEntity) let entityBounds = specimenentity.visualBounds(relativeTo: parentEntity) parentEntity.collision = CollisionComponent(shapes: [ShapeResource.generateBox(size: entityBounds.extents).offsetBy(translation: entityBounds.center)]) parentEntity.generateCollisionShapes (recursive: true) parentEntity.components.set(InputTargetComponent()) if let Left_Hemisphere = attachments.entity(for: "Left_Hemisphere") { //4. Position the Attachment and add it to the RealityViewContent Left_Hemisphere.position = [-0.5, 1, 0] specimenentity.addChild(Left_Hemisphere) } } } attachments: { Attachment(id: "Left_Hemisphere") { //2. Define the SwiftUI View Text("Left_Hemisphere") .font(.extraLargeTitle) .padding() .glassBackgroundEffect() } } .gesture( DragGesture() .targetedToAnyEntity() .onChanged { value in let entity = value.entity var orientation = Rotation3D(entity.orientation(relativeTo: nil)) var newOrientation: Rotation3D if (value.location.x >= lastGestureValueX) { newOrientation = orientation.rotated(by: .init(angle: .degrees(0.5), axis: .y)) } else { newOrientation = orientation.rotated(by: .init(angle: .degrees(-0.5), axis: .y)) } entity.setOrientation(.init(newOrientation), relativeTo: nil) lastGestureValueX = value.location.x orientation = Rotation3D(entity.orientation(relativeTo: nil)) if (value.location.y >= lastGestureValueY) { newOrientation = orientation.rotated(by: .init(angle: .degrees(0.5), axis: .x)) } else { newOrientation = orientation.rotated(by: .init(angle: .degrees(-0.5), axis: .x)) } entity.setOrientation(.init(newOrientation), relativeTo: nil) lastGestureValueY = value.location.y } ) } }

Is there any way to detect if an entity is being looked at in a RealityView. I know it is possible to add a "HoverEffectComponent()" which will highlight the entity a little when you gaze on it, but there doesn't seem to be any way to call a function from this. There is also no GazeGesture or anything similar.

I am working with MeshAnchors, and I am having troubles getting to the classification of the triangles/faces. This post references the MeshAnchor.Geometry, and that struct does have a property named "classifications", but it is of type GeometrySource. I cannot find any classification information in GeometrySource. Am I missing something there? I think I am looking for something of type MeshAnchor.MeshClassification, but I cannot find any structs with this as a property.

Hi! Currently showing the Apple Vision Pro to my clients. Sharing the screen is challenging with Guest Mode on the AVP. You can share the screen but you need to teather the AVP and MBP to an iPhone and then you can AirPlay. A lot of big corporate WiFi networks may not allow AirPlay so teathering is a better option. Does anyone know if Apple is making their AVP demo app available to developers? This would be super helpful for showing off the AVP's capabilities. Thanks! JB

I currently have an iOS app that transmits h264 code through wifi, uses videotoolbox to decode and displays it with MTKView, and I want to implement similar functions in visionOS. What should I do? MTKView is not available on visionOS

It slows down the device, screws with user interaction -- which makes exponentially worse the ridiculous one minute capture time.

I was heavily reliant on using ARGeoAnchor in my iOS application and when started porting the app to visionOS encountered there is no equivalent there. Which is a huge bummer and showstopper to launching on Apple Vision Pro. Is there any technical limitation that didn't allow devs to port this great piece of functionality? Can we expect it to be added in the future visionOS releases?

I've been recently working on a VisionOS app which uses CoreMl to identify specific body parts and display a window with information of the identified body part, since the use of Vision Pro's cameras is blocked, I'm using an iPhone to perform image classification, and then send the label to the headset using Multipeer Connectivity, I'd like to display a volume once the user selects a body part, could my iPhone return enough spatial information for me to be able to fully take advantage of Vision Pro's mixed reality capabilities?

Hi , We are using Scanning objects using Object Capture app provided by apple it was working fine but suddenly it started crashing while scanning the object with bounding box settings. Scanning app crash log Getting device log but showing different reasons for app crash. Device : iPad Pro / iPhone 15 Pro iOS : 17.4 Attaching the device log for crash, waiting for your response.