Reality Composer Pro

RSS for tag

Leverage the all new Reality Composer Pro, designed to make it easy to preview and prepare 3D content for your visionOS apps

Posts under Reality Composer Pro tag

200 Posts
Sort by:

Post

Replies

Boosts

Views

Activity

Blender to Reality Composer Pro 2.0 to SwiftUI + RealityKit visionOS Best Practices
Hi, I'm very new to 3D and am currently porting a SwiftUI iOS app to visionOS 2.0. I saw WWDC24 feature Blender in multiple spatial videos, and have begun integrating Blender models and animations into my VisionOS app (I would also like to integrate skeletons and programmatic rigging, more on that later). I'm wondering if there are “Best Practices” for this workflow - from Blender to USD to RCP 2.0 to visionOS 2 in Xcode. I’ve cobbled together the following that has some obvious holes: I’ve been able to find some pre-rigged and pre-animated models online that can serve as a great starting point. As a reference, here is a free model from SketchFab - a simple rigged skeleton with 6 built in animations: https://sketchfab.com/3d-models/skeleton-character-low-poly-8856e0138f424d68a8e0b40e185951f6 When exporting to USD from Blender, I haven’t been able to export more than one animation per USD file. Is there a workflow to export multiple animations in a single USDC file, or is this just not possible? As a temporary workaround, here is a python script I’ve been using to loop through all Blender animations, and export a model for each animation: import bpy import os # Set the directory where you want to save the USD files output_directory = “/path/to/export” # Ensure the directory exists if not os.path.exists(output_directory): os.makedirs(output_directory) # Function to export current scene as USD def export_scene_as_usd(output_path, start_frame, end_frame): bpy.context.scene.frame_start = start_frame bpy.context.scene.frame_end = end_frame # Export the scene as a USD file bpy.ops.wm.usd_export( filepath=output_path, export_animation=True ) # Save the current scene name original_scene = bpy.context.scene.name # Iterate through each action and export it as a USD file for action in bpy.data.actions: # Create a new scene for each action bpy.context.window.scene = bpy.data.scenes[original_scene].copy() new_scene = bpy.context.scene # Link the action to all relevant objects for obj in new_scene.objects: if obj.animation_data is not None: obj.animation_data.action = action # Determine the frame range for the action start_frame, end_frame = action.frame_range # Export the scene as a USD file output_path = os.path.join(output_directory, f"{action.name}.usdc") export_scene_as_usd(output_path, int(start_frame), int(end_frame)) # Delete the temporary scene to free memory bpy.data.scenes.remove(new_scene) print("Export completed.") I have also been able to successfully export rigging armatures as a single Skeleton - each “bone” showing getting imported into Reality Composer Pro 2.0 when exporting/importing manually. I would like to have all of these animations available in a single scene to be used in a RealityView in visionOS - so I have placed all animation models in a RCP scene and created named Timeline Action animations for each, showing the correct model and hiding the rest when triggering specific animations. I apply materials/textures to each so they appear the same, using Shader Graph. Then in SwiftUI I use notifications (as shown here - https://forums.developer.apple.com/forums/thread/756978) to trigger each RCP Timeline Action animation from code. Two questions: Is there a better way than to have multiple models of the same skeleton - each with a different animation - in a scene to be able to trigger multiple animations? Or would this require recreating Blender animations using skeleton rigging and keyframes from within RCP Timelines? If I want to programmatically create custom animations and move parts of the skeleton/armatures - do I need to do this by defining custom components in RCP, using IKRig and define movement of each of the “bones” in Xcode? I’m looking for any tips/tricks/workflow from experienced engineers or 3D artists that can create a more efficient/optimized workflow using Blender, USD, RCP 2 and visionOS 2 with SwiftUI. Thanks so much, I appreciate any help! I am very excited about all the new tools that keep evolving to make spatial apps really fun to build!
0
0
126
6d
Loading USDZ file take almost 30s
I'm having issue with loading 1.2GB USDZ file with visionOS Here's the details the file is download via backend api file is download to document directory FileManager.default.url(for: .documentDirectory, in: .userDomainMask, appropriateFor: nil, create: true) when loading the asset, it took almost 30s to load the asset Loaded usd((extension in RealityFoundation):RealityKit.Entity.LoadStatistics.USDLoader.rio) in 29.24642503261566 seconds loading asset code let model = try await Entity(contentsOf: assetUrl) USDZ file is exported from RealityComposerPro Did I make any mistake on the flow or is there any other approach to decrease the loading time?
1
0
166
1w
Apply sprite scene as material in Reality Composer pro
Hello, I’m trying to move my app into vision OS, my app is used for pilot to study the airplane system, is a 3d airplane cockpit build with scene kit and I use sprite scene to animate the cockpit instruments . Scenekit allow to apply as material a sprite scene , so I could animate easy all the different instruments and indication there, but I can’t find this option on reality compose pro , is this possible? any suggestions I can look into to animate and simulate instruments.
2
0
144
1d
Add a modifier to a single model in the Reality Composer Pro scene
We can add many models in the Reality Composer Pro scene, but when I use RealityView to display and add modifiers in SwiftUI, the modifiers will have Effect, and I don't want to do this. I hope this modifier will be valid for a single model in the Reality Composer Pro scenario. May I ask how to add modifiers to a single model in the Reality Composer Pro scene?
1
0
135
6d
Object Tracking with RealtyView
When I wanted to call the Reality Composer Pro scene containing Object Tracking, I tried the following code: RealityView { content in if let model = try? await Entity(named: "Scene", in: realityKitContentBundle) { content.add(model) } } Obviously, this is wrong. We need to add some configurations that can enable Object Tracking to Reality View. What do we need to add? Note:I have seen https://developer.apple.com/videos/play/wwdc2024/10101/, but I don't know much about it.
2
0
225
1w
custom usda question
hello i wanna play mp4 file in VideoMaterial avPlayer. so first i make to use reality composer pro. I created matterial using the sphere provided by default in Reality Composer Pro and exported it to usdz. and when i play mp4 file in sphere matterial, it's good play But i wanna custom created matterial (ex. shaper3d create 3d modeling) not good play. i make custom created matterial - it's curved matterial curved matterial in shaper3d and exported it to usdz. curved matterial in Reality Composer Pro Scene and exported it to usdz. when i play mp4 file in curved matterial, it's not good play -> not adjust screen play How can I adjust and display the video in a custom usda file?
0
0
113
1w
Reality Composer Pro inline documentation comments
The WWDC24 video "Build a spatial drawing app with RealityKit" https://developer.apple.com/wwdc24/10104 at 12:04 includes a slide showing a Reality Composer Pro shader graph that features wonderful inline documentation comment boxes: Are shader graph inline comments a new feature that Reality Composer Pro supports? This would be extraordinarily useful, as complex shader graphs can be challenging to decipher. If so, how are inline shader graph comments created in Reality Composer Pro?
1
0
233
2w
Unable to create PhysicsJoint using Entity's Geometric Pin
Hello, I'm trying to attach one entity to another entity via the new PhysicsFixedJoint. I have a usdz that contains a skeletal pose which expose the joints as pins as desired. However the when I access the pin, it is returning a GeometricPin, instead of an EntityGeometricPin as you would expect. I can't use the returned GeometricPin to create the joint. Am I missing something? Shouldn't access the Entity's pins object return EntityGeometricPins instead of GeometricPin? Here is the code sample: var body: some View { RealityView { content in if let scene = try? await Entity(named: "Scene", in: untitledBundle) { content.add(scene) let attack = try! Entity.load(named: "Attack01_SingleSword") let anchor = scene.findEntity(named: "Root") anchor?.addChild(attack) let sword = try! Entity.load(named: "OHS08_Sword") anchor?.addChild(sword) if let swordEntity = findModelComponentEntity(entity: sword) { let swordPin = swordEntity.pins.set( named: "test", position: SIMD3<Float>.zero ) if let attackEntity = findModelComponentEntity(entity: attack) { let attackPin = attackEntity.pins["root/pelvis/spine_01/spine_02/spine_03/clavicle_r/upperarm_r/lowerarm_r/hand_r/weapon_r"]! // This is returning GeomtricPin instead of the EntityGeometricPin that the "pins" object contains let joint = PhysicsFixedJoint( pin0: swordPin, pin1: attackPin // This is a compile error since it is not an EntityGeometricPin type ) try! joint.addToSimulation() } } } } }
1
0
194
5d
Drag Gesture in Immersive Spaces with Reality Kit
I've been trying to get the drag gesture up and running so I can move my 3D model around in my immersive space, but for some reason I am not able to move it around. The model shows up in my visionOS 1.0 Simulator, but I can't seem to get it to move around. Would love some help with this and some resources too that would be helpful. Here's a snippet of my Reality View code import SwiftUI import RealityKit import RealityKitContent struct GearRealityView: View { static var modelEntity = Entity() var body: some View { RealityView { content in if let model = try? await Entity(named: "LandingGear", in: realityKitContentBundle) { GearRealityView.modelEntity = model content.add(model) } }.gesture( DragGesture() .targetedToEntity(GearRealityView.modelEntity) .onChanged({ value in GearRealityView.modelEntity.position = value.convert(value.location3D, from: .local, to: GearRealityView.modelEntity.parent!) }) ) } }
2
0
203
2w
Object Tracking with Rotation of Objects
Hey, In the "Explore object tracking for visionOS" session we explore how a Globe can be tracked, and objects can be anchored to various positions. My question is if the physical Globe is rotated, will the anchored objects also respond to this in real-time? I would like to overlap a virtual map on top of a physical globe, so when the user rotates the physical globe, the virtual map also seamlessly responds. Is this possible using Object Tracking? Thanks
3
1
211
2w
Object anchor not working with ARKit in iOS
With WWDC 24, I was excited to see that apple is bringing their APIs from Vision OS to iOS. I tried using the Object Anchoring component in Reality Composer Pro. Which this works with a Vision Pro, it looks like the entity will spawn at origin if we run the same on iOS and the object anchoring doesn't seem to work. Is this intended? Below is how I'm doing this. I added an Anchoring component and added the .referenceObject file I trained using CreateML. This is the code I'm using to load this scene in. // GrootView.swift // ARTest-New // // Created by Sravan Karuturi on 6/10/24. // import SwiftUI import RealityKit import Box struct GrootView: View { @StateObject private var grootVM = GrootViewModel() @State private var ent: Entity? = nil @State var anchor: Entity? = nil @State var wallAnchor: Entity? = nil @State var floorAnchor: Entity? = nil var body: some View { RealityView{ content in #if os(iOS) await content.setupWorldTracking() content.camera = .worldTracking #endif ent = try? await Entity(named: "Box", in: boxBundle) print(ent?.children) anchor = ent?.findEntity(named: "ObjectAnchor") wallAnchor = ent?.findEntity(named: "WallAnchor") floorAnchor = ent?.findEntity(named: "FloorAnchor") let updateSum = content.subscribe(to: SceneEvents.Update.self){ event in if let anc = anchor, anc.isAnchored { print("Found Item") } if let anc = floorAnchor, anc.isAnchored { print("Found Floor") } if let anc = wallAnchor, anc.isAnchored { print("Wall Anchor") } } content.add(ent!) } } } #Preview { GrootView() } While, something similar seems to work on visionOS, the same doesn't seem to work with iOS. When I run this app, we see all the children and the Found Item is printed constantly even when we don't have the item in the scene. Not really sure if this is just not supported yet on iOS ( I really hope that's not the case ) or if I messed up something somehow
2
1
221
2w
How to ship SDK RealityKit entity components that can be using and applied within a customer's application?
We deliver an SDK that enables rich spatial computing experiences. We want to enable our customers to develop apps using Swift or RealityComposer Pro. Composer allows the creation of custom components from the add components button in the inspector panel. These source files are dropped into the RealityComposer package and directory. We would like to be able to have our customers import our SDK components into their applications RealityComposer package and have our components be visible to be applied by our customer into their scene compositions. How can we achieve this? We believe this will lead to a risk ecosystem components extensions for RealityComposer Pro.
4
0
261
1w