How to ship SDK RealityKit entity components that can be using and applied within a customer's application?

We deliver an SDK that enables rich spatial computing experiences.
We want to enable our customers to develop apps using Swift or RealityComposer Pro.

Composer allows the creation of custom components from the add components button in the inspector panel. These source files are dropped into the RealityComposer package and directory.

We would like to be able to have our customers import our SDK components into their applications RealityComposer package and have our components be visible to be applied by our customer into their scene compositions.

How can we achieve this? We believe this will lead to a risk ecosystem components extensions for RealityComposer Pro.

Answered by Vision Pro Engineer in 790465022

Here's a type safe approach you can follow to release a framework that can be used in two steps. In the consuming application:

  • Create and register a component conforming to a protocol provided by the framework.
  • Register the system provided by the framework.

Let's walk through how to accomplish this. Pretend you aim to ship BubbleSystem / BubbleComponent.

Create a Framework, let's name it BubbleFramework, containing a Protocol (which extends Component) and a generic System to leverages that protocol.

Component Protocol

public protocol BubbleComponentProtocol : Component {
    var radius:Float {get}
}

System

public struct BubbleSystem<T:BubbleComponentProtocol>: System {
    let query = EntityQuery(where: .has(T.self))
    
    public init(scene: RealityKit.Scene) {
    }

    public func update(context: SceneUpdateContext) {
        let bubbles = context.entities(matching: self.query,
                                       updatingSystemWhen: .rendering)

        for bubble in bubbles {
            guard let component = bubble.components[T.self] else {continue}
            // do things with the component...
            }
        }
    }
}

Next, create a new application and add BubbleFramework to the app target's "Frameworks, Libraries, and Embedded Content". Then, create a component named BubbleComponent that conforms to BubbleComponentProtocol in the RealityKitContentPackage associated with the (newly created) application.

import BubbleFramework

public struct BubbleComponent: BubbleComponentProtocol, Component, Codable {
    public var radius:Float = 0.1
    
    public init() {
    }
}

Note:

  • The RealityKitContentPackage is created for you when you create a visionOS application, but if you are targeting other platforms you will need to create it.

  • You must explicitly extend Component, Codable or the properties won't show up in the Reality Composer Pro inspector.

Finally register the BubbleComponent and BubbleSystem

BubbleComponent()
BubbleSystem<BubbleComponent>.registerSystem()

Profit!

Here's a type safe approach you can follow to release a framework that can be used in two steps. In the consuming application:

  • Create and register a component conforming to a protocol provided by the framework.
  • Register the system provided by the framework.

Let's walk through how to accomplish this. Pretend you aim to ship BubbleSystem / BubbleComponent.

Create a Framework, let's name it BubbleFramework, containing a Protocol (which extends Component) and a generic System to leverages that protocol.

Component Protocol

public protocol BubbleComponentProtocol : Component {
    var radius:Float {get}
}

System

public struct BubbleSystem<T:BubbleComponentProtocol>: System {
    let query = EntityQuery(where: .has(T.self))
    
    public init(scene: RealityKit.Scene) {
    }

    public func update(context: SceneUpdateContext) {
        let bubbles = context.entities(matching: self.query,
                                       updatingSystemWhen: .rendering)

        for bubble in bubbles {
            guard let component = bubble.components[T.self] else {continue}
            // do things with the component...
            }
        }
    }
}

Next, create a new application and add BubbleFramework to the app target's "Frameworks, Libraries, and Embedded Content". Then, create a component named BubbleComponent that conforms to BubbleComponentProtocol in the RealityKitContentPackage associated with the (newly created) application.

import BubbleFramework

public struct BubbleComponent: BubbleComponentProtocol, Component, Codable {
    public var radius:Float = 0.1
    
    public init() {
    }
}

Note:

  • The RealityKitContentPackage is created for you when you create a visionOS application, but if you are targeting other platforms you will need to create it.

  • You must explicitly extend Component, Codable or the properties won't show up in the Reality Composer Pro inspector.

Finally register the BubbleComponent and BubbleSystem

BubbleComponent()
BubbleSystem<BubbleComponent>.registerSystem()

Profit!

Thank you for the prompt response.

I made this diagram. I show the Component being defined in my SDK and shipped in source format. The customer would them copy the component source into their app's RCP sources directory.

I will test this out soon.

Two Questions: First, is there a precedent for some SDK files to be shipped as source? (Yes, I guess this is like sample code.) Any recomended practices here to make this feel natural to developers?

Second, can we make a feature request to allow package references, like my framework, to be added to an RCP package and have all the valid public components in the framework added to the RCP components UI?

This would reduce the manual steps of app developers keeping framework components up to date.

More Profit!

Thanks for the Diagram! One nit. The BubbleComponent should be in the Reality Composer Pro package or RCP won't display it in the inspector.

I'm not aware of any precedent for shipping source files. Sample code/app (maybe as part of a 'getting started guide' in your docs) sounds like a good choice.

Yes! Feature requests are always welcome. Please file it using Feedback Assistant.

Accepted Answer

Adding updated diagram to clearly show the BubbleComponent Swift file is copied into the RCP package for the app.

I hope this info i s helpful to others in the future.

How to ship SDK RealityKit entity components that can be using and applied within a customer's application?
 
 
Q