How to add gestures to objects inside other objects

I have a scene with multiple RealityKit entities. There is a blue cube which I want to rotate along with all of its children (it's partly transparent).

Inside the cube are a number of child entities (red) that I want to tap.

The cube and red objects all have collision components as is required for gestures to work.

If I want to rotate the blue cube, and also tap the red objects I can't do this as the blue cube's collision component intercepts the taps.

Is there a way of accomplishing what I want?

I'm targeting visionOS 2, and my scene is in a volume.

Answered by Vision Pro Engineer in 798014022

Hi @kemalenver

This is possible. ModelSortGroupComponent gives you control over the rendering order for entities. It's typically used to control how entities appear, but you can use it to control hit test order. Here's how you can use it to achieve your desired outcome.

RealityView { content in
    let boxSize:Float = 0.4
    
    // Create an entity the size of the box to be used solely as an input target.
    // Note: The gesture should rotate this entity instead of the box.
    let container = Entity()
    container.name = "container"
    let collisionShape = ShapeResource.generateBox(size: [boxSize, boxSize, boxSize])
    container.components.set(CollisionComponent(shapes: [collisionShape]))
    container.components.set(InputTargetComponent())
    container.position = [0, 1.4, -1]

    // Create a box with no CollisionComponent and no InputTargetComponent.
    // This is used solely for appearances.
    let box = ModelEntity(mesh: .generateBox(size: 0.4), materials: [SimpleMaterial(color: .red.withAlphaComponent(0.5), isMetallic: false)])
    
    // Create the spheres with collision shapes.
    let sphere1 = ModelEntity(mesh: .generateSphere(radius: 0.05), materials: [SimpleMaterial(color: .green, isMetallic: false)])
    sphere1.name = "sphere1"
    sphere1.generateCollisionShapes(recursive: false)
    
    let sphere2 = ModelEntity(mesh: .generateSphere(radius: 0.05), materials: [SimpleMaterial(color: .green, isMetallic: false)])
    sphere2.name = "sphere2"
    sphere2.position.x = 0.11
    sphere2.generateCollisionShapes(recursive: false)
    
    // Specify the rendering order
    let group = ModelSortGroup(depthPass: .postPass)

    // Render the container first so it receives gestures last.
    container.components.set(ModelSortGroupComponent(
        group: group,
        order: 1
    ))
    
    // Render the spheres next. They will receive gestures first.
    sphere1.components.set(ModelSortGroupComponent(
        group: group,
        order: 2
    ))
    
    sphere2.components.set(ModelSortGroupComponent(
        group: group,
        order: 2
    ))

    // Render the box last so its material partially obscures the spheres.
    // Since it has no InputTargetComponent, it will not intercept gestures.
    box.components.set(ModelSortGroupComponent(
        group: group,
        order: 3
    ))

    container.addChild(sphere2)
    container.addChild(sphere1)
    container.addChild(box)
    
    content.add(container)
}
.gesture(TapGesture().targetedToAnyEntity().onEnded { gesture in
    print("got tap for", gesture.entity.name)
})
Accepted Answer

Hi @kemalenver

This is possible. ModelSortGroupComponent gives you control over the rendering order for entities. It's typically used to control how entities appear, but you can use it to control hit test order. Here's how you can use it to achieve your desired outcome.

RealityView { content in
    let boxSize:Float = 0.4
    
    // Create an entity the size of the box to be used solely as an input target.
    // Note: The gesture should rotate this entity instead of the box.
    let container = Entity()
    container.name = "container"
    let collisionShape = ShapeResource.generateBox(size: [boxSize, boxSize, boxSize])
    container.components.set(CollisionComponent(shapes: [collisionShape]))
    container.components.set(InputTargetComponent())
    container.position = [0, 1.4, -1]

    // Create a box with no CollisionComponent and no InputTargetComponent.
    // This is used solely for appearances.
    let box = ModelEntity(mesh: .generateBox(size: 0.4), materials: [SimpleMaterial(color: .red.withAlphaComponent(0.5), isMetallic: false)])
    
    // Create the spheres with collision shapes.
    let sphere1 = ModelEntity(mesh: .generateSphere(radius: 0.05), materials: [SimpleMaterial(color: .green, isMetallic: false)])
    sphere1.name = "sphere1"
    sphere1.generateCollisionShapes(recursive: false)
    
    let sphere2 = ModelEntity(mesh: .generateSphere(radius: 0.05), materials: [SimpleMaterial(color: .green, isMetallic: false)])
    sphere2.name = "sphere2"
    sphere2.position.x = 0.11
    sphere2.generateCollisionShapes(recursive: false)
    
    // Specify the rendering order
    let group = ModelSortGroup(depthPass: .postPass)

    // Render the container first so it receives gestures last.
    container.components.set(ModelSortGroupComponent(
        group: group,
        order: 1
    ))
    
    // Render the spheres next. They will receive gestures first.
    sphere1.components.set(ModelSortGroupComponent(
        group: group,
        order: 2
    ))
    
    sphere2.components.set(ModelSortGroupComponent(
        group: group,
        order: 2
    ))

    // Render the box last so its material partially obscures the spheres.
    // Since it has no InputTargetComponent, it will not intercept gestures.
    box.components.set(ModelSortGroupComponent(
        group: group,
        order: 3
    ))

    container.addChild(sphere2)
    container.addChild(sphere1)
    container.addChild(box)
    
    content.add(container)
}
.gesture(TapGesture().targetedToAnyEntity().onEnded { gesture in
    print("got tap for", gesture.entity.name)
})
How to add gestures to objects inside other objects
 
 
Q