How to Drag Objects Separately with Both Hands

I would like to drag two different objects simultaneously using each hand.

In the following session (6:44), it was mentioned that such an implementation could be achieved using SpatialEventGesture(): https://developer.apple.com/jp/videos/play/wwdc2024/10094/

However, since targetedEntity.location3D obtained from SpatialEventGesture is of type Point3D, I'm having trouble converting it for moving objects. It seems like the convert method in the protocol linked below could be used for this conversion, but I'm not quite sure how to implement it: https://developer.apple.com/documentation/realitykit/realitycoordinatespaceconverting/

How should I go about converting the coordinates? Additionally, is it even possible to drag different objects with each hand?

 .gesture(
            SpatialEventGesture()
                .onChanged { events in
                    for event in events {
                        if event.phase == .active {
                            switch event.kind {
                            case .indirectPinch:
                                if (event.targetedEntity == cube1){
                                    let pos = RealityViewContent.convert(event.location3D, from: .local, to: .scene) //This Doesn't work
                                    dragCube(pos, for: cube1)
                                }
                            case .touch, .directPinch, .pointer:
                                break;
                            @unknown default:
                                print("unknown default")
                            }
                        }
                    }
                }
        )
Answered by Vision Pro Engineer in 802113022

Hi @GentaG

Could you try modifying your SpatialEventGesture to use targetedToAnyEntity()? That should give you access to the convert() method you need to convert location3D to RealityKit coordinate space.

.gesture(
            SpatialEventGesture()
                .targetedToAnyEntity()
                .onChanged { events in
                    for value in events.gestureValue {
                        let position = events.convert(value.location3D, from: .local, to: .scene)
                        value.targetedEntity?.position = position
                    }
                }
        )

Let me know if you run into any more issues!

(Given that the implementation exists in the App Store game Loóna: Cozy Puzzle Games, I believe it shouldn't be impossible...)

Accepted Answer

Hi @GentaG

Could you try modifying your SpatialEventGesture to use targetedToAnyEntity()? That should give you access to the convert() method you need to convert location3D to RealityKit coordinate space.

.gesture(
            SpatialEventGesture()
                .targetedToAnyEntity()
                .onChanged { events in
                    for value in events.gestureValue {
                        let position = events.convert(value.location3D, from: .local, to: .scene)
                        value.targetedEntity?.position = position
                    }
                }
        )

Let me know if you run into any more issues!

How to Drag Objects Separately with Both Hands
 
 
Q