Vision Pinch 3D Object

Hello Guys,

I'm looking for how to look at the 3D object around me, pinch it (from far with eyes and 2 fingers, not touch them) and display a little information on top of it (like little text).

If you have the solution you will make me happy (and a bit less stupid too :) )

Cheers Mathis

Answered by Vision Pro Engineer in 800654022

Hi @Mathis06 ,

You'll need to play around with the position a little, but it would look like this:

import SwiftUI
import RealityKit
import RealityKitContent

struct ImmersiveView: View {
    let root: Entity = Entity()
    @State private var isEntityTapped: Bool = false
    
    var body: some View {
        RealityView { content, attachments in
            // Add the initial RealityKit content
            if let immersiveContentEntity = try? await Entity(named: "Immersive", in: realityKitContentBundle) {
                
                let leftSphere = immersiveContentEntity.findEntity(named: "Sphere_Left")!
                leftSphere.name = "Sphere_Left"
                leftSphere.generateCollisionShapes(recursive: true)
                leftSphere.components.set(HoverEffectComponent())
                leftSphere.components.set(InputTargetComponent())
                root.addChild(leftSphere)

                content.add(root)
                root.setPosition([0, 0, 0], relativeTo: nil)
            }           
 
        } update: { content, attachments in
            if let attachment = attachments.entity(for: "selectionText") {
                attachment.name = "selectionText"
                root.addChild(attachment)
                attachment.isEnabled = isEntityTapped
                // move it away from the root by a little
                attachment.setPosition([0,1,-0.75], relativeTo: root)
            }
        } attachments: {
            Attachment(id: "selectionText") {
                VStack {
                    Text("this item was selected")
                }
                .padding()
                .glassBackgroundEffect()
            }
        }
        .gesture(tapGesture)
    }
    
    
    // Build a tap gesture
    private var tapGesture: some Gesture {
        return SpatialTapGesture()
            .targetedToAnyEntity()
            .onEnded { _ in
                isEntityTapped.toggle()
            }
    }
}


#Preview(immersionStyle: .full) {
    ImmersiveView()
        .environment(AppModel())
}

You can use a spatial tap gesture to toggle some variable and then in turn enable the attachment. When enabled, it'll show up wherever it's been positioned. Make sure you have the hover effect and input target components set, otherwise the tap gesture won't work.

Best,

Sydney

code-block

> [```inline-code```](ia.yarsy .com)


Accepted Answer

Hi @Mathis06 ,

You'll need to play around with the position a little, but it would look like this:

import SwiftUI
import RealityKit
import RealityKitContent

struct ImmersiveView: View {
    let root: Entity = Entity()
    @State private var isEntityTapped: Bool = false
    
    var body: some View {
        RealityView { content, attachments in
            // Add the initial RealityKit content
            if let immersiveContentEntity = try? await Entity(named: "Immersive", in: realityKitContentBundle) {
                
                let leftSphere = immersiveContentEntity.findEntity(named: "Sphere_Left")!
                leftSphere.name = "Sphere_Left"
                leftSphere.generateCollisionShapes(recursive: true)
                leftSphere.components.set(HoverEffectComponent())
                leftSphere.components.set(InputTargetComponent())
                root.addChild(leftSphere)

                content.add(root)
                root.setPosition([0, 0, 0], relativeTo: nil)
            }           
 
        } update: { content, attachments in
            if let attachment = attachments.entity(for: "selectionText") {
                attachment.name = "selectionText"
                root.addChild(attachment)
                attachment.isEnabled = isEntityTapped
                // move it away from the root by a little
                attachment.setPosition([0,1,-0.75], relativeTo: root)
            }
        } attachments: {
            Attachment(id: "selectionText") {
                VStack {
                    Text("this item was selected")
                }
                .padding()
                .glassBackgroundEffect()
            }
        }
        .gesture(tapGesture)
    }
    
    
    // Build a tap gesture
    private var tapGesture: some Gesture {
        return SpatialTapGesture()
            .targetedToAnyEntity()
            .onEnded { _ in
                isEntityTapped.toggle()
            }
    }
}


#Preview(immersionStyle: .full) {
    ImmersiveView()
        .environment(AppModel())
}

You can use a spatial tap gesture to toggle some variable and then in turn enable the attachment. When enabled, it'll show up wherever it's been positioned. Make sure you have the hover effect and input target components set, otherwise the tap gesture won't work.

Best,

Sydney

Hello Sydney, that very good !!! it is possible to have the text window placed just over the ball, on top of it, on same place ?

Cheers and THANKS Mathis

Other (last) little question : how to know if the user pinch on no object (empty space) ?

hmm what are you trying to do? there's no API for that specifically.

In fact, I've a balls in my environment. I would like to have the user to pinch one or other and display it distance to another central ball. He can pinch one ball, one other and one other and have information one by one, and when he want to not have information, he pinch in empty space.

Oki, little question : The windows opened is nice and readable. But if I move it away of me on my object, the window and it content are "smaller". Normal! But on a normal glass window, when I move it away, the window size change but the contents is always readable (auto adaptation to have it readable) How can I do that? existing automatic system to set for a window to have this automatic ?

Cheers Mathis

Vision Pinch 3D Object
 
 
Q