Get Window(Group) position / Track Window? (VisionOS)

Question

XWDev OP

Created Aug ’24

Replies 3

Boosts 0

Participants 2

Hi guys,

I'm currently working on a Head Tracking application for visionOS and was wondering if there are any properties or ways to access the position of the app window in an immersive space? I was planning to somehow determine if the window is/is not within the AVP's orientation (through queryDeviceAnchor()) or "visible space". Or is there a way to access a property or data that tells me if the app window is within the user's AVP orientation or not if e.g. the user is turning around having the window behind the back?

I would be extremely thankful for any helpful input!

import SwiftUI


@main
struct HeadTrackingApp: App {
    
    init() {
        HeadTrackingSystem.registerSystem()
    }
    
    var body: some Scene {
        WindowGroup {          // Basically getting spatial coordinates of this
            ContentView()
       
        }
        
        
        ImmersiveSpace(id: "appSpace") {
            
        }
    }
}

Answered by Vision Pro Engineer in 799311022

What I was hoping for would be something that I can access that could tell me if the window of the app is either within the visible space of the user

The docs explain this in more detail, but ScenePhase depends on where you place the @Environment(\.scenePhase) property. If you read this within the App struct then you will get an aggregate value which informs you if the application is closed. If you read this within a View struct then it will tell you when the view's scene isn’t currently visible in the UI.

From my experience, the system typically transitions ScenePhases of views to the .background state after about a minute of being out of view from the user. If you're looking for an immediate indicator of the user not having the window within view this probably won't work for your situation. If you'd like a more general case of the user no longer has been seeing the window this should work great.

Boost

Answer 1

Vision Pro Engineer OP

Apple

Aug ’24

Hey @XWDev,

The first thing that comes to mind is using the ScenePhase property. Depending on your goal this might be a solution. This works in all scene types and will inform you on the scene’s operational state. You could read this from your view and be informed when the window transitions to the .background state, which indicates that the scene isn’t currently visible in the UI. Take a look at the documentation for more details.

However, if doesn't fit your requirements and you need to know the precise location of a window you can use a GeometryReader3D to get the position in the 3D space using .transform(in:). This however will only work in an immersive space.

.onChange(of: proxy.transform(in: .immersiveSpace) ?? .identity) { _, transform in
    print("transform: \(transform)")
}

Let me know if you have any questions,

Michael

0

Answer 2

XWDev OP

Aug ’24

Hi Michael @Vision Pro Engineer

First of all, thank you for your quick answer. I appreciate your effort!

I think it goes in the right direction. To maybe give you a better idea of what I aim to achieve. The goal is basically to have a state of "distraction," which is the case if the user won't have the app window in front of it's vision/orientation of the AVP.

Unfortunately, as far as I understood, ScenePhase only tells me if, let's say, the user closes the app window, it is now in the background, and if reopening the app, it is active, etc. What I was hoping for would be something that I can access that could tell me if the window of the app is either within the visible space of the user (e.g., I just started the app, and the window is in front of me and my AVP is also directed towards the window) or not (e.g., my window is still at the initial spawn space when I started the app but I've turned around so the window would be behind my back, not visible anymore for me).

Furthermore, now that I'm able to track the spatial positioning of the window as you suggested, would there be a way to either be able to track if the window is within the space of the AVP's orientation (e.g., as long as I have the window in my vision or even only a corner of it) OR attach the AVP's originFromAnchorTransform property to the window, so that eventually the app launches and the AVP's orientation rotation data angles are set to 0, having the window basically as the center of orientation, and then making the AVP's orientation depending on the window, speaking of if I would move the window and then look perfectly centered at the new positioned window, I would be back at the 0 values.

0

Answer 3

Vision Pro Engineer OP

Apple

Aug ’24

Accepted Answer

What I was hoping for would be something that I can access that could tell me if the window of the app is either within the visible space of the user

The docs explain this in more detail, but ScenePhase depends on where you place the @Environment(\.scenePhase) property. If you read this within the App struct then you will get an aggregate value which informs you if the application is closed. If you read this within a View struct then it will tell you when the view's scene isn’t currently visible in the UI.

From my experience, the system typically transitions ScenePhases of views to the .background state after about a minute of being out of view from the user. If you're looking for an immediate indicator of the user not having the window within view this probably won't work for your situation. If you'd like a more general case of the user no longer has been seeing the window this should work great.

1