Hey there,
Just tried the visionOS 2.2 ultra-wide remote desktop feature and it's just a-m-a-z-i-n-g!!
I was curious if there's an API that we can use to setup our windows in a similar fashion? (curved + ultra wide).
Thank you!
General
RSS for tagDiscuss Spatial Computing on Apple Platforms.
Post
Replies
Boosts
Views
Activity
My visionOS app uses an immersive view. If the app encounters an error, I want to present an alert.
I tried in a demo app to present such an alert, but it is not shown. Nearly the same code on iOS presents an alert window.
Here is my demo code, based on Apple's Immersive Environment App template:
import SwiftUI
import RealityKit
import RealityKitContent
struct ErrorInfo: LocalizedError, Equatable {
var errorDescription: String?
var failureReason: String?
}
struct ImmersiveView: View {
@State private var presentAlert = false
let error = ErrorInfo(
errorDescription: "My error",
failureReason: "No reason"
)
var body: some View {
RealityView { content, attachments in
let mesh = MeshResource.generateBox(width: 1.0, height: 0.05, depth: 1.0)
var material = UnlitMaterial()
material.color.tint = .red
let boardEntity = ModelEntity(mesh: mesh, materials: [material])
boardEntity.transform.translation = [0, 0, -3]
content.add(boardEntity)
}
update: { content, attachments in
// …
}
attachments: {
// …
}
.onAppear {
presentAlert = true
}
.alert(
isPresented: $presentAlert,
error: error,
actions: { error in
}, message: { error in
Text(error.failureReason!)
})
}
}
Since I cannot see any alert, is something wrong with my code? How should an alert be presented in immersive space?
Hi!
I wanna know if it's possible mirroring a Vision Pro to other Vision Pros.
If it's possible, how do I work on it? Can I get some hints?
Hi!
I'm planning to make visionOS multiplayer app for people in same space(a room). I wanna know that if it's possible to use TabletopKit, Group Activities to create an app that becomes multiplayer(synchronize) with the people who are using it as soon as the app is opened without using SharePlay.
We have discovered that our UIViewRepresentable view isn't being dismantled after its window is dismissed via dismissWindow().
This seems to result in a leak of our custom Coordinator class. Every time the user opens a new window, a new Coordinator is created; if the user then dismisses the window manually, or we dismiss it programmatically, the Coordinator remains in memory with no way to destroy it.
Is this expected behavior? How can we be sure to clean up our Coordinator when the view's window is closed? Thanks.
In some cases, we will put the Apple Vision Pro back on for a short period of time after taking it off, such as less than 1 minute, and want to keep it activte when we take it off so that it can continue to work seamlessly when we put it on again.
The current Apple Vision Pro turns off the display and goes to sleep whenever it is been taken off. This feature is also explained in the user guide, to save power, for safety, etc.
However, we want a seamless experience!
Is it possible to have a screen saver like macOS or iOS, where we can set our own delay time to go sleep?
Or is there any API that can be called to prevent going to sleep?
I want to pursue for a project involving 3D VR visualisation. I would like to know if there is a 3D stereoscopic camrea setup that is able to connect straight to Apple Vision Pro display yet. Of course, with no compatibility issues with MV-HEVC. Any recommendation is appreciated.
We were having an issue wrb the system rotate and scale gestures (two-handed gestures / RotateGesture3D and MagnifyGesture) were extremely difficult to register (make work) in the visionOS simulator.
The solution we found was to:
Launch your app in the simulator
Move the pointer on top of the 3D object for which you are testing rotation and scaling gestures.
Press and hold the Option key to display touch points (ie: the two-handed gesture points).
While maintaining the option key pressed, release the pointer and re-enable it again. I am using a track pad with tap-to-click enabled and three-finger to drag enabled in accessibility, so "release the pointer and re-enable it again" translates simply to removing the three finger and placing them again on the trackpad.
If you have maintained the option key pressed, then you should now be able to rotate and scale the 3D object.
Context if you are interested:
Our issue was also occurring in Apple's own sample project relating to gestures "Transforming RealityKit entities using gestures", at below link.
On Apple's article "Interacting with your app in the visionOS simulator" at the below link, for two-handed gestures it states "Press and hold the Option key to display touch points. Move the pointer while pressing the Option key to change the distance between the touch points. Move the pointer and hold the Shift and Option keys to reposition the touch points."
This simply did not work anymore for rotation and scaling gestures.
These gestures used to be a lot more responsive in Sonoma. Either the article should be updated to what I described above, or there is an issue. Our colleague who is using macOS Sonoma 14.6.1 with the latest release of Xcode is not having these issues.
Here is the list of configurations (troubleshooting we tried!) where it is difficult to achieve rotation and scaling gestures in the visionOS simulator:
macOS Sequoia 16.1 Beta, Xcode 16.1 RC w visionOS 2.1
macOS Sequoia 16.1 Beta, Xcode 16.1 RC w visionOS 2.0
macOS Sequoia 16.1 Beta, Xcode 16.2 Beta 1 w visionOS 2.1
macOS Sequoia 16.1 Beta, Xcode 16.2 Beta 1 w visionOS 2.0
macOS Sequoia 16.1 Beta, remove all Xcodes and installed the build from AppStore (Xcode 16.1)
macOS Sequoia 16.1 Beta, Xcode 16.0 w visionOS 2.0
completely wiped out, and reset entire development machine, re-installed latest releases of sequoia (15.1) and xcode (15.1))
Throughout these troubleshooting I often:
restarted both xcode and sim
erased all derived data
erased all contents and settings from sims
performed fresh git clones
None of the above worked, only the workaround described above works atm. As you can maybe deduce, it was very time consuming to find the workaround, we also wasted some development effort thinking our gesture development was no-good.
Hopefully this will help other devs.
Article Link:
https://developer.apple.com/documentation/xcode/interacting-with-your-app-in-the-visionos-simulator
Gesture sample project link:
https://developer.apple.com/documentation/realitykit/transforming-realitykit-entities-with-gestures
We developed an app for VisionOS 2.0 Beta in XCode 16 Beta. The development was done in the beta versions since we needed features for our app which were not available in VisionOS 1.0, which was the most recent stable release at the time we developed the app. The app was fully functional on our AVP running VisionOS 2.0 Beta Version 5, and we never had any errors. We did not publish the app to the app store since we are a research lab using the app for teleoperating a custom robot.
Last week, we upgraded the AVP from VisionOS 2.0 Beta Version 5 to VisionOS 2.0 (stable release). Unfortunately, once we upgraded to 2.0, we began to have an issue with the app. While the app is running, at seemingly random times, without any new functionality being used within the app (no new buttons being pressed, etc), we encounter the following console error:
assertion failure: 'index < m_size' (operator[]:line 1011) Index out of range. index = 18446744073709551615, size = 0
We could re-upload the app to the AVP and successfully operate the app for several minutes until the same error occurred again. We thought to use Apple Configurator to flash VisionOS 2.0 Beta 5 to the AVP since the error wasn't happening on the previous firmware, but we were unable to flash a beta version of VisionOS via Apple Configurator, so we simply performed a factory reset of the device (on VisionOS2.0, by pressing restore in Apple Configurator with the AVP connected via the developer strap) to see if this might fix the issue.
After doing a factory reset, we thought the console error completely went away. We were able to operate the app for ~3 hours on Sunday with no issues. Then, yesterday (Monday) we operated the app for another 2 hours, and at the very end of using the app, it crashed with the same error. We re-uploaded the app with XCode, and the error occurred again after about 20 mins of using the app. This cycle repeated, and every time we re-uploaded the app, the time it took for the error to occur decreased, until we uploaded the app and the error occurred in <20 seconds.
We decided to test our hypothesis by upgrading VisionOS to 2.1 and using XCode 16. Similarly, we were able to run the app on the AVP for 2 hours, then the error occurred. The next time we ran the app, the error occurred within 20 minutes, then after reloading, occurred within 5 mins, then 2 mins, etc.
We are pretty stumped on why the app would work after a factory reset or a firmware upgrade for hours, then fail faster and faster every time we re-upload the app from Xcode. We are not experienced in debugging Swift and ObjC, so we wanted to inquire if this is an issue that you have ran into before to point us in the right direction. We think that it could be a problem with the cached memory that persists on the device across uploads from Xcode, but that's the extent of our understanding.
P.S., we also experienced this error during some of the app failures, but the one above is the most common:
assertion failure: Index out of range (operatorl]:line
858) index = 576460752303423487, max = 1
Hello everyone,
I’m developing an app for Apple Vision Pro, and I’m trying to retrieve motion data updates aligned to magnetic north by using the following method:
startDeviceMotionUpdates(using: .xMagneticNorthZVertical, to: .main) { ... }
The goal is to get motion data oriented to magnetic north while an ImmersiveSpace() with an immersiveStyle set to .mixed is active. However, with this setup, I receive no updates at all.
If I switch to:
startDeviceMotionUpdates(using: .xArbitraryZVertical, to: .main) { ... }
or
startDeviceMotionUpdates(to: .main) { ... }
then I do receive data, but it’s not aligned as required (I specifically need .xMagneticNorthZVertical).
Has anyone experienced a similar issue, or does anyone know how to enable updates aligned to magnetic north in this configuration?
Thanks in advance for any insights!
SDK: VisionOS 2.0
I'm using hand tracking to detect collisions between fingertips and entities that I have placed in the scene. I'm using the .mixed environment.
However, I want to detect when a fingertip touches a real-world object such as a wall.
No matter what I try, I can't get the collision to fire. I'm using the SceneReconstructionProvider to give me world meshes, which I use to create ModelEntity objects to which I add a CollisionComponent with the shape of the object.
I can render the meshes just fine, but nothing I do seems to allow collisions to work.
Surely this is possible, what am I missing?
I’ve updated to iOS 18.1 public beta when it released and I got the problem when I was in public beta 4 to rc and the problem never fixed with the time limit when I ask for more time it never give me more time it’s just says waiting for parent approval I have this problem until ios 18.1 RC can u fix it I’m on iPhone 11 pro max
I am working on adding synchronized physical properties to EntityEquipment in TableTopKit, allowing seamless coordination during GroupActivities sessions between players.
Current Approach and Limitations
I have tried setting EntityEquipment's state to DieState and treating it as a TossableRepresentation object. This approach achieves basic physical properties synchronized across players. However, it has several limitations:
No Collision Detection Between Dice: Multiple dice do not collide with each other.
Shape Limitations: Custom shapes, like parallelepipeds, cannot be configured.
Below is my existing code for Base Entity Equipment without physical properties:
struct CubeWithPhysics: EntityEquipment {
let id: ID
let entity: Entity
var initialState: BaseEquipmentState
init(id: ID, entity: Entity) {
self.id = id
self.entity = entity
initialState = .init(parentID: .tableID, pose: .init(position: .zero, rotation: .zero), entity: self.entity)
}
}
I’d appreciate any guidance on the recommended approach to adding synchronized physical properties to EntityEquipment.
I'm writing code using ObjectAnchor for Vision OS. If an object is tracked, and then becomes not visible (either because the user looked in a different direction, or because the tracked object was occluded by another object), it is still tracked and you get anchor updates (e.g., object permanence).
For my application, it would be very helpful if I could determine if the object is currently being observed, or it is not currently observed and just assumed to be in the same location as seen previously.
ObjectAnchor.isTracked just seems to indicate whether it is getting anchor updates. I don't see anything in the ObjectAnchor or AnchorUpdate that would allow me to determine if the object is currently observed. Does anyone know of a way to do this, or would this be a feature request?
Regarding the Apple Vision Pro, is there a possibility to get a kind of rental device or leasing device from Apple?
Background:
I'm a fresh VisionOS dev.
I have a low budget.
I'm living in Japan. That means Vision Pro is roughly 150% the USD price.
Regards
I created an Object & Hand Tracking app based on the sample code released here by Apple.
https://developer.apple.com/documentation/visionos/exploring_object_tracking_with_arkit
The app worked great and everything was fine, but I realized I was coding on Xcode 16 beta 3, so I installed the latest Xcode 16 from the App Store and tested by app there, and it completely crashed. No idea why. Here is the console
dyld[1457]: Symbol not found: _$ss13withTaskGroup2of9returning9isolation4bodyq_xm_q_mScA_pSgYiq_ScGyxGzYaXEtYas8SendableRzr0_lF
Referenced from: <3AF14FE4-0A5F-381C-9FC5-E2520728FC65> /private/var/containers/Bundle/Application/F74E88F2-874F-4AF4-9D9A-0EFB51C9B1BD/Hand Tracking.app/Hand Tracking.debug.dylib
Expected in: <2F158065-9DC8-33D2-A4BF-CF0C8A32131B> /usr/lib/swift/libswift_Concurrency.dylib
It was working perfectly fine on Xcode 16 beta 3, which makes me think it's an Xcode 16 issue, but no idea how to fix this. I also installed Xcode 16.2 beta (the newest beta) but same error.
Please help if anyone knows what is wrong!
As mentioned in https://forums.developer.apple.com/forums/thread/756736?answerId=810096022#810096022
Is there any update about the full support to WebXR AR Module, which should enable immersive-ar mode?
Are the features such as DOM overlays and WebGPU bindings on the roadmap?
Is it possible to capture stereoscopic video either internally or externally or via airplay for debugging purposes?
Thanks
Hi, currently tinkering with a little shareplay app for the Vision Pro that allows people to facetime and shareplay to play with random 3d models (as well as move them around, which should sync the model positions for everyone in relative space).
When the users start their facetime call, then open the immersive space to see the 3d models, the models load in properly in context of the group immersive space's coordinate system, and moving the models reflects the new positions real-time for each participant.
The main issue comes if/when users use the digital crown to re-center their view. It appears to re-center the model and view, which is expected. However, it also seems to re-position the model/root entity to match the user's origin. Not sure if this is intentional or not, but this essentially makes it so that it "de-syncs" the model (so me moving the model next to someone does not reflect it 1:1 - it still moves properly, but the new "initial" position after re-centering makes it offset).
Is there a potential solution or work-around for this such that re-centering the view doesn't de-sync the model/entity's position?
Rough code for my RealityView component is below:
RealityView { content, attachments in
content.add(appModel.originEntity)
appModel.originEntity.addChild(appModel.modelContainerEntity)
appModel.setInitialModelPosition()
configureGestures(forModel: appModel.modelContainerEntity)
configureToolbarAttachment(content: content, attachments: attachments)
} update: { content, _ in
// I have modified the Apple provided gesture components to
// send the app model the new positions/rotations
// as well as broadcast the position/rotation to shareplay participants
// When user re-centers view, it seems to also re-position the model
// so that its origin is at the local user's origin, rather than
// the original origin
// Can we receive a notification that user has re-centered view?
// Or some other work-around?
appModel.modelContainerEntity.setPosition(appModel.modelState.position, relativeTo: nil)
appModel.modelContainerEntity.setOrientation(.init(appModel.modelState.rotation3d), relativeTo: nil)
} attachments: {
Attachment(id: "customViewAttachment") {
CustomView()
}
}
.installGestures()
Please let me know if anything wasn't clear or if more information is needed. Thanks!
As the title states, this severely limits the flexibility of multi-window applications in creating a good user experience.
Even effects like the ones shown below cannot be achieved.
I found that there is such a click-to-expand horizontally and smoothly effect in the system application called "message", which is good. I wonder if I can add a similar effect to my own app. If possible, are there any implementation ideas or examples that I can refer to? Thanks!