I'd like to use the eye tracking feature in the latest iPadOS 18 update as more than an accessibility feature. i.e. another input modality that can be detected by event + enum checks similar to how we can detect and distinguish between touches and Apple pencil inputs. This might make it a lot easier to control and interact with iPad-based AR experiences that involve walking around, regardless of whether eye-tracking is enabled for accessibility. When walking, it's challenging to hold the device and interact with the screen with touch or pencil at all. Eye tracking + speech as input modalities could assist here. Also, this would help us create non-immersive AR experiences that parallel visionOS experiences that use eye tracking. I propose an API option for enabling eye-tracking (and an optional calibration dialogue within-app), as well as a specific UIControl class that simply detect when the eye looks at the control using the standard (begin/changed/end) events. My specific use case is that I'd like to treat eye-tracking-enabled UI elements or game objects differently depending on whether something is looked at with the eyes. For example, to select game objects while using speech recognition, suppose we have 4 buttons with the same name in 4 corners of the screen. Call them "corner" buttons. If I have my proposed invisible UI element for gaze detection, I can create 4 large rectangular regions on the screen. Then if the user says "select the corner" the system could parse this command and disambiguate between the 4 corners by checking which of the rectangular regions I'm currently looking at. (Note: the idea would be to make the gaze regions rather large to compensate for error.) The above is just a simple example, but the advantage over other methods like dwell is that it could be a lot faster. Another simple example: Using the same rectangular regions, instead of speech input, I could hold a button placed in just one spot on the screen, and look around the screen with my gaze to produce a laser beam for some kind of game, or draw curves (that I might smooth-out to reduce inaccuracy). OR maybe someone who does not have their hands available. This would require us to have the ability to get the coordinates of the eye gaze, but otherwise the other approach of just opting to trigger uicontrol elements might work for coarse selection. Would other developers find this useful as well? I'd like to propose this feature in feedback assistant, but I'm also opening-up a little discussion if by chance someone sees this. In short, I propose: a formal eye-tracking API for iPadOS 18+ that allows for turning on/off the tracking within the app, with the necessary user permissions the API should produce begin/changed/ended events similar to the existing events in UIKit, including screen coordinates. There should be a way to identify that an event came from eye-tracking. alternatively, we should have at minimum an invisible UIControl subclass that can detect when the eyes enter/leave the region.

Hola buenas no se me puede actualizar IOS 18 beta ni siquiera me aparece la opción de la beta tengo un iPhone 12 Pro

My recently developed app M4G Radio shuts off when the screen times out. It does not do this on Android, only on the apple side. Is there something I can do to remedy this?

Question: Hi everyone, I'm developing a Vision Pro app using the latest visionOS 2, and I've encountered some issues with the new hand gestures introduced in this update. My app is designed to display a UI element when a user's palm is detected. However, the new hand gestures for navigating key functions like Home View, Control Center, and adjusting the volume are interfering with my app's functionality. What I'm Trying to Achieve Detect when a user's palm is open and display a UI element. Ensure that my app's custom hand gestures are not disturbed by the new default gestures in visionOS 2. Problem The new hand gestures in visionOS 2 (such as those for Home View, Control Center, and volume adjustment) are activating while my app is open, causing disruptions to my app's functionality. I want to disable these system-level gestures when my app is running.

I'm waiting to buy my next apple watch until I can take calls on it. That can only work if it goes though my hearing aides like the iphone does.

Hello everyone. Is iPadOS18’s Eye-tracking feautre available on iPad Pro 12.9” 2018? If not, where can I find the compatible devices where Eye-tracking available?

Hello, When I run the following view on my iPhone with VoiceOver enabled, I encounter an issue with the voiceover cursor when I perform the following steps: Move the VoiceOver cursor to the tabview dots for paging. Swipe up with one finger to go to the next tab. --> Tabview moves to the next tab. --> The VoiceOver cursor jumps up to the tab. But instead of the actual tab the previous tab is shown in the tabview again. You can also see that the border of the VoiceOver cursor extends into the previous tab. I suspect it has to do with the fact that despite the clipped modifier, the size of the image remains the same and extends into the previous tab. struct ContentView: View { var body: some View { VStack { TabView { ForEach(1..<6) { index in Button( action: { }, label: { ZStack { GeometryReader { geo in Image(systemName: "\(index).circle") .resizable() .scaledToFill() .frame(width: geo.size.width) .clipped() } } } ) .buttonStyle(PlainButtonStyle()) .padding([.bottom], 50) } } .tabViewStyle(PageTabViewStyle(indexDisplayMode: .always)) .indexViewStyle(PageIndexViewStyle(backgroundDisplayMode: .always)) } .padding() } } How can I fix this? Best regards Pawel

i build apps that act as Screen Readers to 1) add Vim motions everywhere on macOS 2) click (and more) AX Elements through the keyboard 3) scroll through the keyboard. it works extremely well with native apps. with non-native apps, i need to blast them with some extra AX Attributes (AXManualAccessibility, AXEnhancedUserInterface) to get them to expose their AX Elements. but there are a couple of apps tho which i can't get them to expose their AX Elements programmatically. now the weird thing is as soon as i start VoiceOver, those apps open up. or for some, if i use the Accessibility Inspector to go through their AX Elements, then they start opening up. so i'm wondering, is there one public known way that i'm missing to open up those apps, or is Apple using private APIs? any way i could make my apps behave like VoiceOver or the Accessibility Inspector to force those recalcitrant apps to open up? thanks in advance.

Hi, I'd like to mark views that are inside a LazyVStack as headers for VoiceOver (make them appear in the headings rotor). In a VStack, you just have add .accessibilityAddTraits(.isHeader) to your header view. However, if your view is in a LazyVStack, that won't work if the view is not visible. As its name implies, LazyVStack is lazy so that makes sense. There is very little information online about system rotors, but it seems you are supposed to use .accessibilityRotor() with the headings system rotor (.accessibilityRotor(.headings)) outside of the LazyVStack. Something like the following. .accessibilityRotor(.headings) { ForEach(entries) { entry in // entry.id must be the same as the id of the SwiftUI view it is about AccessibilityRotorEntry(entry.name, id: entry.id) } } It kinds of work, but only kind of. When using .accessibilityAddTraits(.isHeader) in a VStack, the view is in the headings rotor as soon as you change screen. However, when using .accessibilityRotor(.headings), the headers (headings?) are not in the headings rotor at the time the screen appears. You have to move the accessibility focus inside the screen before your headers show up. I'm a beginner in regards to VoiceOver, so I don't know how a blind user used to VoiceOver would perceive this, but it feels to me that having to move the focus before the headers are in the headings rotor would mean some users would miss them. So my question is: is there a way to have headers inside a LazyVStack (and are not necessarily visible at first) to be in the headings rotor as soon as the screen appears? (be it using .accessibilityRotor(.headings) or anything else) The "SwiftUI Accessibility: Beyond the basics" talk from WWDC 2021 mentions custom rotors, not system rotors, but that should be close enough. It mentions that for accessibilityRotor to work properly it has to be applied on an accessibility container, so just in case I tried to move my .accessibilityRotor(.headings) to multiple places, with and without the accessibilityElement(children: .contain) modifier, but that did not seem to change the behavior (and I could not understand why accessibilityRotor could not automatically make the view it is applied on an accessibility container if needed). Also, a related question: when using .accessibilityRotor(.headings) on a screen, is it fine to mix uses of .accessibilityRotor(.headings) and .accessibilityRotor(.headings)? In a screen with multiple type of contents (something like ScrollView { VStack { MyHeader(); LazyVStack { /* some content */ }; LazyVStack { /* something else */ } } }), having to declare all headers in one place would make code reusability harder. Thanks

Is there a way for developers to generate IPA notation from user voice input like in the Settings app (Accessibility > VoiceOver > Speech > Pronunciations)? Thought this might be a useful option for AAC apps.

Hi, I met an issue with the accessibility in swiftUI, did anyone met this before? I set accessibilityValue to the button, but when the focus goes to the button, the voiceOver always says "1 update item"? I want Voiceover says the value I set. How could I achieve this?

Hello, I have to develop an application for a customer running on iOS (iPhone). The app needs to read data from and send data to a custom HID-Device connected via USB-C. As I read the docs, I thought HIDDriverKit (https://developer.apple.com/documentation/hiddriverkit) would be perfekt for that, but it is only available for macOS. Is there anything similar for iOS? Or can I go with DriverKit (https://developer.apple.com/documentation/driverkit)? Are ther any code examples? Has anybody done something like that before? Is it possible to acces custom HID-Device on iOS at all? Any help appreciated!!! Immanuel

The triple click accessibility shortcut is assigned to trigger the assistive touch icon , this works very rarely as intended and triggers the shut down screen. I have tried assigning different shortcuts but it the shut down screen is always triggered. On restarting the phone , it seems to work a few more times as intended , but then again reverts to the bugged state. I have create a feedback (FB13459492) for this in December 2023 but haven't received any reply. Updated recently with a screen recording.

Very excited about the new eye tracking in iPadOS and iOS 18. Some general eye tracking questions. Does the initial iPadOS 18 beta include eye tracking? If not, in which beta will it be included? Do developers need to do anything to their app for users to control their app using eye tracking? Will all standard UIKit and SwiftUI views and controls work with eye tracking without code changes? Will custom subclasses of UIControl work with eye tracking without code changes? Looking forward to testing eye tracking.

Is there way to identify process which is handling a global keyboard shortcut? Let's say I press some CMD-ALT-L in any application, some file shows up on disk and there is no any kind of visual indication of where it's coming from. How do I find the offender?

I'm wondering if we can expect an enhancement in the text to speech functionality anytime soon. AVSpeechSynthesizer is a little outdated compared to many speech synthesizers nowadays that are more natural and can distinguish between acronyms and words. It would be an awesome upgrade that it seems Apple could do very well. Thanks.

I am currently developing an Share Extension for iOS and have run into an issue where the NSExtensionActionWantsFullScreenPresentation attribute doesn't seem to be working as expected. I've set this property to true in my Info.plist

I'm trying to access Persona avatar and set camera access permission and allowed manually from Settings > Privacy > Persona Virtual camera > MY_APP => allowed I would like to access Persona image into image view. Looking forward for solution Thanks in advance

Is there any way to give blind users free access to features that everyone else has to pay for?

We currently have an odd issue with VoiceOver spelling a word letter by letter while the same word is spoken as a whole for other items. The app is in German. I have a View in SwiftUI whose button traits are removed, then a label "Start Tab 1 von 5" is added. "Tab is spoken as a whole word here, all fine. If I change the label to "Tab-Schaltfläche" or for example "SimplyGo Tab 3 von 5", then "Tab" is spoken as "T A B", letter by letter. is there a way to force VoiceOver to speak it as a whole?