The crash data opt in rate is 36% for my app. I have another crash reporting framework PLC in the app. The total number of crashes reported by MetricKit health report was 43% of the total crash count from PLC in May. This almost matched the 36% opt in rate. In the month of June we fixed a series of crashes. Now the total number of crashes reported by MetricKit health report is 80% of the total crash count from PLC. The crash data opt in rate has stayed at 36% for the past 90 days and I am not able to understand how the ratio flipped from 43% to 80%. Looking for guidance here.
Metrickit opt in rate and actual crash numbers dont match
How are you determining if the user has opted into MetricKit reporting? Is the "MetricKit health report" something you have developed or something you're using off the shelf?
Are you talking about the "Share with App Developers" option that the user can opt-in from the Settings app? If so, that's not MetricKit. MetricKit, unlike the crash data found in Xcode Organizer and the App Store Connect power performance API, is fully up to you to control. Implement the callback, you should get data.
@edorphy "Are you talking about the "Share with App Developers" option that the user can opt-in from the Settings app? If so, that's not MetricKit"
Don't MetricKit depends from this option? Could you give some link about that, pls? I can't find information about it.
@edorphy Don't MetricKit depends from this option? Could you give some link about that, pls? I can't find information about it.
@edorphy @dkvijay @andreikrn Anybody figured this out yet? Does MetricKit require user permission - Share with App Developers ?
@andreikrn @atul3189 @dkvijay
The conversation I had with a MetricKit engineer during WWDC20 or WWDC21 labs was that the user setting to "Share with App Developers" is not used in MetricKit when deciding to generate a metric or diagnostic payload. If you read the footer carefully, "Share with App Developers" is an Apple thing, Apple is sharing it with developers. MetricKit allows YOU, the developer, directly receive the data and then handle it responsibly. I have been operating under this assumption for years. I just wrote up a feedback requesting a docs update for this. Fingers crossed.
FB13618353 - MetricKit / Developer Documentation: Add description to the behavior of MetricKit with respect to "Share with App Developers" setting
Before operating under this assumption, please consider giving your users the option to control data collection. #privacy-by-design
So now that you've thought about letting users choose to participate in this data collection within a feature or setting in your app, it is easy to validate the assumption that MetricKit is not tied to "Share with app developers".
- Turn off the "Share with App Developers" setting
- Create a button that crashes your app.
- Generate logs or save the payload in your subscriber didReceive callback
- Install this app on a device via Xcode or TestFlight
- Ensure you are NOT debugging, as the crash will be caught by Xcode
- Tap the crash button
- Relaunch the app, your app SHOULD get a callback
- Verify that your log is printed or the payload is persisted via your process
An alternate path if you don't log or save the MXDiagnosticPayload would be:
- Set a breakpoint on your didReceive diagnostics method
- Attach the debugger (run app) on your device with Xcode after the app is crashed
- Verify during app launch that you didReceive diagnostic callback is invoked.
I just verified this with iOS 17.3.x in one of my apps using the alternate path and Xcode 15.2.
NOTE: For all of this to work, you should be setting the .addSubscriber method somewhere during app startup--my favorite spot is didFinishLaunching. Even for a SwiftUI application, I still have a UIApplicationDelegateAdaptor and in that implementation I put it in didFinishLaunching.
Now a note on privacy and MetricKit data collection.
Because the user can't control my app's MetricKit payload collection via the global setting "Share with app developers", I have explicitly put reference to MetricKit metrics and diagnostics in my privacy statement. Furthermore I have given users the option to enable/disable directly within my app. With that having been said, I do make a decision that, if you use a TestFlight build of my app for beta testing, both diagnostics and metrics will be recorded by default to help improve my app--that is after all the whole purpose of TestFlight right?
P.S. Don't forget to update your App Privacy Label with respect to this data collection too!
I conditionally detect installed from TestFlight and add a disclaimer to the very top of my Privacy Policy.
If the user did in fact install a TestFlight build, they will see this in the settings to enable/disable collection. Again, note the disclaimer.
My apps don't always crash, but when they do MetricKit is there for me to records it. As an added perk, MetricKit can capture events that other off-the-shelf crash reporters just can't do, so I use MetricKit exclusively to build the most privacy preserving crash reporting solution I can.