Explore wellbeing APIs in HealthKit

Hi! I’m Leah, and I’m a software engineer in Health. Today, I’m going to introduce you to Mental Wellbeing, and show you the new APIs we’ve added in HealthKit to support mental health. And I’m Matt, and I’m also a software engineer in Health. I’m going to help incorporate these new features in an app I’ve been working on with my team.

Mental health has never been more important or relevant in our society. At Apple, we believe that small steps can make a large impact in taking care of one’s mental health.

Last year, we announced a suite of features focused on mental wellbeing, making it easy and accessible for people to engage in meaningful practices across their devices.

In the Health app, people can reflect on how they’re feeling and log State of Mind to keep track of their mental wellbeing.

They can also log State of Mind from Apple Watch, making it easy to check in at any point in the day.

And for a closer look at their mental wellbeing, people can take standardized questionnaires for anxiety and depression in the Health app. We’re excited about the way these features have already empowered people to invest in their mental wellbeing, and we want even more to benefit from these tools.

Now, the data types that power these experiences are available as API. In this session, we’ll dive into the emotion science behind Mental Wellbeing, as well as show you how to read and write these samples to HealthKit.

First, let’s start with some terminology. The GAD-7 and PHQ-9 are standardized questionnaires that are made available by Pfizer. They’re used by doctors and clinicians around the world for mental health screening. The GAD-7 consists of 7 questions to assess a person’s risk of anxiety and the PHQ-9 consists of 9 questions to assess risk of depression.

This year, you’ll be able to read and write the results of the GAD-7 anxiety and PHQ-9 depression questionnaires. With these new data types, people can check in on efficacy of treatment, or save the results from their doctor's office.

It’s important to give these assessments in accordance with Pfizer’s standards, so check out our developer documentation to learn more.

For the rest of the session, we’ll take a closer look at a third data type, State of Mind.

State of Mind is a way to represent mood and emotion that requires active reflection from an individual.

The difference between Mood and Emotion is the period of time that inspires a feeling. Emotion is a short-lived feeling, lasting a few seconds to a few minutes, While mood is a longer-term feeling, lasting hours or even days.

Taking the time to pause, reflect, and identify one’s current emotional state has many benefits. If you’re experiencing an unpleasant feeling, reflection can lessen the amount of time that you’ll feel poorly.

If you’re experiencing a pleasant feeling, you can savor that feeling and enjoy it for a longer period of time.

It’s normal and healthy to experience a wide range of feelings - whether pleasant, unpleasant, or neutral. This practice of naming one’s feelings develops long-term emotional awareness and resilience. The more specific someone is while describing their feelings, the more clinical benefit they will get.

We designed the State of Mind APIs with these principles in mind, in close collaboration with Emotion Science experts. Let me show you how it works.

State of Mind has 4 parameters. The first is feeling kind, which can consist of a daily mood or momentary emotion. When deciding which to use in your app, consider the context of the feeling, and whether it represents how someone feels in the moment, or over a longer period of time.

Next is valence, a self-reported measure of how good or bad someone is feeling. This is measured on a continuous scale from negative to positive 1. If I’m feeling very poorly, I would choose a value near negative 1. If I’m feeling great, a value closer to positive 1 would be more appropriate. More neutral values near 0 are also appropriate for a range of feelings.

Labels describe how someone feels, with choices like Passionate, Overwhelmed, and Relieved.

There is an exhaustive list of choices, and you can specify as many as you like to fully represent someone’s state of mind.

Finally, Associations describe the cause of a feeling, with choices like Family, Identity, and Work. As with labels, there are many to choose from and you can specify as many as you like.

State of Mind was created as a methodology for reflecting on one’s emotional state to provide real clinical benefit to people. However, the use cases aren’t and shouldn’t be limited to apps that are strictly focused on Mental Wellbeing.

State of Mind can be used in a wide variety of applications, anywhere it may be appropriate for someone to pause and reflect.

Through small, simple interactions, reflection and mindfulness can be integrated into any experience. Hey Matt, how do you think you’re going to use this API in our team’s app? After learning more about State of Mind, I think I can use the new APIs to associate how someone feels around calendar events. For context, I’m working on an app that visualizes how someone spends their time. Let me show you what I’ve been working on.

Right now, my app shows all of my events for my Office, Social, and Workout calendars.

The app provides visualizations and insights on how people spend their time, and I’m excited to go even deeper with the new State of Mind API. Specifically, I want to let someone add their State of Mind to indicate how an event felt. Now if you don’t mind, let’s build it out.

First, I’ll request access to read and write State of Mind samples through the standard HealthKit authorization flow. All health data is private and secure, and authorization helps keep people in control and specify what data they want to share.

For more information about authorization, check out “Getting started with HealthKit” from WWDC20.

Now, on to the app experience. There are a lot of different ways to show mood and emotion. Personally, I think emojis are a fun, lightweight way to represent how someone feels, so I’m going to use a set of emojis that will map to a State of Mind sample that we can save in HealthKit.

To start, I’ll pick these 5 emojis, where each one matches with a State of Mind label. Next, I want to think about how I’d reflect each one in a State of Mind sample.

Let’s make a new enum called EmojiType that will contain each emoji we will use to create a sample. Then, I’ll make a function that creates a State of Mind sample for a calendar event and an EmojiType. Because we are creating a sample for a recent event, I’ll use momentaryEmotion for the feeling kind Next, we need a valence value to represent how pleasant or unpleasant the feeling is. I’ll modify my existing EmojiType enum to have a new valence property, which maps each emoji to a valence represented by a Double.

After looking at the State of Mind labels that Leah showed, I’ll create another property on EmojiType called label to define the State of Mind label we picked for each emoji.

I can also infer which association to use from the event based on the event calendar. For example, my Office calendar maps to the Work association.

Now, I have everything I need to create my sample using the HKStateOfMind initializer.

And to save the sample, all I have to do is use the existing save method on the health store.

That’s it! And just in time for my team standup event too, where my State of Mind is pretty happy. Let’s try saving that feeling in the app.

When I tap a calendar event, I present the new emoji picker, which let’s me select an emoji, and then save a sample to HealthKit, just like that.

The app encourages me to take a moment to see how I feel after each event, which helps me build emotional awareness and resilience. It’s a simple and easy way to encourage anyone to track their mental wellbeing and receive real clinical benefit. - Hey Leah, what else can we do? - Let me show you! The benefits of State of Mind logging aren’t limited to the initial point of reflection. Looking at how you felt in the past can be very informative. You may be able to identify trends, like that you feel poorly when you aren’t getting enough sleep or that prioritizing exercise can boost your mood.

In the Health app, you can view your range of feelings over time and focus on comparisons of different areas of life, like Friends, Health, or Hobbies.

You can also view comparisons with other data types, like Exercise Minutes, Mindful Minutes, or Sleep.

Understanding the larger story of someone’s feelings can lead to big leaps in how someone lives their life, inspiring them to take better care of their health, both mentally and physically. With State of Mind, you can contextualize moments in someone’s life based on how they feel, and deliver personalized insights. We’re introducing 4 new HealthKit predicates.

You can query by Kind to distinguish between emotion and mood, highlighting changes between moments or days.

With the Valence predicate, you can fetch samples by how pleasant they are.

You can search for State of Mind samples with a particular label, highlighting moments that evoke a certain feeling like Calm or Stressed.

And with the Associations predicate, you can query feelings by a particular area of life, like Dating or Hobbies.

With these predicates, you can quickly get the information you need and focus on providing compelling insights, whether it’s highlighting the moments that make people feel their best, or providing resources for support in a challenging moment. Hey Matt, what can you do with these? I thought you’d never ask or should I say query.

So far, the app already offers some insights on how someone spends their time. Let’s see how I can query for specific State of Mind samples to provide more descriptive insights. In the Insights tab, we show a Work-Life Balance score for the week, as well as a Most Meaningful Event highlight. Work-Life Balance is calculated by measuring the ratio of scheduled events and free time during the day. So, if the majority of my calendar is filled with scheduled meetings, I’ll have a poor Work-Life Balance. While the Work-Life Balance metric can be helpful, it is only one perspective of a broader story. For example, a busy day might make some people feel fulfilled, or some events in my calendar might be fun activities. One way I can improve upon this metric is to take into account how I feel during my day. And I can measure just that by querying the associated State of Mind samples for each day First, I’ll create a query predicate to fetch all samples in a specific date range that have an association related to any of the calendar events in the app.

I’ll pass that into the new stateOfMindPredicate method to find samples that match the predicate.

Then, I’ll construct my HealthKit query and fetch the results.

Using the results, I’ll transform and combine all of the valence properties on those samples to find the average valence as a percentage to represent a metric on how someone is feeling.

Now, we can add this new metric called Calendar Quality to the Insights tab, where we highlight how someone is feeling throughout the week. But wait, there’s more! I have another insight in the app that represents the most meaningful event in the week. Right now, we’re calculating this metric by querying for the longest calendar event for the week. But on second thought, an event’s duration doesn’t really confirm how meaningful an event was Sorry, Mrs. B. So, let’s fix that. Let’s instead try querying HealthKit for one of the labels from my Emoji picker, such as “Happy”. Here, we use both an association and label predicate to find exactly the type of sample we’re looking for.

Then, I can identify the sample with the highest valence, and find the closest calendar event around that time.

That’s better! In addition to the Happy label, we can do this for multiple different State of Mind labels to provide several different insights into someone’s week. Well, that certainly makes me happier. With State of Mind data, you can enhance your app and provide new insights based on how someone feels, making a truly personalized experience. And we’re just getting started. Check out “Get started with HealthKit in visionOS”, where our teammates Zach and Sirinda will bring our app to Vision Pro. They’ll show you how to build experiences powered by HealthKit that leverage the spatial canvas of visionOS, and adjust the app’s behavior for Vision Pro, including support for handling a Guest User. But for now, I think our app is in a much better state. - What do you think, Leah? - It's awesome!! I love the idea of incorporating how I feel as a reflection of my week, instead of just how I spend my time.

There are even more advancements being made to support Mental Wellbeing across your devices.

Check out “Enhanced suggestions for your journaling app”, where our friend Rene will show you how to enable personalized State of Mind suggestions for journaling.

Today, Matt and I covered a lot. State of Mind is grounded in principles of emotion science to provide real clinical benefit.

Each property on the sample provides an important lens to represent how someone feels. These new APIs make it easy for you to create thoughtful experiences. State of Mind can be a valuable tool in Mental Wellbeing applications, like meditation, journaling, and other mindfulness practices. And because people experience feelings in all areas of their life, you can enhance your app to bring mindfulness into any experience. Incorporating State of Mind into your app is a thoughtful way to provide people with compelling insights like helping someone build emotional resilience, celebrating positive moments, and recognizing important patterns in their life. Before I let you go, here’s a couple of things to keep in mind. Remember that Mental Wellbeing matters in all areas of life. Look for ways you can use these new APIs in your app. And let us know what you think using Feedback Assistant. You can also check out some of the sessions we mentioned to learn more about Health & Mental Wellbeing.

Whether you’re incorporating State of Mind into your existing app or feeling inspired to make a new one. We’re excited to see what you create.