Build device-to-device interactions with Network Framework

♪ ♪ Hello, everyone. I'm Elliot Garner. I'm an engineer on the Internet Technologies team, and today I'm going to be discussing how to build device-to-device experiences using Network framework, and its new companion framework, DeviceDiscoveryUI. We'll start off by learning about cross-device connectivity, and how it can improve your apps. We'll explore how a new framework, DeviceDiscoveryUI, which pairs with Network framework to make that connectivity as simple as possible. We'll walk through a code example of how to update your apps to use DeviceDiscoveryUI, including an in-depth look at how to discover devices using the new system device picker, And how to connect to those devices using Network framework. So let's start with cross-device connectivity. Connecting to, and exchanging data between, nearby devices is often essential for building seamless, integrated experiences within your apps. Fitness and meditation apps often work best on a large screen where a coach can demonstrate moves. People benefit from seeing their heart rate, calorie burn, and movement data collected from their Apple Watch. Likewise, gaming experiences can be made more immersive with input and actions from a connected iPhone, or augmented by extending a second-screen experience to a nearby iPad. All of these experiences– games, fitness, and wellness apps– all share a need for reliable and easy-to-use cross-device connectivity. And that all starts with discovering a device. To help kickstart that discovery, we're introducing DeviceDiscoveryUI– a new framework that provides secure, privacy-preserving discovery of nearby devices. New in iOS 16, DeviceDiscoveryUI pairs with Network framework to enable device-to-device connectivity between your Apple TV apps and nearby iPhones, iPads, and Apple Watches.

DeviceDiscoveryUI provides system UI for easy discovery of nearby devices. Once a device has been discovered, your app can easily open a connection to that device. Because this connection was established using system UI, you don't need to worry about managing permissions for local network access. And because the system handles securely establishing this connection, you no longer need to implement your own key exchange, because the platform encrypts transmitted data for you. Here we can see that system UI in action. On the left, your app's icon is displayed, along with a required usage description string explaining what your app does with cross-device connectivity. On the right is the list of discovered devices. Because some apps may only be available on specific platforms, you can filter discovered devices by platform.

Upon selection of "Adam's iPhone", the system prompts for permission to create a connection between those devices. With this explicit user consent, you no longer need to request access to the entire local network. And once permission has been granted, your app will be launched on the selected device to handle incoming connections.

This means your app no longer needs to be running on both devices before a connection can be established. When permission is granted on "Joe's Apple Watch", the system immediately launches your app so the two devices can connect.

And if your app isn't installed on that device, the system offers to take people to the App Store. That way they can immediately download your app and begin using your new features. Here's how that looks on watchOS. Tapping the button will immediately launch your app's page in the App Store, for a quick and easy download. So now that we've discussed what DeviceDiscoveryUI can do for your apps, let's look at how to adopt this new framework.

In previous sessions, we built the Tic-Tac-Toe app using Network framework. Today we're gonna be updating it to use DeviceDiscoveryUI for discovery of and connections to nearby devices. In earlier iterations, our app only supported playing games between iOS devices. So to start, we've updated our project for Universal Purchase and added a tvOS and watchOS target that all share the same bundle ID. And we've changed the app so that instead of competing against another player, you're using your device to play against an AI on the TV. Next, we need to make additions to the Info.plist on both our tvOS application as well as the other platforms to declare our new Tic-Tac-Toe application service. Finally, we'll present the new device picker and use the resulting endpoints to make our connection. Now I'm gonna shift over to Xcode and show you what those new Info.plist keys are, and how they need to be added to the app. On tvOS, our app needs to tell the system what application services to discover, and what the platforms those services support. To do that, we need to add our new "Application Services" Info.plist.

This dictionary will map to one of two different arrays. On tvOS, we need to declare the "Browses" array, which contains all of the application services that our app discovers. Each entry represents a different application service. Our first item represents our Tic-Tac-Toe application service. It is a dictionary containing our service identifier, usage description, and platforms this service supports. The service identifier is the name of our service, "TicTacToe". The usage description is a string that will be displayed in the device picker to explain why our app needs access to other local devices. Finally, we have our platform support. This array contains the platforms supported by the service, and the system will filter discovered devices accordingly. You can see here that our service supports iOS, iPadOS, and watchOS devices. We've already updated the Info.plist used for iOS and iPadOS with the corresponding "Application Services" dictionary. Now we're gonna update the plist for watchOS. This time, we declare the "Advertises" array. Because DeviceDiscoveryUI will launch our app when it's not running, the system uses this array to know which services to advertise. Our entry in the "Advertises" array only needs the "TicTacToe" service identifier. Make sure that the service identifier is the same for every platform that was declared in the tvOS Info.plist. Now that Tic-Tac-Toe is configured to browse for and advertise our application service, let's have it actually display the device picker UI. Our usage description appears on the left. We made sure this string is active and that it informs people of why they want to connect a device and what data they're sharing. For Tic-Tac-Toe, we're using the connected device as a controller to play the game. Back in Xcode, because DeviceDiscoveryUI replaces the need to manually browse for nearby devices and secures the connection for us, we can remove the PeerBrowser file and our passcode extension to NWParameters since they're no longer necessary. Now, we need parameters to describe how we'd like to connect to nearby devices.

First, we use the new convenience initializer, applicationService, on NWParameters, which gives us everything we need for this kind of local connectivity.

Next, we can use our existing framer for communicating gameplay actions, without any changes, and simply add it to the protocol stack in those parameters. Now, we're ready to create and show the device picker. The device picker is how our application discovers nearby iPhones, iPads, and Apple Watches. First, we need to check to see if the device picker is supported on the current device. To do that, we call the isSupported function with the browse descriptor and parameters that we'll be using with the picker. Assuming it returns true, we can go ahead and create the device picker.

We instantiate the device picker with the parameters we created earlier, and a browse descriptor of type .applicationService, specifying the name of the application service that we defined in the Info.plist.

Now that the device picker has been created, we need to present the view controller. The device picker needs to always be presented as a full screen modal view.

Next, we need to access the picker's async endpoint property. Once the connection has been confirmed and our application has been launched on the remote device, we'll receive an NWEndpoint, and execution will continue. Now that our application has received an NWEndpoint, we can use it to connect to the selected device, just like we did in prior Tic-Tac-Toe versions. We'll use the same parameters when opening the connection as provided to the device picker. Aside from the parameters, opening a connection to this device looks exactly the same as it did before.

Moving to the selected device, once our application has been launched, we need to immediately fulfill the promise our application made to the system by creating an NWListener. The NWListener needs to be created as soon as the app is launched to accept any incoming connections for that application service. The NWListener needs to be created with the exact same parameters as before, and we need to set the application service on the listener using the identifier from the Info.plist. When the TV opens a connection to this device, the listener that we established will receive that connection here, in the new connection handler, same as before. So now that the connection has been properly established, we need to handle application state transitions. When our application is backgrounded, the connection will transition to the failed state with the associated error ECONNABORTED. If we want to continue communication between the devices, we establish a new connection from the TV to the same endpoint. Once started, this new connection stays in the preparing state and moves to the ready state once the app is resumed on the selected device. On that device, the new connection will be delivered to the same NWListener and can be used to resume our activity. And that's it. Everything we need to do to migrate to DeviceDiscoveryUI is now complete. Let's see our game in action. The app launches on the TV, and we press Find Opponent to present the device picker.

When we choose a device, it prompts for permission. When granted, the app is launched immediately into our game session.

Now, we can play against the TV by placing emojis on the board.

Great. And that's how easy it is to connect devices to Apple TV. Tic-Tac-Toe is available on the developer website, so you can download it and explore all of the code we discussed today. If you have any questions, post them on the Developer Forums. Consider adopting DeviceDiscoveryUI in your apps to provide seamless device-to-device experiences. And please provide us feedback. We know that device-to-device communication is an essential experience. DeviceDiscoveryUI was created in response to developer feedback. We want to build these features with you, so please file reports in Feedback Assistant for anything that you'd like to see in DeviceDiscoveryUI. We're excited to improve this technology together to help make your apps even better. Thank you so much for joining me, and have a great WWDC 2022.