Первые в Беларуси соревнования по информационной безопасности AgatCTF-2018 проходят в Минске

Hi! Welcome to Bring your app’s core features to users with App Intents. I’m Christopher Nebel, an engineer on the App Intents team.

This is my app. There are many like it, but this one is mine. Actually, it’s sample code; it’s available from the developer site. It’s a catalog of trails; we’ll be seeing it throughout this video. Just, work with me here. Anyway! This is my app, and it is awesome. People tell me that when they’re in it, it’s a delight to use. What about when they’re not in it? It’s mostly invisible, it’s just a box on their Home Screen? Not necessarily! With the right code, I can lift my app’s features out of that box and elevate them to other places in the device, so that people can perform actions from my app directly from Spotlight, or by speaking to Siri.

Or they can add information in a widget to their Home Screen.

Or add actions and status to Control Center.

So my app will continue to delight people even when they aren’t in the app.

This is what App Intents and the features that it powers delivers. How does that work? Well, you’re in the right place to find out. I’m going to talk about improving flow when using your app, and a family of system features that help you do that.

What App Intents is, and how it fits in with those features, and how to get started building App Intents for your app.

An important part of a delightful experience is flow. In psychology, that’s the melding of thought and action. Whatever you think to do, you do it, effortlessly. On a device, that means the next thing you need is always nearby and easy to get to. If you have distracting extra steps in the way of your flow, that’s called friction. Metaphorically, anything that resists getting things done. The same metaphor shows up in lots of other expressions: Things with high friction are rough. You get stuck. They’re a real drag, man. Things with low friction are fluid. They’re like a well-oiled machine. They’re smooth. A little friction can be a good thing, don’t get me started on completely frictionless surfaces but generally, you want to reduce friction wherever you can. Friction doesn’t have to be a single big barrier. Often, it’s a small bump that you hit over and over. An example when using your device is switching apps.

Your app defines a little world, its own box, that you have carefully crafted, and within that app, it’s perfect. The catch is that people don’t use just one app to get things done. They use many. What happens when they need to move between boxes? Because each of them is in their own box. And by default, the only way to see or do anything in a box is to go into it, which means now you can’t see or do anything in any of the other boxes. You have to leave your current box and go into another one. That is, you have to switch apps.

And switching apps isn’t really hard, but it’s not free. It takes time and thought while you navigate there and back. That’s friction. What if the device understood what’s in your box, your core features? Then, it could present them higher up, outside your box, and make them quicker and easier to get to. There’s a whole family of system features based on this idea, making the experience of using your device flow. Let’s look at a few examples, all of which I’m going to build later.

These are based on my example app, which shows a catalog of trails in the area, with various details about them.

Some features help me get to the right place in the app faster, such as searching in Spotlight.

Say I’m at the Home Screen, and I want to look at my pinned trail. So I trigger Spotlight. Before I even type anything, Spotlight Suggestions guesses that I want my example app and offers the app shortcut that I need. Imagine that it hadn’t guessed that, so I have to type: e, x, a. Now it’s sure I want my example app, so there it is as a Top Hit, with the app shortcut even more prominently placed.

Or, from anywhere, I could have just said what I wanted to Siri.

Some features don’t just make it easier to switch apps. They make it so I don’t need to switch apps in the first place. Let’s say I want to keep an eye on the conditions for my favorite trail.

Instead of opening the app every time I think of it, I can add that information to my Home Screen with a widget. Now I can glance at it as I’m doing other things. I could add it to the Lock Screen, and then I wouldn’t even need to unlock my phone.

Or I can add a custom control to Control Center, which I can reach while using any other app.

There are many other features like these, and we keep adding them. Last year, we added the Action button. This year, we’re adding Camera capture, Apple Pencil Pro: squeeze, and more.

All of them elevate your app's features out of their box, to become part of the whole device experience, and streamline the flow between thinking I want to do this and it’s done. There's an important distinction among these features: Who chooses what to elevate? With features like Spotlight, you, the developer, choose. You highlight important actions in your app, and the device presents them automatically in the right spots. Likely actions from your app are always close by, no matter where people are on their device, giving them that sense of flow.

With features like Widgets and Controls, people choose. You provide flexible components, and then people can choose precisely the ones that matter, and build the right work flow for them. In other words, they can personalize their experience of your app making their device an extension of, and even an expression of, themselves. That’s good for them, and it’s good for you, the app developer. Because the more people can mold your app to fit themselves and their lives, the more likely they are to keep using it.

At this point, you might be saying, this sounds great, I want my app to flow, what do I do? Glad you asked! The answer is, you adopt App Intents. And you say, wait, what? I thought we were talking about Siri, and Control Center, and all those other cool features. And I say, we are. Allow me to explain.

App Intents is not a feature in itself. It’s a common foundation for building features. All these features we’ve talked about, Siri, Spotlight, Shortcuts, and so on, have a common theme: They take the core features from inside your app, actions and content that are meaningful to someone using your app, and present them outside your app.

To do that, you need to expose your app’s core features in a way that the system can understand. App Intents is the framework to do that. It handles two big parts. First, it lets you define what your core actions and content are, so system features like Siri and Spotlight know what they can present. Second, actually presenting something means a bunch of back and forth communication between your app and the presenter. For example, if someone tells Siri to do a thing in your app, Siri has to send a message to your app to do the thing, and then your app has to send a message back saying that it did it, and here are the results.

App Intents handles that communication, so you can concentrate on implementing the actual behavior of your features. And once you’ve done that to support one system feature? That same work is useful across a range of system features, because they’re all based on App Intents. So you can share code between them, and even support several features with the exact same code.

There are three top-level things you’ll write, and I’ll show how in the next section.

Intents perform an action, like Opening a view or Starting a hike. They’re the commands, or verbs, in your app.

Entities are objects, like a Trail or a Collection. They’re the nouns. If intents are verbs, and entities are nouns, then App Shortcuts are sentences. They package together a verb and a noun or a blank to fill in a noun, and whatever other options are needed into a single thing that describes an important function of your app, something that you know anyone who uses your app will want to do.

Features like Spotlight and the Action button will then offer that function, that sentence, so people can do it quickly and easily.

Together, these define how your app appears across the device. Now that you have a general idea, let’s write some code.

I’m going to add some features to my app to help it flow. Five of them, in fact: A simple Shortcuts action with no parameters. A Parameterized Shortcuts action. A Home Screen Widget. A Control Center control. And app shortcuts for Spotlight and Siri. That sounds like an awful lot, but they’re all based on the App Intents framework, and that’s going to help me share a lot of code between the features. Let’s get started.

The Shortcuts app is a powerful tool where people can explore, combine, and remix actions to create their own custom shortcuts. For example, my teammate loves mountain biking with his daughter, so they used the actions from my app to build a custom shortcut that picks a random biking trail in an area.

As the developer, I might not prioritize this feature, it’s kind of specific. But I can offer flexibility in the form of Shortcuts actions, so they can build it for themselves, and tune it to be exactly what they want.

I recently added a feature to Pin a trail to the top of the list. That way, when I’m planning a trip there, I can open its details quickly, so I can answer questions like how do I get there, or can my friend bring their dog? Works great, but I want to be able to get to that trail quickly even if I’m not currently in the app, so let’s make a Shortcuts action that opens the pinned trail details in the app.

When I’m done, it’s going look like this in the Shortcuts app.

In the App Intents framework, a Shortcuts action is an intent, all intents appear by default as actions in the Shortcuts app.

An intent is a type that conforms to the App Intent protocol.

It has two required pieces: A localizable title, which will appear as the name of the action, and a perform method, which does the action.

Intents should always be actions that are meaningful to someone who uses the app. Here, OpenPinnedTrail not internal implementation.

Perform methods always have a result, but the result may be empty, as it is here.

For this intent, I want it to open the app when it runs, because I’m showing a view in the app. App Intents will handle that for me if I tell it to, which I do with openAppWhenRun like this.

An intent can also have extra parameters, extra information needed to do its job. This intent doesn’t need any, but I’ll come back to one that does.

And that’s it! My new intent shows as an action in the Shortcuts app library.

If I make a shortcut with this action, I can run it in the Shortcuts app. I can take that one step further and add it to my Home Screen.

If I tap it, it runs, and my app opens with the pinned trail details, just like I wanted.

That’s not bad, but I also want an action that lets me pick the trail to open, so I can make a shortcut that opens a particular trail, or that asks me for a trail when I run it.

To do that, I need to make an intent with a parameter. This is going to look very similar to the first intent.

In fact, I’m going to start by cloning the first intent and changing its name. Intents that open their parameter in the app have a special protocol, OpenIntent, so I’m going to add that.

Open Intent implies openAppWhenRun, so I can delete that.

And, I need to add the parameter that the protocol defines.

A parameter is just a normal property marked with @Parameter so you can add extra metadata like a localizable title.

I said it should be a trail. Trails are a core content type in my app, so I’ll make it a trail entity.

Two things to remember here: First, entities should be things that are meaningful to someone who uses the app. So this isn’t a database row, or whatever my implementation is, it’s a trail. Second, a parameter that refers to an entity should be an entity, not data describing one. So this isn’t a trail name, or UUID, it’s a TrailEntity.

I update the perform method to navigate to the parameter. And I’m done, except for the part where I have to define the TrailEntity.

Fortunately, that’s not very hard. An entity is a type that conforms to the AppEntity protocol. You can conform your model type directly. This works well if your model is small enough to all fit into memory at once, so all the necessary instances already exist. Or, you can make your entity its own type that merely refers to your implementation. This is useful if you only create model instances as you need them, or if your model instances have expensive properties that aren’t needed for your intents.

In this example, I‘m going with the second approach, so TrailEntity will refer to my underlying model type Trail.

An entity may have Properties here, the trail name. It’s very similar to a parameter, but it’s on a content object, not a command.

An entity must have three things: A display representation, so the device can do things like draw them in a menu, a persistent identifier, and a query.

Right. What’s a query? Before I tell you what a query is, I should tell you what a query does. A query turns questions asking for entities into actual entities.

And choosing a value for the intent parameter is going to ask two questions: First, what entities are there? So the device can show a list of options someone can pick from.

When they pick one, it’s going to save the ID for that entity. Then, when it’s time to run the intent, it sends that saved ID, and the query needs to answer the second question: What entity has this ID? Given that answer, it can fill in the intent parameter, so your intent code gets an entity, and not just an ID. What does that code look like? A query is a type that conforms to EntityQuery. There are several different ways to ask for entities, not just the two I already mentioned, what options and what has this ID, but also searching for a string, searching by predicate, and so on.

So EntityQuery has several sub-protocols, one for each kind of question.

The what entities are there question, can be handled by EnumerableEntityQuery, which defines a method allEntities, like this. This is the conceptually simplest form of query. App Intents can derive the more complicated ones from this, if you build with the iOS 18 SDK. Of course, it only works if you can actually return all your entities at once, so your whole model has to fit into memory. If that’s not the case, or if you can do a better job than the automatically derived queries, then you should use one of the other query protocols. But this example is simple enough for it to work.

The what entity has this ID question is answered by EntityQuery itself, with this method, entities(for:).

Together, these handle everything parameter configuration needs to work.

And now my intent is ready to run. But there’s one more thing I should add, which is a parameter summary.

Without one, my intent looks like this in the Shortcuts app. This works, but it’s hard to read and understand, because the trail that it’s going to open is below the fold.

A parameterSummary, is a natural language sentence that describes what the intent will do, including the values of all the essential parameters, in this case, the trail to open. Now it’s easier to see what this intent does, and it’s readable as I edit the parameter.

Let’s see it in action. When I tap the Trail placeholder, I get a sheet to pick a trail. The query provides the options, and powers searching. When I pick one, it appears inline so I can immediately read what this action will do when it runs. That’s the parameter summary at work.

I actually don’t want to choose a trail now. I want to choose one when it runs. So I’ll clear that parameter and save the shortcut to my Home Screen.

Now, when I tap the shortcut, I get a similar picker, again, powered by the query, and it works straight from my Home Screen or anywhere I can trigger the shortcut.

Shortcuts actions are the ultimate in adding flexibility to your app. People will build them into all kinds of workflows that you never imagined. Let’s look at a more focused form of flexibility, that lets you view information at a glance: A widget. Most details about a trail don’t change often, if ever. It’s not going to move to a new location. Probably. We do have earthquakes.

Conditions, however, can and do change routinely, so it would be handy to keep an eye on them.

That’s a perfect use for a widget. They’re designed for glanceable information. I could make a widget that just shows the details of my pinned trail, but I want to keep an eye on a few different trails, so I can choose whichever one is nicest. To do that, I need to make my widget configurable so I can set which trail to show. That means my widget is going to have a trail parameter, much like my Open Trail action.

In order to make a widget, I need a widget definition. This isn’t a WidgetKit video, so I’m skipping a number of details like the view and timeline provider. If you want to know more about that, refer to the WidgetKit documentation, and for more details about how widgets work with App Intents, check out "Explore enhancements to App Intents" from WWDC23.

The relevant part here is the body. And especially the intent parameter, where I set the configuration intent type for this widget.

The configuration intent, is an intent that conforms to WidgetConfigurationIntent. Funny how that works.

Like all intents, it needs to have a title.

And I said I wanted this widget to be configurable with a trail, so I’ll add a parameter.

It’s supposed to be a trail, and I already defined a trail entity for my open intent. Can I re-use that? Funny you should ask. Yes, you can.

I’m making it optional, so it can be empty when the widget is added to the Home Screen, but it’s otherwise the same type as the parameter in the Open Trail intent. Now, I have a trail-conditions widget that’s configurable for any trail, so I can add as many as I need. The trail entity and its query that I already wrote handle picking a value for the configuration, so I didn’t need to write that again.

That was remarkably straightforward, so let’s do another one: A Control Center control, which have a new API in iOS 18.

Like the widget, I want the control to be configurable with a trail. When tapped, it should open that trail’s details in the app.

This is only going to cover the App Intents-related parts of Controls. For the full Control details, see “Extend your app’s controls across the system”. With that in mind, let’s get started.

The basic outline of a control is that it’s a special type of widget, ControlWidget. Which has a body that defines what the control looks like and what it does.

A configurable control, which is what I’m aiming for, is configured using an App Intent. Much like a configurable widget, so I add an AppIntentControlConfiguration.

I said it should act as a button, so I'll add a ControlWidgetButton.

This is the basic shape for a configurable button control. I still need an intent type to hold the configuration so I can build the button contents, and an intent instance to handle when the button is tapped.

Let’s think about this for a second. I said I wanted the configuration to have a trail parameter. That sounds kind of like the Open Trail intent I made earlier. Can I re-use that? You bet I can. The one thing I need to change about OpenTrail is to declare that it conforms to ControlConfigurationIntent. That’s an extension, and doesn’t require anything that the intent doesn’t already have, so the body is empty.

Once that’s done, I can fill in the type and use the configured instance of that type to build the image and text for the button.

And here’s the really clever bit. I still need an action, which is an intent instance, and it should open the configured trail. Again, that sounds like OpenTrail. But, I don’t need to make a new OpenTrail instance, because I already have one as the configuration intent. I can simply pass that as the action, because it already has the Perform method I need and is configured the way I want. Let’s see it work.

This is the new configuration mode for Control Center. I’ve already added the new control at the lower left, here, and configured it to open the Monterey Bay Coastal Trail. If I stop editing, the control is now live, so if I tap it, the intent runs, and my app opens to the trail details I configured.

Now, I’ll make one of my app’s actions available automatically in Spotlight and Siri, opening the details of the pinned trail. To do that, I need to make an App Shortcut. An App Shortcut is a wrapper that you, the developer, create around an intent that highlights it as an important function of your app.

Various features will then offer that App Shortcut such as Spotlight, the Action button, and Apple Pencil Pro, without needing your app to be running or even have run at all. Let’s look at the code for that.

I define a single AppShortcutsProvider with a static member, which is a list of app shortcuts. An AppShortcut wraps an intent. Here, the OpenPinnedTrail intent I created earlier.

Notice that this is an intent instance, not a type. That means that I can pre-fill some or all of the parameters here. So I could take a very general intent with lots of parameters, and wrap it as an AppShortcut that does something very specific. It will prompt for any unfilled required ones when the shortcut runs, so it works either way. It depends on what experience you want to provide. This intent doesn’t have any parameters, so the parentheses are empty.

Since App Shortcuts are available from Siri, there’s a list of phrases to speak, each of which must contain the app name.

And for places where it’s displayed visually, such as Spotlight, a title and image.

Notice one thing that’s not here: Any sort of registration code. The App Intents framework automatically detects the provider and handles registration, so my App Shortcuts are available as soon as the app is installed.

With that, the App Shortcut now appears in Spotlight.

And can be spoken to Siri.

Both Spotlight and Siri are handled automatically; someone using their device doesn’t need to do anything more than install the app. But, if they want to customize their device, App Shortcuts are also available: In the Action button and Apple Pencil Pro. So that’s four features for one piece of code. And if you have existing App Shortcuts? They already work with Apple Pencil Pro, same as they automatically worked with the Action button last year.

This is looking good, but the Siri interaction isn’t quite right. First, it’s going to show a view which means I have to at least be looking at my device. Second, it’s going to open the app, which is going to bounce me out of whatever app I’m currently using. So let’s replace that intent with a slightly different one that returns the information instead.

I’ll make an intent, but this one won’t Open the pinned trail. It will show it in a snippet, and speak it if the user isn't looking at their screen, or doesn’t have a screen, like a HomePod or AirPods.

I’ll start with a stub perform method. Step one is clearly getting the pinned trail. But what’s step two? How do I show or speak information without using my app? I use the result.

The result of a Perform method isn’t just a value. It’s a rich combination of things, including dialog that Siri can speak and a view snippet it can show.

If I say that my result conforms to ProvidesDialog and ShowsSnippetView, I can use a different method to create the result that takes dialog and a view. I can supply just dialog or just a view if that makes sense, but here I’m going to do both.

I’ll write what I want Siri to say, which may be different from the text in the snippet. I’m using an interpolated string here for convenience, but this parameter is actually an Intent Dialog instance, which allows separate Full and Supporting dialog if you need it.

And I supply a SwiftUI view. I’ll spare you the exact details of my view and use a separate function to get it, but I could have put it inline, using a trailing closure with a view builder expression.

In either case, the view is archived like a widget, so it can be relayed to Siri. So you can use any SwiftUI features that widgets support.

Now, when I ask Siri, it will show me the pinned trail conditions in a snippet, if I'm looking at the screen. When I dismiss it, I'm right where I left off, so I don't have to find my way back. If I'm not looking at the screen, Siri will speak the dialog to me. If my phone is in my pocket because I'm busy looking for my boots, I can ask my HomePod across the room for the conditions and I don't even need to take my phone out.

Ok, wow. I just added a redonkulous number of features to streamline how people can use my app, and it wasn’t even that much code. This is what App Intents does. It lets you express your core concepts once, and then re-use them across many related features. Wrapping up: You can improve flow when using your app. People will get things done faster, they’ll enjoy it more, and they’ll be able to tune how they use your app and build it into their lives. To do that, adopt features like Siri, Shortcuts, widgets, and more, all of which let people use your app even when they’re not in your app. The way you adopt those features is to use App Intents, because it’s the foundation for all of them. That means you need to learn fewer different APIs, and you can share code between features.

Want to learn more? Of course you do! Here are some great videos to watch next: If you’re still fairly new to App Intents and honestly, even if you're not see "Designing App Intents", on what makes a great App Intent. If you want to find out more about how App Intents work with Siri, see "Bring your app to Siri".

If you already have App Intents and want to find out what’s new since last year, see "What’s new in App Intents".

Thanks for watching, and we can’t wait to see what you do with App Intents!