Design for spatial user interfaces

♪ Mellow instrumental hip-hop ♪ ♪ Miquel Estany Rodriguez: Hi, I'm Miquel Estany Rodriguez. Lorena Pazmino: And I'm Lorena Pazmino, and we're from the Apple Design team. Today, we're going to talk about how to design spatial user interfaces. To help you get started on this platform, we've built a visual language that maintains consistency and familiarity with our existing platforms, while also evolving certain elements to adapt to immersive and spatial experiences. Miquel: We'll start by talking about the UI foundations and design principles to consider when creating app icons and interfaces that are legible and usable in an environment. Then we'll go over key concepts and best practices to create layouts that are both ergonomic and easy to target. And finally, we'll conclude by showing how this platform can bring your app from a screen to spatial, walking you through all our system components, most of which you are already familiar with and some that are completely new. Let's dive deep into UI foundation. These are the core design principles we will cover on this section. Let's start talking about app icons. We wanted to keep home view familiar and we've gone one step further, making app icons three-dimensional and realistic in your space. When people look at them, they expand. And the system enhance this visual effect by adding specular highlights and shadows to reinforce the subtle depth between layers. So how can you design a great icon? First of all, use multiple layers. Other platform use layers to create a parallax effect. Here, just by using flat layers, the system can create this truly 3D effect. All app icons can have up to three layers, a background layer and up to two foreground layers on top. Each layer is a square image, and their size is 1024 by 1024 pixels. Both foreground layers should have a transparent background. Design your background layer as an edge to edge square image. Then all layers will get cropped by a circular mask. And finally, when layers match together, a glass layer is applied automatically, adding depth, specular highlights, and shadows to them. Try to keep graphics centered. If they are too close to the edge, they could look off-center when expanded. Avoid using large regions of semitransparent pixels. Layers with reduced opacity will blend with the shadow cast behind it. And that's how you can easily create three-dimensional app icons. Next, let's talk about materials. When people tap your app icon, apps will launch in many different places like this one, a broad daylight room, or in a small space like an airplane, or at nighttime. Apps need to adapt to the lighting conditions in your surroundings. They should be easy to place in the space around you, easy to use at any distance, and easy to view with any lighting conditions. And that's why we've designed a new visual language, the glass material. This system-defined beautiful glass window feels part of your physical world. And its unique properties allow light from people's surroundings and virtual content to show through. In addition, specular highlights and shadows reinforce its scale and position in your space. And it works as a canvas for your contained UI, making it feel lighter and adding a sense of physicality to it. This lightweight material also gives people a sense of what might be behind a window, like other apps or people. To deliver great spatial experiences, it's important to be aware of your surroundings. Avoid using solid colors on windows. Too many opaque windows can feel constricting and make the interface feel heavy. Glass responds dynamically to lighting, adjusting the contrast and color balance to feel part of your space, like in this transition from day to night. Unlike iOS and macOS, this platform does not have a distinct light or dark appearance. Instead, glass and UI naturally adapt when placed in front of light and dark backgrounds. System-defined vibrant materials take great advantage of this feature, ensuring your app looks great across different scenarios and lighting conditions, Let's take a look at them. First, we start with a glass window. If you want to separate sections of your app, like a sidebar, use a darker material. Or a lighter material to bring attention to interactive elements, like buttons. Or you might even consider using darker materials to increase contrast for standard components, like input fields. Here is how Music app looks with all the UI elements. As you can see, text defaults to white. So, imagine that you want to design a lock-up like this one, and you want to add a lighter button. It is great to place it on top of the glass material. Or if you want to get more contrast, use a darker cell behind it. Try to not stack lighter materials on top of each other, as it impacts legibility and reduces contrast. Now, I'll show you how typography has been precisely adapted to this platform. To keep things familiar, all font styles use semantic names that work on all platforms. They have been tuned on our point-based unit system, ensuring they are legible at any distance. To learn more, check out the session "Principles of spatial design." We've also modified some of the font weights to improve legibility. Let's take a closer look. To improve the contrast of text against vibrant materials, font weight has been modified to be slightly heavier. For example, on iOS, we use regular weight for the body text style. On this platform, we use medium. And for titles, instead of semibold, we use bold, keeping text clear all the time. Consequently, the tracking has been slightly increased to help with legibility. In addition to the well-known font styles we share across platforms, we've added two brand-new font styles that take advantage of the wide, editorial-style layouts you can use. Here is an example of the Extra Large Title 1 on Arcade. Cool, right? Even though windows can scale up to incredible large sizes, custom smaller or lightweight fonts can still be difficult to read. To improve that, consider increasing the weight or using a typeface designed for optimized legibility, like system fonts. Next, let's talk about vibrancy. This is one of the most important details to maintain legibility across the system. Vibrancy brightens foreground content that displays on top of a material and works by pulling light and color forward from what's behind it. On this platform, since the background can be constantly changing, vibrancy updates in real time to make sure your text is always legible. To show what I mean, let's look at an example. Vibrancy works on top of the glass material, enhancing legibility and making system materials feel richer and sophisticated. Make sure to use the system components when possible. By default, they take advantage of this vibrancy effect. Let's pause for a second and turn vibrancy off. As we just saw, vibrancy improves legibility and how materials look. Let me turn it on and off to show you the difference. Now let's see how you can take advantage of it. Use vibrancy to indicate hierarchy for text, symbols, and fills. There are three modes: primary, secondary, and tertiary. Use primary for standard text. Or use secondary for description text, footnotes, and subtitles. You've seen the glass material can be many different colors, since it shows the color of what's behind it. Colorful elements on top of the glass may be hard to see if the color of the glass is similar. Most of the time, consider using white text or symbols so they are always clearly visible. If you need to use color, use it in a background layer or an entire button so people can see it. When possible, use system color instead of custom color, as they have been calibrated for legibility and dynamically adapt to maintain hue and contrast on the glass. So those are the design principles and the updates we've made to the core components, like materials, typography, and vibrancy, to make sure your app gets elevated on this platform. Now I'm going to hand it off to Lorena who will show you how to create ergonomic layouts that are easy to target as you take your app from screen to spatial. Lorena: Thanks, Miquel. Let's start by exploring some of the best ways to position content so that it feels comfortable. Then show you how to properly size content so that it's easy to target. And lastly, show you how to create layouts that work with our new and beautiful eye focus feedback. Because people primarily interact with their eyes and hands on this platform, it's important to create experiences that prioritize people's physical comfort and safety. More than ever, you need to consider the ergonomics of your design, ensuring that the placement of content is intentional and doesn't cause eye or neck fatigue. For example, because of the neck's range of motion, it's easier for most people to turn their head farther to the right and to the left rather than up and down. So keep your UI in people's field of view and be careful about placing anything too high up or too far down. If you need a large canvas for your app, go with a wider aspect ratio than taller. For example, in Freeform, the canvas extends horizontally and not vertically. Prioritize having the most important information centered in your app so people can comfortably view your content. Now that we've covered best practices for content placement, let's talk about content size. Each person is unique and our eyes have slight variations from one person to another, which is why elements should use sizes that are easy to target. Let's get into the details. Interactive elements must have a tap target area with at least 60 points of space, so they can be easily selected. This means that your UI element can be visually smaller, like the standard button, which is 44 points, as long as you add enough space around it. So for this button, it always needs to have at least eight points of empty space around it so it meets the minimum of having a target area of 60 points. As an easy thing to remember, if you need to have several buttons in a stack, use standard system buttons with at least 16 points of spacing in between them. Now, let's say you needed to use a visually smaller element, like a disclosure control. You can use a mini button, which is 28 points, in an area like the section header. Even though this button might look small, because it has 60 points of space around it, it's easy to select. Let's look at another example. Here, this button still has enough space because the text next to it, it's not interactive. Similarly, using large and extra large buttons requires less spacing around them. So, always remember to give all interactive elements at least 60 points of space. Now that we've talked about how to size elements in your layouts, let's talk about focus feedback. Focus feedback is a powerful tool built into every interactive element in this platform. When people look at system-provided components, they automatically display a subtle visual brightening or hover effect. This hover effect allows people to understand which parts of an interface are interactive just by looking at them. When an item becomes inactive, like the Share button, it no longer gives focus feedback. This gives people confidence that they are focused on the intended element, which means that they can tap their fingers to select it. As you go creating layouts, it's important to take the hover effect into account. For example, if you're creating a list or a menu, you need to account for a small amount of padding in between each item to avoid having the hover effect overlap. Four points is recommended. As you're designing a lock-up, include a shape that will allow the system to display the hover effect as people focus on it. Here, we have some images with text below them. Each lock-up is a single interactive element. You need to define a custom region so it can brighten as people look at it. This helps them understand the entire area is a single element that can be selected. And remember to keep that small space in between each containing shape. Another thing to keep in mind is to ensure nested elements have relative corner radii and padding that keeps them concentric with each other. To help you figure out how to size your corners, you can use this simple formula. Corner radius of the nested element, plus the padding in between it and the outer container, gets you the outer corner radius. And remember, in order to keep your corners smooth, be sure to set them to be continuous corners. Throughout the system, every element is concentric to one another, starting from the relationship of the window to the elements in the corners. So remember to keep nested elements concentric with each other, as this will help them feel like they belong together. Now that we've covered best practices to create layouts, let's take a closer look at the components that you'll be using. In this section, we'll show you how to take your knowledge from existing platforms and apply it to spatial computing. We'll start with a brief overview of our input model, then dive deep into how to set up the core structure of an app with a window, tab bar, and sidebar. Then we'll talk about a new way of presenting content. And last, we'll explore modalities. Let's get started with inputs. As we've mentioned earlier, people interact with a device with their eyes, hands, and voice. They do this by looking at elements and tapping their fingers, primarily from a distance. However, people can also select an element directly by touching it. Additionally, people can also connect a keyboard or a trackpad, allowing them to interact with the system with a wide array of inputs.

Interacting with the system feels magical, but it means elements have to provide the appropriate feedback for each of those models. All of our system components have been built to support each input. So, use system components. They help you create interfaces quickly so you can spend time finding what makes your app unique in this new medium. To learn more about inputs, be sure to check out the session "Design for spatial input." Now, let's walk through some examples.

If you're familiar with iOS, many of the components in this platform will look familiar to you. Let's begin with a window, tab bar, and sidebar. Let's start with the core structure of an iOS app. You need a window, which has an opaque material and provides the canvas for all elements to sit on top. As Miquel said, the window is made of the glass material with a window bar below that allows people to move your app fluidly in their space. Similarly, on top of it, you'll have your content.

On iPhone at the bottom of the window, laid out horizontally, you have a tab bar controller for your main navigation, giving people quick access to the top-level sections of your app with just a single tap. Here, the tab bar is vertical, floating in a fixed position on the left side of the window. Designed to be out of the way and easy to access, it's there where you need it, allowing people to see where they are in your app without taking away from your main content. In general, you want to keep those feeling light, so avoid having more than six items. When people look at the tab bar, they can quickly select an item. And if they look at it for a little longer, It automatically expands, showing labels for each section. When people look away, it automatically closes again, allowing people to focus on the content. If you need to provide sub navigation, like with an album in this example, sidebars live in the window alongside the tab bar. This gives people a clear sense of where they are in that tab. Now that we've covered the core structure of an app, let's get into a new presentation style we've added, ornaments. In the Photos app, we have a floating accessory element at the top center to navigate between years, months, and days. On this platform, since we're not bounded by a screen, that accessory is placed as an ornament at the bottom, slightly in front of the window. This gives people additional persistent controls that are easy to access using depth to create hierarchy. Ornaments are a great way to present toolbars. They allow people to perform quick actions related to the content in a convenient location. And they add a bit of depth to your app. Since ornaments are usually a collection of buttons and sit on their own glass container, this is a perfect place to use borderless buttons. In this case, it's clear that these are interactive elements, and they'll still get the hover effect as people look at them. Here's another great use of an ornament in the Music app. The Now Playing controls are presented as an ornament that persists no matter where you are in the app. This allows people to search for their next song while being able to control their music. When ornaments sit at the bottom edge of the window, place them so they overlap the bottom edge by 20 points. This helps ornaments to feel like they're integrated with the main window without blocking too much content. And they look really nice, pulling colors from the content through the glass as you scroll.

Ornaments can appear and disappear entirely. However, it is only recommended while focusing on a single piece of content. For example, looking at a photo or watching a movie. Like in this example, with just a tap, we give people quick access to important controls without distracting them from the main experience. Additionally, ornaments can expand, presenting additional content, or even have their own navigation hierarchy. So find where ornaments fit in your app. They are flexible and give your app depth without occluding content. Now that we've covered ornaments, let's get into menus, popovers, and sheets. On iPad, menus align to the leading edge of the invoking button, and the button dims as the menu opens. Popovers point to the invoking element and the navbar becomes inactive. On this platform, menus and popovers can expand outside the window. They appear centered by default, ensuring that the content always appears right where the user is looking. One thing to always keep in mind is to change the button that invokes them to its selected state. On this platform, we always use black labels on a white background to show buttons are selected. This helps people have clear feedback of which button invoked the popover without the need of arrows. As a general rule on this platform, avoid using buttons with white backgrounds unless they are selected. Finally, let's talk about sheets. On this system, sheets are presented as modal views, and they appear at the center of the app. Modals contain the same Z position as the parent window. The parent window pushes back and dims. This helps focus the experience and prevents people from interacting with the parent view until the sheet is dismissed. If you need to present another sheet, a secondary modal can appear in front with an additional layer of dimming pushing everything back. So let's look at it again from the side. Because we're stacking all these elements in Z, consider using push navigation for nested views. Here's a quick example of push navigation in a modal. The secondary view will present a back button instead of close. Also, notice how the close and back buttons are in the top-left corner. As a system pattern, always place close buttons on the top-left corner. Miquel: Now that you understand the fundamentals of building an app and using system components, think about what makes your app special and how to take advantage of the space outside windows to create rich spatial moments. For example, in Photos, we kept the browsing experience familiar and focused on creating new ways to experience your images, like Spatial Captures, which give you an immersive way to relive your moments and experience content in a unique and special way. We've covered how to design and develop accessible, familiar, and intuitive apps, using materials that blend with your surrounding. Interfaces that take advantage of different layers of depth and endless components that are not only visually appealing, but also familiar and easy to use in this new platform. All our system components have been designed to support our wide array of inputs, while maintaining consistency with the rest of the Apple ecosystem so you, as a designer or developer, can focus on generating groundbreaking immersive and spatial experiences. Thanks for watching. Both: Ciao! ♪