Catch up on accessibility in SwiftUI

Hi I’m Tommy an engineer on the accessibility team. Accessibility is a fundamental part of any great app, allowing everyone to create, connect, and play with the experiences you build. SwiftUI helps you bring those experiences to life across Apple platforms. Today, we’ll peek under the hood at how SwiftUI provides built-in accessibility out of the box and tools to help refine and craft your app’s accessibility.

Next, we’ll discuss places you may need to provide more information to SwiftUI to improve the accessibility of your views.

Finally, we’ll explore how to build accessible rich interactions from taps to drag and drop.

SwiftUI is built from the ground up with accessibility support for built-in views and to make it easy to provide additional information to accessibility technologies to refine the experience of your app. I’ve been building an app in SwiftUI to share memories from my favorite past time: visiting the beach. The app lets me send a message describing those beautiful ocean views and with just a tap, I can attach a location, a recording of the waves, or rate the beach. Everyone can comment about the trip, and I can favorite and reply to their messages.

I can even create a custom sound for when different friends reply! This is great, but lets explore how SwiftUI makes my features accessible.

SwiftUI creates accessibility elements as one of its primary outputs. Accessibility elements are the fundamental building block that accessibility technologies like VoiceOver, Voice Control, and Switch Control use to present and interact with content in your app. An Accessibility element represents one or more views and provides attributes to describe the contents of a view and actions which expose how a view can be interacted with from taps to complex gestures.

Accessibility technologies like VoiceOver only interact with apps through their elements so only content from views that are included in an element will be accessible to them.

Here is the toggle view to control whether my friends can comment on a trip I’ve posted about.

This toggle is declared in my view body, SwiftUI uses that to form outputs, including what is shown on screen and accessibility elements.

For accessibility elements, the comments label is used to create the element’s label. The isToggle and isSelected traits are applied to the element. And a press action is provided to toggle the setting.

When VoiceOver focuses on the toggle the element is described, and a double tap will perform the press action. "Settings, heading, comments, switch button off, on" SwiftUI combines the text and toggle into a single element. Multiple views can be represented by a single element to simplify navigation and connect relevant information together. When customizing the look and feel of the toggle to fit my app, preserving the attributes and actions for its element is important. SwiftUI’s powerful view styling system is the key to letting you change how views visually appear without losing their built-in accessibility support.

Attributes and actions are applied to the view even when changing a view’s visual style.

When changing the style using the toggle style modifier, the visuals of my toggle will update but, its element keeps its label and traits.

And when the custom toggle is tapped, its element’s attributes reflect the new state of the toggle.

View styles are supported across many different controls like button and toggle, groupings like label and labeled content, and indicators like progress view and gauge. Use the styling system over creating a custom view whenever possible otherwise, you’ll have to re-apply the same accessibility properties you could be getting automatically. Using styles and built-in views will provide a good accessibility experience in your app. But the more information for accessibility you can provide to SwiftUI the better the experience will get. You can provide this information using the accessibility modifiers.

Accessibility modifiers let you customize how a view is represented by an accessibility element. Attributes like a label or trait can be added to a view. And actions for interactivity like custom gestures can be exposed.

The modifiers even allow you to combine views together into a single element to improve navigation.

Before using modifiers to customize the accessibility of your views the best way to understand where you can improve the experience for your app is by using it with a technology like VoiceOver.

Lets check out a recent trip to the beach in my app with VoiceOver to understand what can be improved.

"Weekend adventure, heading" "Saturday, It was a beautiful weekend! The waves were calm, the sun was shining on the bridge, and there wasn't a cloud in the sky. Image" "Looks like a wonderful time, favorite, button, reply, button, Nick" "Absolutely beautiful, favorite, button, reply, button, Jack" SwiftUI has created a good experience for my app. VoiceOver was able to navigate all the content and refinements like indicators for headings and buttons were provided. There were though three things I noticed that can be improved. First the comments section took a long time to navigate with so many elements. It also quickly got confusing to know which button was related to each comment. And there also were some views like my unread indicator that weren’t being described by VoiceOver. Let’s examine how I can make this experience even better.

Here’s my comment view. I have a horizontal stack of elements representing a comment’s message with an unread indicator and buttons to reply and favorite the comment. SwiftUI creates three accessibility elements for this view, one for my title, and two buttons for my actions. My stack and spacer views aren’t elements since they just control the visual layout of my view. Theres not an accessibility element for the unread indicator by default because its represented as a shape view, so lets first add a label to describe if a comment is unread.

The accessibility label modifier will first create an element for my indicator view since it doesn’t have one by default and will set the element’s label to be unread.

When the indicator view is unread the element will always be visible to accessibility technologies. SwiftUI helps when the comment becomes read. When a comment is read the indicator’s opacity becomes zero and is visually hidden. SwiftUI will also automatically hide the element from accessibility technologies.

To simplify navigating the comments view I can make each stack a single element and turn each of my buttons into actions for this element. By applying the accessibility element children combine modifier the attributes and actions of multiple elements are combined together.

The label of the element will be the combined labels of all of the views. Important traits will be merged and any actions from controls like buttons will be turned into custom actions. Now a single element represents a message, whether its unread, and its actions, making it much easier to navigate my comments.

When I navigate each of those comments again with VoiceOver the experience is much easier to understand. "Unread, Looks like a wonderful time, Nick, button" "Favorite, reply" While SwiftUI has done the heavy lifting to make my views accessible, providing more information to accessibility technologies in the right contexts can fix missing content and provide a more enjoyable experience. I’ve made my beach trip messages so much better, but I want to give a little extra love to some of my friends comments.

In my app, when I double tap on a favorited comment it becomes a super favorite. Super favorites use a new symbol to indicate how important they are but this change causes a new problem. Before, my combined action for favoriting was correctly labeled but now VoiceOver announces the sparkle name for super favorites.

"Unread, absolutely beautiful, Jack, button" "Favorite" "Reply" "Unread, Hope you wore sunscreen, Beth, button" "Sparkle, reply" When using symbols in views, many provide default labels when a meaning can be inferred, like a star or a reply symbol. Others need an explicit label to be added if they symbolize content and fallback to their name as the label. Even when relying on a symbol's default label in a view, always try using it with an accessibility technology to make sure it accurately describes your interface. For super favorites, I just need to update the incorrectly labeled sparkle symbol.

I can add an accessibility label using its modifier, but when its applied, it will override the correct default SwiftUI already provides to us when the message is not a super favorite.

In iOS 18, you can now add an isEnabled parameter to accessibility modifiers. When the modifier is enabled, the attribute is applied to the view and when its not the modifier doesn’t go into effect. Here the super favorite label is only applied when the comment is a super favorite and SwiftUI will fallback to the star symbol’s default label when its not a super favorite. This is great when SwiftUI provides the right accessibility for you by default but sometimes you need to conditionally customize it.

Going back to the comment view in my app, the super favorite action is back and perfectly labelled. "Unread, Hope you wore sunscreen, Beth, button" "Super favorite, reply While missing information, like a label should always be added, providing simpler ways to surface information that take multiple interactions can elevate the experience of using your app with an accessibility technology. A great opportunity to explore these enhancements are when views dynamically appear from hover, key presses, or gestures. I’ve been working on bringing my app to macOS, and a great interaction I want to add is hover to show my trip attachments.

When hovering over the trip image view attachment buttons including its location, a recording, and the beaches rating appears. This removes clutter so the focus remains on the beautiful photos of the beaches, but there are some considerations to make sure these attachments are accessible. Content that dynamically appears like a hoverable view may take longer to navigate to with an accessibility technology or an alternative input source to touch like a keyboard or a switch.

When moving a mouse cursor, the view appears and the interaction can be performed in a matter of seconds.

But VoiceOver has to do a lot of steps to navigate to this view, the hoverable view has to be focused, the hover has to be invoked using a command, once the view appears the subview has to be navigated to, and then after the interaction focus has to return to the original view.

This interaction can be simplified by appending the accessibility attributes and actions of the hoverable view as part of the main view.

Not every trip may have a recording or a location attached. So the exact number of actions or labels to add to the trip view is not fixed. SwiftUI provides accessibility modifiers for just these cases. Let’s start with actions.

Here's the hoverable overlay for the trip view. To make these controls easier to access, I can turn them into custom actions on the trip view.

The accessibility actions modifier accepts the same view for the trip attachments. It extracts controls and transforms them into custom actions on my trip view. Now, the same logic can be shared for the overlay and the custom actions for the trip view. With this change, VoiceOver can now perform each attachment without having to present the overlay.

"Saturday, It was a beautiful weekend! The waves were calm, the sun was shining on the bridge, and there wasn't a cloud in the sky. Image" "Actions menu, 2 items, location, recording" There is one piece missing which is the rating I have for my beach. The rating appears on hover but its a centerpiece of my trip, so I want to make this more prominent on my trips accessibility element.

When important information requires multiple interactions you may want to consider appending that information to elements that don’t include it visually but are easier to navigate to.

If you’ve modified the attributes of an element, such as combining multiple elements, you may want to include additional content thats provided by decorative views that symbolize content.

Both of these cases require modifying the label of a view with content that may not be static. SwiftUI provides tools to handle both of these cases.

I want to append the rating to my trip element’s label to simplify finding it with VoiceOver.

A regular label modifier would wipe out the great descriptive contents that SwiftUI already provides to the element. And because the rating only sometimes appears I only want to include it on some messages.

The accessibility label modifier now also accepts a view and will extract the text from the view and set it on the trip element’s label. It even provides a placeholder to the view’s existing label, so the rating for my trip can be combined with the existing label that SwiftUI has created for the trip element.

Going back to the trip view, VoiceOver now correctly reads the rating and the post of the trip as a single label. "Half star, Saturday, It was a beautiful weekend! The waves were calm, the sun was shining on the bridge, and there wasn't a cloud in the sky. Image" Sounds good. Hover is just the start of where you may consider using these APIs. Make sure that any change in content is accessible to technologies like VoiceOver.

When building interactivity into your app its important to remember to design great experiences for alternative forms of input to touch or a mouse.

One interaction thats at the heart of so many of the experiences you build is Drag and Drop.

Accessibility technologies like VoiceOver and Voice Control work with drag and drop and SwiftUI supports accessible experiences using the "on drag" and "on drop" modifiers.

The drag and drop surface is flexible so there are a few ways to enhance the experience for your app. My app uses drag and drop to create custom alerts for friends when they comment. Lets try it out with VoiceOver. I can drag up to three sounds for a contact to create an alert. I’m using a custom drop delegate to support multiple drop points on a single view, so VoiceOver won’t know about the multiple places that sounds can be dropped. "Synth, draggable" "Drag" "Cheers, draggable, alert, no sounds" "Drop, drop ready" "Drop complete' The accessibility drag and drop point modifiers define points on your view that can be dragged or dropped. On the comment alert view I’ve defined three drop points that describe the three different sounds that I can set for my alert. Each point also provides a label that describes the interaction that will be performed. Now VoiceOver can access each drop point on my alert view. "Synth, draggable" "Drag" "Cheers, draggable, alert, bells, birds" "Drop, drop ready, set sound 3, drop ready "Drop complete' The new experience is great and I can quickly drag a sound to my contact from within my app.

When building drag and drop experiences its important to always try them out with accessibility technologies to make sure the functionality they provide is usable. You might also want to consider a custom action in addition to drag & drop when a drag has finite locations or actions it can perform.

And make sure that if an element supports multiple interactions at different points, each of these points are exposed to accessibility technologies. Interactions can expand outside the bounds of your app into interactive widgets. In widget’s views don’t update in real-time so the accessibility action modifiers won’t work.

App Intents allow widgets to create interactive views like Buttons and Toggles, but there may be places to elevate the experience in your widget through additional custom actions. I’ve been working on a new widget for my app that lets me quickly rate and post about beaches I like to go to. With just a tap I can even update the ranking of my favorite beach. Each of these buttons is labeled by SwiftUI and can be activated with VoiceOver.

Heres my beach view with the three interactive buttons to post and rank a beach. I want to make the experience of using my widget even better with custom actions to perform the most important functions of my widget.

Accessibility action modifiers now accept an app intent which when invoked will perform the intent and update your widget.

Here I’ve added a custom action to set a beach as my favorite beach and a magic tap action to take a photo with just a double tap.

With these changes I can now mark my favorite beach and it’ll move to the top of the list. "Funston beach, no rating, button" "Baker beach, no rating,button" "Favorite" "Baker beach, favorite, button" Now that we’ve covered bringing accessibility to your app, make sure to try your app with an accessibility technology like VoiceOver. Investigate where the accessibility APIs can help make the experience of your app even better. And explore the accessibility sample project to better understand how accessibility APIs can be used to refine common patterns in SwiftUI.

The accessibility team can’t wait to see the amazing accessible experiences that you build in your apps. Thank you.