Accessibility: AirPlay for Apple Watch, door detection, automatic subtitles

Apple has released a new batch of accessibility-related news that will arrive over the next few months in all its products. The company is not waiting for announcements from WWDC, which will take place in less than a month, which seems to be becoming an annual tradition since it had already done it in May 2021.

Two new features announced by Apple: detection of doors on the left, mirror mode of the Apple Watch on the iPhone in the center and on the right.

Door detection

Users who have vision problems will now be able to use an iPhone or iPad equipped with LiDAR to detect doors. Upon arriving near their destination, they will be able to pull out the device and get several pieces of information about the door. They will know how far they are from the opening, whether the door is open or closed and whether to push, turn or lower a handle to enter. Finally, the OCR integrated into iOS will take care of reading any inscription on the door to them, just to check that it is the right place.

Door detection in the Magnifier app.

This new feature will be integrated into the Magnifier app, where it will be offered as one more mode, alongside people detection and image description. In addition, Maps will add sound and vibrations managed by the haptic engine of the iPhone to allow people who cannot see it to find the starting point of a route.

Control Apple Watch from iPhone

A future version of iOS will be able to display the screen of an Apple Watch associated with an iPhone in video mirroring via a variant of AirPlay. We will see a representation of the watch on the smartphone screen and the user can also control it through this. This will be useful for users who need accessibility features like voice control, which are not available on watchOS.

All functions of the Apple Watch will be accessible through this, including reading the heart rate and the level of oxygen in the blood, or even an app like Mindfulness, which does not exist elsewhere. An Apple Watch Series 6 will be required at least to benefit from this new feature, older models will not be supported.

Another novelty, this time only on the Apple Watch side: a new gesture will be added as part of AssistiveTouch to control watchOS. Quick actions will allow, for example, answering a call, pausing music playback or an exercise, or even taking a photo with a two-finger pinch. Apps will be updated to support this new gesture.

1652824659 664 Accessibility AirPlay for Apple Watch door detection automatic subtitles
A two-finger pinch will notably allow you to manage the Exercise (left) and Mindfulness (right) apps.

Automatic captions on all Apple devices

Users with hearing problems, or simply those who cannot listen to their device at any given time, will be able to activate an automatic captioning function. Available on iPhone 11 and above, iPads with Apple A12 or better and Macs with Apple Silicon chip, this new feature can be used for all audio content, whether it’s a video displayed by the device, a phone call, a videoconference with FaceTime or another app. We can even use it for someone who speaks beside you.

Automatic subtitles in action.

Macs will even benefit from an additional function: the user will be able to type an answer and macOS will take care of broadcasting the audio version on the speakers of the computer. Apple specifies that no server will be involved for this novelty, all processing will be done locally.

Which implies that besides the technical limitation on the compatible devices, there is also a serious limitation on the language. Only English-speaking users in the United States and Canada will initially be able to benefit from it when the feature is released in beta later this year. We imagine and hope that the deployment in the rest of the world will be done quickly when this novelty is finalized.

Other novelties

It is not finished ! Apple also mentions twenty new languages ​​for VoiceOver, the function that reads what is displayed on the screen, including Bengali, Bulgarian, Catalan, Ukrainian and Vietnamese. Dozens of new voices will also be added.

Sound recognition, which is already integrated into iOS, will now be able to recognize a sound that was not provided for in advance by Apple. The system will ask to save it five times in a row and it will then be able to recognize it and follow up on an action. The most obvious use is ringing a doorbell, but one can imagine other uses, with specific alarms in particular.

1652824660 763 Accessibility AirPlay for Apple Watch door detection automatic subtitles
Custom sound detection on the left, new settings in Books on the right.

The Books app will add more themes and more advanced text customization options. We can adjust the spacing of lines, words and even characters, change the thickness of the font, etc. Siri will also be able to wait longer before responding to you, with a setting that can be adjusted. In another area, we can configure several controllers to play, so that a user can ask for help if he cannot handle a controller alone.

And since Thursday is World Accessibility Awareness Day, Apple will be hosting several events throughout the week. In several Apple Stores, sessions will be dedicated to accessibility. A new shortcut will be added to the app of the same name, to recommend accessibility features based on user needs. Apple also announces the arrival in Canada of the SignTime function, which allows you to contact an Apple Store or Apple support in sign language.

In addition, the services are not left out, with specific content for several of them:

  • Fitness+ made room for sign language, with a specific lesson to highlight accessibility features on the service and dubbing of all videos (ASL only at the moment). The “Place à la marche” or “Place à la course” sessions will be renamed to be more inclusive for all those who are in a wheelchair.
  • Maps will integrate guides for accessible places, including one on American national parks which have adapted programs.
  • L’App Store will post articles about apps and developers in the context of accessibility.
  • Adapted content collections will also be added to Apple Music, Books, Podcasts and the Apple TV app.

We would like to say thanks to the author of this short article for this outstanding material

Accessibility: AirPlay for Apple Watch, door detection, automatic subtitles

You can find our social media profiles , as well as other related pages