A new Facebook tool will help users who are blind or have visual impairments “see” photos by describing what’s in them. With automatic alternative text, people using screen readers on iOS devices will hear a list of items that a photo on Facebook contain.
Without the new automated photo captioning tool, screen-reader users skimming their News Feed would only be told that a friend posted a photo. Screen-reader software turns Web pages and documents into synthesized speech for people who are blind or have severe visual impairments. “We really want to start with a set of concepts that frequently appear in photos that add a lot of value to the narrative of the photo,” Jeff Wieland, Facebook’s head of accessibility said.
More than two billion photos are shared every day on Facebook and Facebook-owned Instagram, Messenger and WhatsApp services. The flood of imagery means people who are blind or have visual impairments are often unable to decipher what’s in the photos, leaving them out of the conversation. In its first iteration, Facebook’s new automated photo captioning tool can identify rudimentary things: cars and boats, basketball and baseball, beards and eyeglasses.
In time, Facebook aims to include a much fuller description of what’s in a photo including the identity of the people in it. It also plans to expand the tool to more devices and languages as well as to all of Facebook’s family of apps. For now, the tool is only being tested on iOS screen readers set to English.
Facebook is taking steps to re-engineer its website and mobile apps to make them more accessible. It’s also brainstorming a new generation of futuristic products that harness the power of artificial intelligence to improve the experience of Facebook for people with disabilities.
Automatic alternative text generates a description of a photo using advances in object recognition technology. The tool was requested by users with visual impairments frustrated that they were excluded from the conversation surrounding photos in their News Feed.
