When/How to use Deaf Accessibility Tools
According to the World Health Organization (2016), over 360 million people in the world have mild to profound hearing loss. To narrow down further, approximately 48 million Americans have some degree of hearing loss. That is roughly 15% of the population who are hard of hearing or deaf. And that’s a lot.
Within the video game industry, we strive for players of all backgrounds to enjoy our games, including but not limited to those who are d/Deaf and hard of hearing (d/Deaf/HoH). So how can we make our games more accessible?
The two major tools to promote deaf accessibility in video games are (1) subtitles/captions and (2) visual cues. This basic guide will define the tools and outline when/how to integrate them into game design (tl;dr at the end).
Table of Contents
- Subtitles and Closed Captions:
- Subtitles vs. Closed Captions
- Best Practices
- Visual Cues
- Integrating Deaf Accessibility Tools into Video Games
- Why should the tools be added?
- When should the tools be added?
- How should the tools be added?
Subtitles and Closed Captions
Summary: Subtitles = Dialogue // Closed Captions = Dialogue + Sound Effects
Subtitles derive from text within a written script or spoken dialogue. As a tool, subtitles are designed under the assumption that the user is hearing. Still, there are many benefits to subtitles for both hearing and d/Deaf/HoH individuals. For example, if the dialogue is in another language, subtitles can provide a thorough and accurate translation for non-native speakers. Here is a quick example.
Hearing individuals will use subtitles for varying purposes, including but not limited to understanding poor audio quality, heavy accents, and dialogue in a noisy environment. Additionally, subtitles serve a functional purpose for those who are watching a video on their lunch break at work or trying not to wake a baby.
Generally, subtitles can also assist in comprehension. If content is designed to be informative (i.e. a lecture or “how-to” video), subtitles can provide extensive technical information in written form to assist users in following along with the dialogue. Additionally, users find that subtitles help with following a story by alleviating concentration fatigue. They also improve literacy, which is an added plus.
A friend recently joked that once he turned on subtitles in Red Dead Redemption 2, he found that he could never go back to playing games with the subtitles off. Subtitles complimented the game audio and dialogue, as well as assisted him in comprehension.
His experience is reflected by recent data released by Ubisoft, which shows that 95% of Assassin’s Creed Odyssey and 97% of Far Cry: New Dawn players kept subtitles on.
Red Dead Redemption 2 (2018)
However, though subtitles are helpful for deaf and hard of hearing individuals, it does not immediately equate to equal access.
The term “closed captions” is often used interchangeably with subtitles, but both features are different. Closed captions, often displayed as [cc], are intended to not only accommodate for dialogue, but also other sound effects. Oftentimes, closed captions will include indicators for music or background noise (i.e. explosions, gunshots, breaking glass, etc.). As a tool, closed captions are designed under the assumption that the user is deaf or hard of hearing. The intent of the tool is to provide equal access to d/Deaf/HoH individuals.
It’s a Dog’s Life With Bill Farmer (2020): Two examples of closed captions from a Disney film.
If there is a sound effect or if something occurs off-screen, then captions will include a written cue to notify deaf and hard of hearing individuals.
The main benefits of closed captions is that they provide additional context and therefore, improve immersion for d/Deaf/HoH individuals. Users are able to focus more on the story and content, rather than spending their time trying to decipher subtitles and wondering what they might have missed.
NOTE: In the entertainment industry, captions are categorized as either open or closed. What makes them “closed” is that a user can toggle the captions on and off, whereas “open” captions are embedded in the product and cannot be turned off. In the video game industry, captions are almost always categorized as “closed.”
Best Practices for Subtitles and Closed Captions
Now that we know the difference between subtitles and closed captions, how do we add them to a game? The rule of thumb is to consider the following:
Accuracy: Within the FCC closed captioning rules, it states that all spoken words in dialogue needs to be accurate to the fullest extend possible. Keep in mind that scripts may not be the end product, as voice actors may add their own dialogue. If the team is strapped for time, Rev, GoTranscript, and Ai-Media are popular closed captioning services. However, game companies are still responsible for formatting.
Labels: Given the nature of video games, the text should always label the speaker(s). Typically, it’s recommended that speakers are labelled using all capital letters. Other designers will use color-coded names to indicate speakers. Just make sure that it is consistent. Additionally, if the speaker is off screen, use an arrow to indicate dialogue is occurring out of view.
Here are some examples:
< SUSANA: When I know more, I’ll let you know.
JAMES: What are you doing?
MAN: I don’t want to do that now.
BOTH: We want to play too!
ALL: Happy birthday, Timmy!
If the same speaker is continuing on to the next line or in a fresh set of subtitles/captions, the speaker does not need to be labelled.
Intonation and emotion: When applicable to the dialogue, intonation and emotion can be labelled within the text. The format is as follows:
(SLURRED): But I love you!
JAN (WHISPERS): Don’t let him near you.
Another option is to integrate intonation and emotion into the displayed text. For example rather than labelling text as (STUTTERS) it can be displayed as JOHN: I’m g-g-going home. Here are some other examples:
Everything that matters…is a mystery
What do you think you’re…?!
You mean you’re going to marry him?!
It’s a BOOK, damnit!
Lastly, a single subtitle/caption can indicate critical intonation that is not affiliated with a speaker.
APPLAUSE DROWNS SPEECH
For additional technical examples, please check out the BBC Subtitle Guidelines.
Accents: Only indicate an accent when it is critical to the dialogue. Text can be displayed as
Steve [AMERICAN ACCENT]: I don’t know what you mean.
The Office (2005)
Music: Always label music and it’s source. For instance, if someone is whistling a tune, text should read HE WHISTLES AN UPBEAT TUNE. When applicable, describe incidental music using the following format:
MUSIC: “God Save The Queen”
Designers can also combine music source with incidental music, such as JOHN HUMS “God Save The Queen”. If there isn’t a source and the song name is not necessary, music can be labelled: EERIE MUSIC or UPBEAT MUSIC. Label mood music only when required.
Sound Effects: For closed captions, critical sound effects should be properly labelled in the text. The BBC rule of thumb:
- Sound effects should be capitalized
- Describe the sounds, not actions (i.e. GUNFIRE)
- Subject + verb should be brief (i.e. FLOOR CREAKS)
Hamilton (2020): The speakers are indicated and the text is properly labelled being sung.
Font: The font should be simple and legible. Rev recommends the following fonts: Lucida Grande, Arial, STIXGeneral, Verdana, Helvetica Neue, Times, and Futura. Essentially, any standard document-style font.
Size: Make sure the size is appropriate (and can be properly viewed from a couch). Well-researched standards state that the font size needs to fit within a minimum line height of 8% of the active video height. Other game experts recommend size to be a minimum of 46px for a 1080p screen and allowing scaling both up and down from there.
BBC Guidelines (2020): 0.5em = 8px
Text background: To improve legibility, the text should always contrast with the background. To assist with contrast, it is recommended to include a text background (opacity 50%) to accentuate text. On both sides of each line, the rectangular background should have a 8px gap (as seen above).
Color: BBC guidelines recommend that subtitles/captions should be white, yellow, cyan, or green (in order of priority). Colors can also assist in distinguishing names and other core concepts within combat. Just make sure to not go overboard and be mindful of colorblindness.
Gears of War 5 (2019): An example where the text is an appropriate size and the background properly contrasts with the text to improve legibility.
Wolfenstein II: The New Colossus (2017): An example where text is too small, the contrast between the background and text makes it illegible, and the speaker is not labelled.
Darkest Dungeon (2016): The text is not a standard color and does not contrast with the background, making it illegible.
Positioning: Traditionally, text is horizontally displayed on the bottom of the screen. Depending on the interface, it may be helpful to move the text to another location. Just make sure that the text is not obstructed by the gameplay and vice versa, as according to FCC guidelines on positioning.
Length: d/Deaf/HoH users utilize active reading skills, as they constantly need to flick their eyes between text and the gameplay. Therefore, do not present too much data all at once, as users should be able to easily read along with the dialogue. Again, BBC states that only two (at most three) lines should be displayed at a time and a single line should not exceed 38 characters.
Break at natural points: Text should always break in logical points. Ideally, a line should be broken in the event of punctuation or the changed speakers. If the game requires split speech, it is advised by experts to avoid splitting in the event of a(n):
BBC Guidelines (2020)
Left, right and center justification can be used to identify sound location or speaker. Just make sure breaks also need to consider eye-movement.
BBC (2020) example of an unnatural line break.
Shadow of the Tomb Raider (2018): An example where the text is properly contrasted and easy to read, but includes an arbitrary, unnatural line break.
Time: When text appears on the screen, users should have an ample amount of time to read the text. Generally, users can read 160-180 words per minute. Channel 4 recommends a rough estimate of 2 seconds per line. But when adding combat or character movement to the equation, it is safe to say that readers may have a decreased reading rate. Since there isn’t data on this topic yet, be ready to have playtesters available. Additionally, as users cognitively adjust to the game’s subtitles/captions, make sure that the display time is consistent.
Synchronization: The text within the subtitles should always match what is happening in the gameplay. According to FCC guidelines, text should be displayed with corresponding dialogue and sound effects “to the greatest extent possible.” It wouldn’t be fun to read a spoiler before it happens on the screen.
Outcast – Second Contact (2017): The text is too long for users to read and comprehend in an appropriate amount of time, thus slowing down gameplay.
Assassin’s Creed Odyssey (2018): An example of text that is presented at an appropriate length, uses a clear text color with a suitable background contrast, and properly identifies the speaker.
Directionality: When possible, always indicate the direction of the dialogue, music, and sound. It can be as simple as an arrow point left or right.
Minecraft (2020): An example of closed captions indicating the location of the sound using < or >.
Consistency, consistency, CONSISTENCY: Whether the team decides to use colors or brackets, make sure text is consistent. Doing so will ensure intuitive use for the product.
Games require a large amount of information to be translated to the user. The most commonly utilized method for helping a user understand and react to a game’s mechanics or intent is through the use of visual cues. Essentially, visual cues are used within game design to notify users of pertinent gameplay information so that the user may make informed decisions.
For deaf and hard of hearing users, visual cues are the key to success.
Valorant (2020): An example of a user taking damage and the game notifying the user through a damage directional indicator.
Visual cues can come in many shapes, sizes, colors, and forms. More often than not, designers will use visual elements that users are already familiar with, as doing so will make gameplay intuitive for users. Depending on the gameplay and designer’s intent, visual cues are often accompanied by a sound cue and at times can be subtle.
Some common examples of visual cues include:
- Damage directional indicators
- Pathfinding hints
- Timing cues during combat (i.e. glint on weapon before a swing)
- Highlights over loot and drops.
Remnant: From the Ashes (2019): An example of a user being visually notified of drops.
Integrating Deaf Accessibility Tools in Video Games
Now that we know about the tools to promote deaf accessibility, what do we do with them? It’s hard to say “these types of games should have these types of features” since genres are very much blended together. However, here are some general design suggestions.
Why should the tools be added to the game?
When it comes to Deaf Accessibility, there is no single solution. It’s easy to add closed captions and call it a day. But accessibility involves interactive and integrated design.
For example, only adding closed captions does not immediately equate to Deaf Accessibility. Hearing people’s processing speed index benefits from both visual and auditory processing. However, from d/Deaf/HoH individuals, auditory processing is decreased or removed from the picture.
To create a more accessible product, designers will need to add additional visual information to assist with d/Deaf/HoH processing. The downside of only including closed captions is that active reading takes up a large portion of cognitive flexibility. This might work for TV or movies, but video games require player engagement. What if the gameplay is unpredictable and includes lots of sounds? For d/Deaf/HoH users, they’re forced into a multi-tasking dilemma and oftentimes, experience cognitive overload. They do not benefit from the interconnection between auditory and visual processing.
This is why the Deaf Accessibility tools are critical and, more so, why there needs to be healthy balance of both subtitles/captions and visual cues.
When should the tools be added to the game?
Accessibility should always be considered in the early stages of development. The reasoning is that oftentimes, foundational design choices prevent the application of accessible features.
Within the game development pipeline, a game typically goes through five stages: concept, pre-production, production, launch, and post-launch. In pre-production, timelines are made, storylines are drafted, gameplay mechanics are determined, and level design is established. The game is taking shape and details are being defined.
Deaf accessibility tools need to be integrated during this pre-production phase. Once the game enters production, it is extremely difficult to go