Featured Blog | This community-written post highlights the best of what the game industry has to offer. Read more like it on the Game Developer Blogs or learn how to Submit Your Own Blog Post
Accessibility Heuristics and Evaluation Criteria for Mobile Games
This research article delves into the most up-to-date design guidelines and evaluation criteria applied to mobile game accessibility, while presenting a relevant and comprehensive guide to building better accessible mobile games.
Introduction
Since the inception of the mobile app stores, the popularity of mobile games has grown substantially. With the rising popularity of mobile devices, the number of players with accessibility needs using those devices has risen as well. This warrants an understanding of the most up-to-date accessibility research specifically for mobile game developers to incorporate into their game production process. The worldwide population in 2021 using smartphone devices is 3.8 billion people, a 6.1% year-over-year growth (Newzoo, 2021). Mobile gaming market revenues in 2021 will account for 52% of the global gaming market at a $90.7 billion valuation (App Annie, 2020). This industry is projected to continue growing year-over-year (Knezovic, 2020) and its players are becoming increasingly more diverse. More than one in five (20.5%) players of casual video games have a physical, mental, or developmental disability; this compares to 15.1% of the American population overall who are disabled, according to the latest U.S. Census data (PopCap Games & Chauteau, 2008). Older adult audiences also experience general physical and sensory decline over time (Rienzo & Cubillos, 2020). Mobile-only players between the ages of 37-64 make up 42% of the overall demographic (Globalwebindex, 2020). With this in consideration, the concept of accessibility has in recent years become a rising cultural subject within the mobile gaming industry. It has become even more popular in the face of increased dependence on user acquisition and targeted marketing campaigns for generating future business growth. The accessibility advocacy topic is also coupled with ideas around raising awareness around diverse player needs, expanding usability requirements for players with disabilities, and improving the readability of digital products for the average consumer (Rehman et al., 2021). Accessible content also aims to make accommodations for various human conditions and impairments, for example, blindness and low vision, deafness and hearing loss, limited movement, speech disabilities, photosensitivity, and cognitive limitations (Ballantyne et al., 2018). However, initiatives towards implementing a holistic approach for making mobile games more accessible are often dismissed by the argument of its marginal impact on business growth.
Current Research and Accessibility Guidelines
There are many formal and informal standardized guidelines used by developers and researchers worldwide for evaluating and assessing the effective accessibility of their games. These guidelines, or heuristics, are tools derived from institutions and organizations such as the World Wide Web Consortium (World Wide Web Consortium [W3C], 2018), Nielsen Norman Group (Nielsen, 2020), AbleGamers Charity (AbleGamers Charity, 2018), Game Accessibility Guidelines (Game Accessibility Guidelines, 2012), and the International Game Developers Association (Bierre et al., 2004). They are also adopted by private development companies for broadening their player-base and even governments for writing and passing non-discrimination policies or laws. Although these heuristics are primarily used and applied to the development of games on all platforms, they can be used to generate comparable and equally effective guidelines specifically for mobile games.
Many of the heuristic guidelines defined in the research of this paper have evolved from several years of iteration and continue to grow in scope. Through more than two decades of rapidly evolving technologies, the development of evaluation criteria for accessible mobile games gathers strong influences from several core industries including the web, consumer products, and information technology. A strong example would be the W3C’s Mobile Accessibility Guidelines which are derived from the Web Content Accessibility Guidelines (WCAG) 2.2 and which are now in their fifth iteration since 1998 (World Wide Web Consortium [W3C], 2021; World Wide Web Consortium [W3C], 2020). It was through these guidelines which established commonly led accessibility principles and standards of perceptibility, operability,understandability, and robustness in UX design. The principle of perceptibility states that information and user interface components must be presentable to users in ways they can perceive. This is one of the most misused in mobile gaming considering the small screen space of mobile devices. Guidelines under perceptibility include providing reasonable default font size for content and interaction controls, minimizing the set of presented information on screen to a certain degree, and maintaining specific contrast ratios between legible text and background (World Wide Web Consortium [W3C], 2020). The principle of operability states that interface components and navigation must be sufficiently operable by the user. This is the critical aspect to reaching effective player interactions with the game in hand. Guidelines under operability state that interactable icons such as widgets must be big enough and have enough distance from each other to ensure safe targeting by touch (W3C, 2020). Moreover, widgets must be easily reached on the screen considering the limited range of the thumb and the way the user may hold the device. The principle of understandability states that information provided by a mobile application and its operation of user interface must be understandable. Mobile games lose their engagement when the player cannot understand its content or how to control it. Guidelines under understandability include clearly indicating that interactable elements (such as widgets) are distinguishable from non-interactable elements, maintaining consistent layouts and iconography, and utilizing native-device meta-data for detection by screen readers (W3C, 2020). The principle of robustness states that the application’s content must be correctly applied across all devices the user attempts to access it with. This has direct implications on multi-platform functionality and testing of mobile games. These guidelines also suggest that developers work with emulator tools such as Apple’s Accessibility Inspector, which allows them to check the accessibility of widgets across all device formats (Dobosz, 2017). Developers need to verify robustness in the game and on every device model for which it is intended to be played. The admissible implementation of these principles may also be confirmed by local regulations and laws which exist to allow acceptable levels of accessibility for software.
Governments of the United States, Canada, and the European Union have adopted many of the success criteria for accessible software under law using the WCAG. Since the establishment of these laws, many private companies have also introduced initiatives and formed proprietary accessibility guidelines in the development of mobile technologies. Many companies have published their guidelines in an attempt to drive development forward showing unique accessibility solutions. One example is IBM’s Accessibility Handbook which contains three specially crafted checklists for designers, developers, and project managers working in information technology. These checklists include verifications of focus enablement, color and styles, content structure, clarity, and successful use of the device’s voice recognition or screen reader settings (IBM Design Accessibility Handbook, 2017). Checklists like these serve as a positive and proactive example for how game developers could move forward with making their own checklists for various crafts and teams associated with the game making process. Interestingly, there have also been attempts by technology developers, like Apple Inc., to increase accessibility and usability of applications using third-party software. By scanning applications through this tool, it enables hardware-based screen readers to operate on top of formerly non-operating applications (Zhang et al., 2021). This research has shown significant improvements in the performance of user interactions using mobile devices and could provide the industry with another constructive approach to developing tools with similar considerations.
Another example of a published industry-recognized set of accessibility guidelines is the Game Accessibility Guidelines (Game Accessibility Guidelines, 2012). These guidelines have been a collaborative effort between multiple studios, researchers, and academics to produce a developer friendly reference for ways to maximize accessibility in games. The guidelines are broken down into six categories which apply to each of the key disability groups: motor, cognitive, vision, hearing, speech, and general. Each category is divided into three difficulties which reflect the effort and effectiveness of each accessibility feature: basic, intermediate, and advanced. Basic features can be applied to a wide range of game mechanics and include allowing control remapping, toggling on or off haptic feedback, and adequate sizing of interaction controls. Intermediate features require more planning or resources and include support for multiple input devices, adjustability of game speed, and full interface customizability. Advanced features require complex adaptations and include full screen reader support, voice over support, and including players with disabilities as a part of the core testing group of game features.
Furthermore, private research facilities have made significant progress in the mobile Serious games genre by improve gaming experiences for the middle-aged and older adult population (Salvador-Ullauri et al., 2020). Serious games are a category of video games designed for educational purposes. Korea’s Advanced Institute of Science and Technology developed a specific Mobile Game Design Guide supported by a practical list of evaluation criteria using games like Homescapes, Brawl Stars, and Fruit Ninja with a diverse audience having visual, auditory, and cognitive impairments (Lee et al., 2021). These design guidelines include various general mobile game improvements in the areas of interface, game rules, and payment systems. For example, in the area of advertisement and payment, reward items that are provided for advertisements is much preferable to interrupting interstitial advertisements. Similarly, allowing advertisements to be easily closed or canceled reduces frustration with the choice of viewing an advertisement which the player is not particularly interested in viewing completely. They also include sections on increasing usability for players with audio-visual and motor impairments. For example, providing compatibility with supportive equipment such as touch pens or controllers. The method in establishing the guidelines used a direct approach with a representative sample group of the older adult population, a survey questionnaire, and speak-out-loud in-person playtesting. This method is a robust example of how developers could advance their own design guidelines with a user-centric approach.
Finally, with the wide-spanning attention around accessibility in the private and public sectors, several non-profit organizations and foundations have begun working closely with the games industry and developed their own guidelines using a constructive audience of dedicated players with disabilities. The AbleGamers Foundation has created a set of access and challenge design patterns taught under their Accessible Player Experience Design Certification (APX). The 12 access design patterns are guidelines to reach better understanding of content by as many players as possible, including those with disabilities. Several of the access design patterns such as improved precision, clear text, and clear channels are also relevant for mobile game accessibility. Improved precision, for example, explains that specific players with disabilities may need to adjust the precision of actions so that they can successfully target, move, or navigate in the game or its interfaces (AbleGamers Charity, 2018). The 12 challenge design patterns are guidelines for enabling consumption and use of content by these players as well. Several of the challenge design patterns such as helping hand, slow it down, and training ground may be highly applicable to improving mobile game accessibility. Training ground, for example, is a feature that allows players to practice in a variety of ways at their own pace so that they can gain the skills needed to succeed in the game (AbleGamers Charity, 2018). These access and challenge design patterns are rapidly being utilized by several large AAA and mobile gaming companies to implement accessible features for players. The AbleGamers Charity also offers a service for developers to test their games and receive feedback or recommendations from players with disabilities. This could potentially be an even more robust opportunity for mobile game developers to improve accessibility of their games by testing them directly with disabled players. A consolidated list of mobile accessibility guidelines specifically selected from each of the aforementioned resources may be found below.
Mobile Game Accessibility Guidelines
ID | Category | Guideline | Reference |
---|---|---|---|
GL1 | Perceptibility | Text must have a default font size of at least 18px or 14px bold. | (World Wide Web Consortium, 2018) |
GL2 | Perceptibility | “Text can be resized without assistive technology up to 200%.” | (World Wide Web Consortium, 2018) |
GL3 | Perceptibility | “Line spacing is at least space-and-a-half within paragraphs and paragraph spacing at least 1.5 times larger than line spacing.” | (World Wide Web Consortium, 2018) |
GL4 | Perceptibility | Content images not used as decoration may only represent information of text on that screen. | (World Wide Web Consortium, 2018) |
GL5 | Perceptibility | Content must be presented without the use of scrolling in either horizontal or vertical directions. | (World Wide Web Consortium, 2018) |
GL6 | Operability | Time-limits may be disabled by the user before encountering them. | (World Wide Web Consortium, 2018) |
GL7 | Operability | Function that uses dragging movement for operation can be achieved by tappingwithout dragging, unless it is essential. | (World Wide Web Consortium, 2018) |
GL8 | Operability | Interactable elements are within thumb range from bottom corners of the screen in portrait device-orientation and bottom corners of the screen in landscape device-orientation, if applicable. | (World Wide Web Consortium, 2018) |
GL9 | Understandability | Components that have the same functionality are identified consistently. | (World Wide Web Consortium, 2018) |
GL10 | Understandability | Controls are visible on screen and accessible via a menu option on screen. | (World Wide Web Consortium, 2018) |
GL11 | Understandability | Consistent help is given/offered throughout the introduction of new controls and/or functionality | (World Wide Web Consortium, 2018) |
GL12 | Robustness | On device meta-data is integrated for use of native screen reader setting and voice over setting. | (World Wide Web Consortium, 2018) |
GL13 | Robustness | Conformance or partial-conformance to local laws and WCAG guidelines are to be clearly illustrated on screen. | (World Wide Web Consortium, 2018) |
GL14 | Access Design | Improved precision: Players are able to adjust the precision of their actions so that they can successfully target, move or navigate. | (AbleGamers Charity, 2018) |
GL15 | Access Design | Distinguish this from that: Players are able to adjust the presentation of information in a way that allows them to distinguish important elements of information in the game and/or its interfaces. | (AbleGamers Charity, 2018) |
GL16 | Access Design | Clear Channels: Players can select/deselect additional channels (sound, haptic, or VFX) of information so that they can reliably take in the information. | (AbleGamers Charity, 2018) |
GL17 | Challenge Design | Slow it down: Players can reduce the speed, volume, and variety of events in the game so that they can successfully progress. | (AbleGamers Charity, 2018) |
GL18 | Challenge Design | Training ground: Players can practice in a variety of ways without consequence to progression at their own pace so that they can gain the skills needed to succeed. | (AbleGamers Charity, 2018) |
GL19 | Challenge Design | Total Recall: Players can bring up help information about the game that has already been shown so that they can review and refresh their knowledge about them. | (AbleGamers Charity, 2018) |
GL20 | Motor | Touch hit areas must be at least 2.4cm in diameter for tablet, 0.96cm for phone formats. | (Game Accessibility Guidelines, 2012) |
GL21 | Motor | “Make interactive elements that require accuracy (eg. Touch controlled menu options) stationary” | (Game Accessibility Guidelines, 2012) |
GL22 | Motor | Ensure that multiple simultaneous actions (eg. drag or swipe) are not required to progress and may be carried out with an alternative input method. | (Game Accessibility Guidelines, 2012) |
GL23 | Motor | Allow interfaces to be customized and rearranged and resized. | (Game Accessibility Guidelines, 2012) |
GL24 | Cognitive | “Allow the game to be started without the need to navigate through multiple levels of menus.” | (Game Accessibility Guidelines, 2012) |
GL25 | Cognitive | Ensure no essential information (especially instructions) is communicated by text alone.Reinforce essential information with visualcues and/or audio. | (Game Accessibility Guidelines, 2012) |
GL26 | Vision | “Ensure screen reader support for mobile devices.” | (Game Accessibility Guidelines, 2012) |
GL27 | Vision | Avoid placing essential temporary information outside the player’s center-of-focus. | (Game Accessibility Guidelines, 2012) |
GL28 | Hearing | Ensure no essential information is conveyed by sounds alone. | (Game Accessibility Guidelines, 2012) |
GL29 | General | Provide details of accessibility features in-game. | (Game Accessibility Guidelines, 2012) |
GL30 | General | “Include every relevant category of impairment (motor, cognitive, etc.) amongst play-testing participants, in representative numbers based on age/demographic of target audience.” | (Game Accessibility Guidelines, 2012) |
GL31 | Hearing | Provide voice dubbing when presenting stories or speech bubbles. Indicate who is speaking in dialogues. | (Lee, 2021) |
GL32 | Cognitive | Provide game rules and tips during loading times. | (Lee, 2021) |
GL33 | Monetization | Make it easy for players to turn off ads by indicating a close option clearly or an option to pay. | (Lee, 2021) |
Evaluation Criteria
The W3C’s Mobile Accessibility Guidelines, IBM’s Accessibility Handbook, Game Accessibility Guidelines, Korea’s Mobile Game Design Guide, and the APX Certification are all transformational and scientifically established guidelines which have been utilized by industry leaders. But how effective are they as heuristics in the evaluation of accessibility in mobile games? There has been expansive research evaluating the effectiveness of these heuristics. Significant peer-review and development has been made around accessible games for mobile devices. Such research has included the exploration and assessment of direct solutions related to specific disabilities such as visual impairments, hearing impairments, and cognitive impairments (Jaramillo-Alcázar & Luján-Mora, 2017; Jaramillo-Alcázar & Luján-Mora, 2018; Jaramillo-Alcázar et al., 2017). Each of these articles have utilized and applied accessibility guidelines to the game genre of Serious games on mobile platforms and found severe violations of guidelines and accommodating players with their respective disabilities. The discrepancies for players with visual impairments included lack of readable default text size, lack of customizable text sizing options, and options to reduce distracting visual effects (Jaramillo-Alcázar & Luján-Mora, 2017). The discrepancies for players with hearing impairments included lack of consistent audio-visual feedback, use of audio-only cues, or no indication of who is speaking in game dialogues (Jaramillo-Alcázar & Luján-Mora, 2018). The discrepancies for players with cognitive impairments included inability to save settings, not enough hints for accomplishing game goals, and lack of controls customization (Jaramillo-Alcázar et al., 2017). These cases show us that using guidelines can help us also in identifying issues with our current games. In addition to this evaluation research, several qualitative studies, with a conceded effort to legitimize certain industry-leading accessibility guidelines, have found most top-grossing mobile applications to be completely inaccessible (Ballantyne et al., 2018). Here, design heuristics have been applied to identify accessibility issues for players with motor, hearing, cognitive impairments, and those influenced by the consequences of physical and cognitive decline due to aging. The study simply utilizes the guidelines listed from the W3C, reports with an observational analysis of mobile applications used by people with disabilities, selects the relevant accessibility guidelines, and then creates a coherent and digestible list for mobile developers to utilize. This method of evaluation can be then translated into digestible improvements that can be integrated into the mobile game production process. Considering the limited resources game teams often have, it is important to sequentially prioritize the improvements according to severity and total estimated effort before assigning new content tasks (Rehman et al., 2021). Additionally, for scientific analysis of the results, it is essential to have the necessary A/B testing infrastructure in order to assess whether improvements have resolved any violations or how key performance indicators have been impacted. This approach can be a cyclical and iterative process to creating positive change in the user experience for players with disabilities or impairments. A summary and practical step-by-step approach to evaluation criteria with the cycle for iteration can be found below.
Evaluation Criteria and Iteration Cycle
Steps | Description | Department |
---|---|---|
1 | Read and familiarize yourself with the guideline resources. | Business Intelligence |
2 | Extract relevant guidelines applicable to your player base, genre, and game platform. | Leadership |
3 | Identify areas of violation in your game's design and use the accessibility guidelines provided to resolve those violations. | Design |
4 | Distribute each violation into an easily digestible development initiatives/tasks to be prioritized based on severity and place them into the game production process.(Rehman et al., 2021) | Production |
5 | Develop and implement the solutions. | Development |
6 | Test and measure with a relevant and adequate sample group the implementated solutions. | Quality Assurance/UX Research |
7 | Create a report and analysis of the measure to gain an understanding of whether violations have been resolved and the user experience has benefited key performance indicators (KPIs). | Business Intelligence |
Conclusion
There have been decades of research to form several acknowledged accessibility standards for the use of software, web, information technology, and games. After all this time and effort, enough thorough qualitative and quantitative analysis of industrially standardized accessibility guidelines has taken place and spawned an important niche group of researchers and developers who strive for positive change in our industries. Much of this success is due to the growth of rapidly accelerating technologies and passionate researchers around accessibility. Meanwhile, it is still clear that players with visual, auditory, and motor disabilities could experience more improvements in their enjoyment of mobile games if developers applied well-researched methods and solutions. A considerate approach to the application of guidelines using mobile Serious games can show developers ways of providing gameplay enhancements for those who need it. Robust and user-centric approaches by Korea’s Advanced Institute of Science and Technology show developers how to increase validity in the application of accessibility heuristics with a user-centric approach. There are a multitude of resources attached to this thesis research to help developers improve their design, implementation, and testing of features for making more accessible mobile games. Furthermore, as the need for more cohesive guidance and practical processes in the growing mobile gaming industry increases, this research provides appropriate examples and a robust starting point for that increasing need to be fulfilled. Lastly, this research is intended as a source of advocacy for all players to have the ability to enjoy mobile games that are played and loved by millions every day.
References
AbleGamers Charity. (2018, February 13). Ablegamers accessible player experience. Accessible.Games. https://accessible.games/accessible-player-experiences/
App Annie. (2020, December 18). Global Market Share. Retrieved July 29, 2021, from https://www.appannie.com/en/
Ballantyne, M., Jha, A., Jacobsen, A., Hawker, J., & El-Glaly, Y. N. (2018, November 25). Study of accessibility guidelines of mobile applications [Research article]. Conference on Mobile and Ubiquitous Multimedia, Cairo, Egypt. https://doi.org/10.1145/3282894.3282921
Bierre, K., Hinn, M., Martin, T., Mcintosh, M., Snider, T., Stone, K., & Westin, T. (2004, June 29). IGDA accessibility in games: Motivations and approaches. G3ict. https://g3ict.org/publication/igda-accessibility-in-games-motivations-and-approaches
Dobosz, K. (2017). Designing mobile applications for visually impaired people. In J. Estrada (Ed.), Visually impaired: assistive technologies, challenges and coping strategies (pp. 103–126). Nova Science Publishers, Inc. https://www.researchgate.net/publication/321319458_Designing_Mobile_Applications_For_Visually_Impaired_People
Game Accessibility Guidelines. (2012). A straightforward reference for inclusive game design. http://gameaccessibilityguidelines.com
Globalwebindex. (2020). Gaming personas - an audience profile on the digital attitudes and behaviors of six gamer types[PDF]. globalwebindex.com. https://strivesponsorship.com/wp-content/uploads/2020/01/GlobalWebIndex-Gaming-Personas-Audience-Report-2020.pdf
IBM design accessibility handbook [PDF]. (2017). IBM Accessibility Handbook. Retrieved February 26, 2021, from https://accessibility-handbook.mybluemix.net/design/a11y-handbook/
Jaramillo-Alcázar, A., & Luján-Mora, S. (2017). Mobile serious games: an accessibility assessment for people with visual impairments. TEEM 2017: Proceedings of the 5th international conference on technological ecosystems for enhancing multiculturality, (66), 1–6. https://doi.org/10.1145/3144826.3145416
Jaramillo-Alcázar, A., & Luján-Mora, S. (2018). An approach to mobile serious games accessibility assessment for people with hearing impairments. In Proceedings of the international conference on information technology & systems (icits 2018) (pp. 552–562). Springer International Publishing. https://doi.org/10.1007/978-3-319-73450-7_52
Jaramillo-Alcázar, A., Luján-Mora, S., & Salvador-Ullauri, L. (2017). Accessibility assessment of mobile serious games for people with cognitive impairments. 2017 International conference on information systems and computer science. https://doi.org/10.1109/INCISCOS.2017.12
Knezovic, A. (2020, October 9). Mobile gaming statistics: 85+ statistics for 2020 | udonis. Udonis. https://www.blog.udonis.co/mobile-marketing/mobile-games/mobile-gaming-statistics
Lee, S., Oh, H., Shi, C.-K., & Doh, Y. (2021). Mobile game design guide to improve gaming experience for the middle-aged and older adult population: User-centered design approach. JMIR Serious Games, 9(2), e24449. https://doi.org/10.2196/24449
Newzoo. (2021, July 1). Newzoo global games market report 2021 | free version | newzoo [PDF]. https://newzoo.com/insights/trend-reports/newzoo-global-games-market-report-2021-free-version/?utm_campaign=GGMR 2021
Nielsen, J. (2020, November 15). 10 usability heuristics for user interface design. Nielsen Norman Group. https://www.nngroup.com/articles/ten-usability-heuristics/
PopCap Games & Chauteau, G. (2008, June 11). Popcap games research. GamesIndustry.biz. Retrieved October 10, 2020, from https://www.gamesindustry.biz/articles/popcap-games-research-publisher-s-latest-survey-says-that-casual-games-are-big-with-disabled-people
Rehman, U., Abbasi, A., Shah, M., Idrees, A., Ilahi, H., & Hlavacs, H. (2021). Analyzing and prioritizing usability issues in games. In Lecture notes in computer science (pp. 110–118). Springer International Publishing. https://doi.org/10.1007/978-3-030-77277-2_9
Rienzo, A., & Cubillos, C. (2020). Playability and player experience in digital games for elderly: A systematic literature review. Sensors, 20(14), 3958. https://doi.org/10.3390/s20143958
Salvador-Ullauri, L., Acosta-Vargas, P., Gonzalez, M., & Luján-Mora, S. (2020). Combined method for evaluating accessibility in serious games. Applied Sciences, 10(18), 6324. Retrieved December 17, 2020, from https://doi.org/10.3390/app10186324
World Wide Web Consortium. (2018, June 5). Web content accessibility guidelines (WCAG) 2.1. Web Accessibility Initiative (WAI). https://www.w3.org/TR/WCAG21/
World Wide Web Consortium. (2020). Web content accessibility guidelines (WCAG) 2.2. Web Accessibility Initiative (WAI). Retrieved July 27, 2021, from https://www.w3.org/TR/WCAG22/
World Wide Web Consortium. (2021, May 14). Mobile accessibility at W3C. Web Accessibility Initiative (WAI). Retrieved July 27, 2021, from https://www.w3.org/WAI/standards-guidelines/mobile/
Zhang, X., de Greef, L., Swearngin, A., White, S., Murray, K., Yu, L., Shan, Q., Nichols, J., Wu, J., Fleizach, C., Everitt, A., & Bigham, J. P. (2021, May 8–13). Screen recognition: creating accessibility metadata for mobile applications from pixels [Research Article]. 2021 CHI Conference on Human Factors in Computing Systems, Yokohama, Japan. https://doi.org/10.1145/3411764.3445186
Read more about:
Featured BlogsAbout the Author
You May Also Like