@article{
                      
                    
                    
                    }
                    }
                    
                  

Abstract

Accessibility reviews collected from app stores may contain valuable information for improving apps accessibility. Recent studies have presented insightful information on accessibility reviews, but they were based on small datasets and focused on general accessibility concerns. In this paper, we analyzed accessibility reviews that report issues affecting users with visual disabilities or conditions. Such reviews were identified based on selection criteria applied over 179,519,598 reviews of popular apps on the Google Play Store. Our results show that only 0,003% of user reviews mention visual disabilities or conditions; accessibility reviews are associated with 36 visual disabilities or eye conditions; many users do not give precise feedback and refer to their disability using generic terms; accessibility reviews can be grouped into general topics of concerns related to different types of disabilities; and positive reviews are generally associated with high scores and negative feedback with lower scores.

Research Questions

RQ1: How many accessibility reviews are associated with visual disabilities or eye conditions?

Our aim is to provide evidence of whether and how often users associate positive or negative feedback concerning the accessibility of the app with a visual disability or eye condition. Identifying whether a review is associated with a visual disability or eye condition, however, is not a trivial task given the lack of detailed information usually found in user reviews. Therefore, in our first effort to characterize such a type of user feedback, we focused on reviews in which disabilities and conditions are explicitly mentioned by the users themselves and not according to our own interpretation.

RQ2: What are the disabilities or eye conditions associated with the accessibility reviews?

Our purpose is to characterize the accessibility reviews based on the visual disabilities or eye conditions mentioned in the user reviews. This knowledge can provide evidence that users may present different eye conditions and, therefore, might have different concerns when it comes to the app interface. To answer this question, we manually analyzed each review identified in the previous research question to assign a label based on the visual disability or eye condition mentioned by users.

RQ3: What are the main topics addressed by the accessibility reviews associated with visual disabilities or eye conditions?

Our goal is to identify users’ concerns commonly expressed in the accessibility reviews. As manual content analysis is labor intensive, in this work, we resorted to topic modeling techniques that consist of automatically processing data (e.g., text) to summarize information and understand the main topics that stand out in well-defined clusters of data.

RQ4: What are the interface components and resources mentioned in the accessibility reviews associated with visual disabilities or eye conditions?

As many accessibility guidelines are oriented by interface components and resources, our aim is to identify which components and resources are mostly mentioned in accessibility reviews to provide some indication of which of them might be more relevant in this context.

RQ5: What are the scores of accessibility reviews associated with visual disabilities or eye conditions?

Our intent is to understand the associations between accessibility reviews and the scores assigned to the app. In particular, we want to know whether positive feedback leads to high scores and negative feedback leads to low scores, which would emphasize the difference between this type of reviews from those found in previous studies in which users express more preferences concerning accessibility than real barriers.