AccessDroid: Detecting Screen Reader Accessibility Issues in Android Applications via Semantics Trees

# AccessDroid> AccessDroid is a lightweight dynamic analyzer for detecting screen reader accessibility issues in Android app pages. It runs on Android devices or emulators, generates accessibility reports, and visualizes semantics trees to help developers debug apps from the perspective of screen reader users.## Contents- [Features](#features)- [Environment Requirements](#environment-requirements)- [Compilation and Installation](#compilation-and-installation)- [Usages](#usages)- [Output Format](#output-format)- [A Quick Example](#a-quick-example)- [Trouble Shooting](#trouble-shooting)- [PoseCheck](#posecheck)- [Comparison Tools](#comparison-tools) - [Accessibility Scanner](#accessibility-scanner) - [LLM-based Approach](#llm-based-approach)## Features- Detects screen reader accessibility issues in runtime app pages.- Generates detailed reports, UI trees, and semantics trees of an app page.- Visualizes semantics trees on the device in real time.- Supports simulated touch interactions (tap, swipe) for automated testing.- Cross-platform support on Windows, macOS, and Linux.- Simple command workflow through the provided `ad` script.## Environment RequirementsAccessDroid is a lightweight tool designed for Android developers. If you already have a standard Android development environment (Android Studio, ADB, and an Android emulator), AccessDroid can be set up and run in a stable manner with minimal effort.In practice, installing Android Studio alone is sufficient, as it can automatically configure most required Android development components. Therefore, Docker is not required for running AccessDroid.AccessDroid has been tested on macOS and Windows.> **Note**: The estimated setup time below is for reference only and may vary depending on your network environment.1. **Android Studio** (For compilation and android debug bridge) [~**10** minutes] - Used to build the AccessDroid APK from source (a prebuilt APK is also provided in the ` /AccessDroid/AccessDroid.apk`). - Official download and installation instructions are available at [this](https://developer.android.com/studio).2. **Android Emulator** or Android device [~**10** minutes] - Android Studio offers built-in tools for downloading and managing emulator images. - Running AccessDroid on a physical Android device is also supported, but requires: - USB connection to the host machine - Developer Mode and USB debugging enabled on the device - AccessDroid supports Android 12 and higher versions. (An Android 12 image without Google Play is recommended for a clean system environment.)3. **ADB** (Android Debug Bridge) [~**2** minutes] - ADB is installed automatically as part of the Android SDK when using Android Studio. - Ensure that `adb` is available in your system PATH. (e.g. `export PATH="$PATH:$ANDROID_HOME/platform-tools"`) - Run `adb help` in the terminal to verify that ADB has been installed successfully.4. **A shell environment** is required to run the `ad` script. We provide two versions of the `ad` script: one written in `Zsh` syntax and the other in `PowerShell` syntax (`ad.ps1`). [~**2** minutes] - For macOS: [Zsh](https://github.com/ohmyzsh/ohmyzsh/wiki/Installing-ZSH) is the default shell. - For Linux: `Zsh` can be installed easily via package managers. (e.g. `apt install zsh; chsh -s $(which zsh)`) - For Windows: `Powershell` is the default shell and fully supported.5. Hardware Requirements - AccessDroid is designed with a lightweight architecture and has minimal resource consumption. - The only practical requirement is that the host machine can run an Android Emulator smoothly. - AccessDroid itself has no special hardware dependencies and is suitable for a wide range of development and testing environments.## Compilation and Installation> This section describes how to build and install AccessDroid. You may either compile the project from source or use the prebuilt APK provided in the artifacts.1. Download the source code (` /AccessDroid`).2. Build the APK (Optional) - Open the project in Android Studio and build the project. - A **precompiled APK** is also available at ` /AccessDroid/AccessDroid.apk`. This APK can be used directly without rebuilding the project.3. Install APKs on the emulator/device - Install AccessDroid.apk and the target application you want to analyze on the Android emulator or physical device. - Both APKs can be installed by dragging and dropping them into the emulator window. - For convenience, we provide 50 sample target applications in ` /PoseCheck`4. Add tools to PATH - Ensure that both `adb` and the `ad` script are accessible from your terminal. - For macOS/Linux: `export PATH="$PATH:$ANDROID_HOME/platform-tools: /AccessDroid/scripts`. - For Windows: Find platform-tools folder in Android SDK and ` /AccessDroid/scripts` to the system environment variables. [Help](https://www.solveyourtech.com/how-to-set-environment-variable-in-windows-11-a-step-by-step-guide/) - The `ad` script does not require any modification before execution and can be run directly once it is available in PATH.## UsagesAll interactions with AccessDroid are performed via the `ad` script, which wraps ADB commands for convenience.| Command | Description ||----------------|------------------------------------------------------------------------------------------------------------------------------------------------|| ad enable | Grants Accessibility permission for AccessDroid. When successful, the main interface shows service on. || ad disable | Revokes Accessibility permission and stops AccessDroid service. || ad scan | (For Jetpack Compose Apps) Analyzes the currently displayed page on the device. Red bounding boxes highlight screen reader focusable elements. || ad vscan | (For Other Apps) Analyzes the currently displayed page on the device. Red bounding boxes highlight screen reader focusable elements. || ad pull | Retrieves the latest accessibility report, including the UI tree and semantics tree, to the current directory. || ad tap x y | Simulates a tap at screen coordinates (x, y). || ad swipe_right | Simulates a swipe gesture to the right. || ad clear | Removes red bounding boxes from the screen. || ad debug | Outputs the UI tree and semantics tree to the file. || ad screen | Save the current screenshot |**Notes**- Coordinate-based commands such as ad tap depend on the current screen resolution and orientation.- Output files generated by `ad pull` and `ad debug` are saved to the current working directory.## Output Format> AccessDroid generates both visual annotations on the device screen and text-based analysis results that can be inspected locally.**Visual Output**- After executing `ad scan`, AccessDroid highlights detected semantics nodes on the device screen using red bounding boxes.- For each analysis run, a corresponding annotated screenshot is saved at: ` /Results/ / .png`.- Red bounding boxes are visual overlays only; they do not interfere with app interaction.**Exported Files**- The `ad pull` command exports the following files to the local machine: - Accessibility report (` /Results/ / _issues.txt`). - UI Tree (` /Results/ / .txt`). - Semantics Tree (` /Results/ / _semanticsTree.txt`).All exported files are plain text and can be opened with any standard text editor.**Report Example**Below is an example snippet from the accessibility report:```txt[OnClick Label Omission]: (8)View(View)[Semantics Separation]: (15)TextView(TextView);AccessDroid (14)View(Button)```**Report Interpretation**- The content inside square brackets indicates the type of accessibility issue detected.- Each issue is followed by one or more affected node entries, which include: - A node ID (e.g., 8, 15, 14). - The node type (e.g., View, TextView, Button). - Optional semantic information such as text content- Node IDs are internally assigned by AccessDroid and are not related to Android system IDs.- These node IDs are consistent across all outputs, including: reports, UI tree, semantics tree, and red bounding boxes displayed on the screen. This consistency allows users to easily correlate reported issues with their corresponding visual elements on the device.## A Quick Example> The following example demonstrates a typical end-to-end workflow for analyzing an Android application with AccessDroid.> [A Video Demonstration](https://www.youtube.com/watch?v=tDU0Cs4yfLY)1. Install AccessDroid.apk on the emulator, along with the target application to be analyzed.2. Grant Accessibility permissions: ```shell ad enable ```3. Launch the target application and navigate to the specific screen you want to analyze.4. Run accessibility analysis. - For Jetpack Compose–based applications: ```shell ad scan ``` - For applications built with other UI frameworks ```shell ad vscan ```5. (Optional) Dump UI trees and semantics trees ```shell ad debug ```