Touch screen exploration of visual artwork for blind people

Abstract

This paper investigates how touchscreen exploration and verbal feedback can be used to support blind people to access visual artwork. We present two artwork exploration modalities. The first one, attribute-based exploration, extends prior work on touchscreen image accessibility, and provides fine-grained segmentation of artwork visual elements; when the user touches an element, the associated attributes are read. The second one, hierarchical exploration, is designed with domain experts and provides multi-level segmentation of the artwork; the user initially accesses a general description of the entire artwork and then explores a coarse segmentation of the visual elements with the corresponding high-level descriptions; once selected, coarse segments are subdivided into fine-grained ones, which the user can access for more detailed descriptions. The two exploration modalities, implemented as a mobile web app, were evaluated through a user study with 10 blind participants. Both modalities were appreciated by the participants. Attribute-based exploration is perceived to be easier to access. Instead, the hierarchical exploration was considered more understandable, useful, interesting and captivating, and the participants remembered more details about the artwork with this modality. Participants commented that the two modalities work well together and therefore both should be made available. © 2021 ACM.