Synthaisthesia (Breakout 7)


Tuesday, April 06, 2021: 12:00pm - 1:30pm - - Demonstration

Ian McDermott, Smithsonian, USA

Using augmented reality for accessibility; a new way to hear visual art. Access Smithsonian gives a sneak peek of their unreleased software that will allow visitors who are blind to experience the visuals within museums using a new type of image file, the .sas file. The .sas file embeds audio within regions of an image, so a user can navigate the image using their hands and ears rather than their eyes. In this way, a painting’s composition becomes like a two-dimensional song, with instruments, sounds and descriptions fading in and out based on the user’s input. This allows users can have a self-guided, spatial understanding of works of art, and pinpoint key elements that might be lost in a general description. By being able to experience museum objects in a way parallel to the act of seeing, museums can open new discussions and fuel greater empathy between its visitors, regardless of ability. Join us for a live demonstration of the project’s open-source hardware and software, and a discussion of its future possibilities.

Bach-y-Rita, P. (1972). Brain mechanisms in sensory substitution. New York: Academic Press.

Cytowic, R. E., & Eagleman, D. (2011). Wednesday is indigo blue: Discovering the brain of synesthesia. Cambridge, MA: MIT Press.

Albers, J. (2006). Interaction of color (Revised and Expanded ed.). New Haven: Yale University Press.