Accessible Visual Artworks for Blind and Visually Impaired People: Comparing a Multimodal Approach with Tactile Graphics
Motivation and Objective
- A formative study performed with the help of eight blind and visually impaired participants, art museum and gallery staff, and two artists to understand the different needs of these stakeholders and the current state of the accessibility tools available to experience visual artworks.
- A low-cost alternative implementation of an interactive multimodal guide that enables blind and visually impaired people without previous training to independently access and experience visual artworks.
- In collaboration with an accessible art gallery and a school for blind and visually impaired people, we performed two art exhibitions using the proposed guide. Within those exhibitions, we performed a survey with eighteen blind and visually impaired participants to compare the proposed interactive guide and a tactile graphics alternative.
2. Related Work
2.1. Tactile Graphics
2.2. Interactive Tactile Graphics and 3D Models
2.3. Interactive Multimodal Guides for Blind and Visually Impaired People
3. Materials and Methods
3.1. Formative Study
3.1.1. Accessible Visual Artworks for Blind and Visually Impaired People
3.1.2. Accessible Visual Artwork at Art Museums and Galleries
3.1.3. Accessible Visual Artwork and Artists
3.1.4. Design Requirements
- Simple to learn and use. The guide should offer a low entry barrier to the user. It should avoid the need for Braille literacy for operation and exploration to improve the access for blind and visually impaired people without or limited Braille literacy. It should avoid, as much as possible, the need for training or previous experience for its operation. For example, using a limited set of intuitive and well-known interaction gestures and interfaces to avoid cognitive load.
- Self-contained. The guide should avoid requiring blind and visually impaired people to carry external devices or install software on their own. Blind and visually impaired visitors often already carry several items such as a personal bag, white cane, leaflets, and audio guides. External devices add to their carrying load, add the need to check-in and out the device, as well as to learn the device operation and interface.
- Facilitate access to information. Exploring the artworks by touch is essential to understand the spatial arrangement of the artwork. The design of the model should be simple and abstract enough for easy comprehension, while avoiding oversimplification. Audio descriptions should be detailed but not long. Users should be able to skip them if desired.
- Promote active engagement. The IMG should promote active user engagement by facilitating exploration rather than just providing information. As much as possible, the guide should encourage critical thinking, reflection, and emotional responses.
- Unobtrusive and versatile. The guide should avoid being obtrusive to the original artwork within an art museum and gallery environment such that it can be colocated and avoid user isolation. The IMG should be able to support different artwork styles, sizes, and shapes.
3.2. Interactive Multimodal Guide (IMG)
3.2.1. 2.5D Relief Model
- A high resolution picture of the visual artwork is taken or obtained. Figure 1a.
- The height-map image is modified using a digital image software to correct, modify, abstract, simplify or accentuate features and objects in the painting to improve their legibility and recognition by blind and visually impaired people.
- A three-dimensional model is generated from the original picture and the height-map image using the ’Embossing Tool’ in the ZW3D 3D drawing software. Figure 1c.
3.2.2. Control Board
3.2.3. External Hardware
3.2.4. Interaction Design
3.2.5. Information Hierarchy
- General information: Refers to the general information of the artwork such as name, author, short visual description, and any information that is not already present in the artwork or related to information that can be accessed in a single point of interest.
- Localized information is information related to a specific point of interest in the artwork such as the object name, detailed description, color, meaning, and their relationship with neighboring points of interest and their sound, among others.
3.3.1. Accessible Exhibitions Using IMG and Tactile Graphics
3.3.3. Materials and Apparatus
4. Results and Discussion
4.1. General Impressions
5. Conclusions and Future Work
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
|IMG||Interactive Multimodal Guide|
|STEM||Science, Technology, Engineering and Mathematics|
|Author - Name||Input||Output||Focus|
|Parkes  NOMAD||- Touch (Surface)||- Tactile overlay - Verbal descriptions||- Mathematics, Geometry, Geography, and Biology Education - Orientation & Mobility|
|Landau et al.  The Talking Tablet||- Touch (Surface)||- Tactile overlay - Verbal descriptions||- Mathematics, Geometry, Geography, and Biology Education - Orientation & Mobility|
|Gardner et al.  IVEO||- Touch (Surface)||- Tactile overlay - Verbal descriptions||- Education & Scientific Diagrams|
|Taylor et al. ||- Touch (Touchscreen)||- Tactile overlay - Verbal descriptions||- Orientation & Mobility|
|Gotzelmann et al.  LucentMaps||- Touch (Touchscreen) - Voice||- Tactile overlay - Visual agumentation||- Orientation & Mobility|
|Brule et al.  MapSense||- Touch (Touchscreen) - Tokens (Capacitive)||- Tactile overlay - Smell and taste infused tangible tokens - Verbal descriptions||- Geography Education - Map Exploration - Orientation & Mobility|
|Shen et al.  CamIO||- Touch (Mounted camera)||- Tactile graph - Tactile 3D Map - Tactile Object Verbal descriptions||- Access to 3D objects - Map Exploration - Access to appliances - Access to documents|
|Baker et al.  Tactile Graphics with a Voice||- Touch (Mobile Camera)||- Tactile graph - Verbal descriptions||- STEM Education|
|Baker et al.  Tactile Graphics with a Voice||- Touch (Wearable Camera) - Voice||- Tactile graph - Verbal descriptions||- STEM Education|
|Fusco et al.  The Tactile Graphics Helper||- Touch (Mobile Camera) - Voice||- Tactile graph - Verbal descriptions||- STEM Education - Map Exploration|
|Holloway et al. ||- Touch (Embedded capacitive sensors)||- Tactile 3D Map - Verbal descriptions||- Orientation & Mobility|
|Vaz et al. ||- Touch (Embedded capacitive sensors)||- Tactile Objects - Verbal descriptions - Visual augmentation||- Museum Object Exploration|
|Anagnostakis et al. ||- Touch (PIR and touch sensors)||- Tactile Objects - Verbal descriptions||- Museum Object Exploration|
|Leporini et al. ||- Touch (Physical buttons)||- Tactile 3D Map & Model - Verbal descriptions||- Archeological site exploration - Artwork exploration|
|Reichinger et al. [44,45,46]||- Touch (Camera) - Hand gestures (Camera)||- Tactile 3D Artwork Model - Verbal descriptions||- Artwork exploration|
|Landau et al.  The Talking Tactile Pen||- Touch (Pen device)||- Tactile graph - Verbal descriptions||- STEM Education - Map Exploration - Games|
|D’Agnano et al.  Tooteko||- Touch (Ring NFC reader)||- Tactile 3D model - Verbal descriptions||- Archelological site exploration - Artwork exploration|
- Maintaining Accessibility in Museums. Available online: https://www.ada.gov/business/museum_access.htm (accessed on 9 November 2020).
- United Nations Department of Economic and Social Affairs (UN-DESA). Convention on the Rights of Persons with Disabilities (CRPD). Available online: https://www.un.org/development/desa/disabilities/convention-on-the-rights-of-persons-with-disabilities.html (accessed on 9 November 2020).
- The Guggenheim Museums and Foundation. Mind’s Eye Programs. Available online: https://www.guggenheim.org/event/event_series/minds-eye (accessed on 9 November 2020).
- The Metropolitan Museum of Art. For Visitors Who Are Blind or Partially Sighted. Available online: https://www.metmuseum.org/events/programs/access/visitors-who-are-blind-or-partially-sighted. (accessed on 9 November 2020).
- Museum of Moden Art. Accessibility. Available online: https://www.moma.org/visit/accessibility/index#individuals-who-are-blind-or-have-low-vision. (accessed on 9 November 2020).
- Hoyt, B. Emphasizing Observation in a Gallery Program for Blind and Low-Vision Visitors: Art Beyond Sight at the Museum of Fine Arts, Houston. Available online: https://dsq-sds.org/article/view/3737/3278 (accessed on 9 November 2020).
- Krantz, G. Leveling the Participatory Field: the Mind’s Eye Program at the Guggenheim Museum. Available online: https://dsq-sds.org/article/view/3738/3277 (accessed on 9 November 2020).
- Mesquita, S.; Carneiro, M.J. Accessibility of European museums to visitors with visual impairments. Disabil. Soc. 2016, 31, 373–388. [Google Scholar] [CrossRef]
- Phillips, A.; Beesley, L. Braille Profiling Project. 2011. Available online: https://www.rnib.org.uk/sites/default/files/braille_profiling.doc (accessed on 9 November 2020).
- The Braille Literacy Crisis in America. Available online: https://www.nfb.org//images/nfb/documents/pdf/braille_literacy_report_web.pdf. (accessed on 9 November 2020).
- Museo del Prado. Touching the Prado. Available online: https://www.museodelprado.es/en/whats-on/exhibition/touching-the-prado/0d94a9bf-07d7-491a-866a-37976169f929 (accessed on 9 November 2020).
- The Andy Warhol Museum. Available online: https://www.warhol.org/accessibility-accommodations/ (accessed on 9 November 2020).
- Candlin, F. The dubious inheritance of touch: Art history and museum access. J. Vis. Culture 2006, 5, 137–154. [Google Scholar] [CrossRef]
- Holloway, L.; Marriott, K.; Butler, M.; Borning, A. Making Sense of Art: Access for Gallery Visitors with Vision Impairments. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems, Glasgow, UK, 4–9 May 2019; pp. 1–12. [Google Scholar]
- Schiff, W.; Foulke, E. Tactual Perception: A Sourcebook; Cambridge University Press: Cambridge, UK, 1982. [Google Scholar]
- BANA. Guidelines and Standards for Tactile Graphics. Available online: http://www.brailleauthority.org/tg/web-manual/index.html (accessed on 9 November 2020).
- Sinha, P.; Kalia, A.A. Tactile picture recognition: Errors are in shape acquisition or object matching? See. Perc. 2012, 25, 287–302. [Google Scholar] [CrossRef] [PubMed][Green Version]
- Brock, A.M.; Truillet, P.; Oriola, B.; Picard, D.; Jouffrais, C. Interactivity improves usability of geographic maps for visually impaired people. Hum. Comput. Interact. 2015, 30, 156–194. [Google Scholar] [CrossRef]
- Landau, S.; Wells, L. Merging tactile sensory input and audio data by means of the Talking Tactile Tablet. In Proceedings of the EuroHaptics, Dublin, Ireland, 6–9 July 2003; pp. 414–418. [Google Scholar]
- Tatham, A.F. The design of tactile maps: Theoretical and practical considerations. In Mapping the Nations: Proceedings, Volume 1; Rybaczuk, K., Blakemore, M.J., Eds.; International Cartographic Association: Bern, Switzerland, 1991. [Google Scholar]
- Parkes, D. “NOMAD”: An audio-tactile tool for the acquisition, use and management of spatially distributed information by partially sighted and blind persons. In Proceedings of the Second International Conference on Maps and Graphics for Visually Disabled People, Nottingham, UK, 20 April 1988; pp. 24–29. [Google Scholar]
- Landau, S.; Holborow, R.; Jane, E. Use of the talking tactile tablet for delivery of standardized tests. In Proceedings of the CSUN 2004, Los Angeles, CA, USA, 15–20 March 2004. [Google Scholar]
- Gardner, J.A.; Bulatov, V. Scientific Diagrams Made Easy with IVEOTM. In Computers Helping People with Special Needs; Miesenberger, K., Klaus, J., Zagler, W.L., Karshmer, A.I., Eds.; Springer: Berlin, Germany, 2006; pp. 1243–1250. [Google Scholar]
- Taylor, B.; Dey, A.; Siewiorek, D.; Smailagic, A. Customizable 3D printed tactile maps as interactive overlays. In Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility, Reno, NV, USA, 23–26 October 2016; pp. 71–79. [Google Scholar]
- Götzelmann, T. Visually Augmented Audio-Tactile Graphics for Visually Impaired People. ACM Trans. Access. Comput. 2018, 11, 1–31. [Google Scholar] [CrossRef]
- Brule, E.; Bailly, G.; Brock, A.; Valentin, F.; Denis, G.; Jouffrais, C. MapSense: multi-sensory interactive maps for children living with visual impairments. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, San Jose, CA, USA, 7–12 May 2016; pp. 445–457. [Google Scholar]
- Shen, H.; Edwards, O.; Miele, J.; Coughlan, J.M. CamIO: A 3D Computer Vision System Enabling Audio/Haptic Interaction with Physical Objects by Blind Users; Association for Computing Machinery: New York, NY, USA, 2013. [Google Scholar] [CrossRef]
- Baker, C.M.; Milne, L.R.; Scofield, J.; Bennett, C.L.; Ladner, R.E. Tactile graphics with a voice: using QR codes to access text in tactile graphics. In Proceedings of the 16th international ACM SIGACCESS conference on Computers & Accessibility, Rochester, NY, USA, 20–22 October 2014; pp. 75–82. [Google Scholar]
- Baker, C.M.; Milne, L.R.; Drapeau, R.; Scofield, J.; Bennett, C.L.; Ladner, R.E. Tactile graphics with a voice. ACM Trans. Access. Comput. (TACCESS) 2016, 8, 1–22. [Google Scholar] [CrossRef]
- Fusco, G.; Morash, V.S. The tactile graphics helper: Providing audio clarification for tactile graphics using machine vision. In Proceedings of the 17th International ACM SIGACCESS Conference on Computers & Accessibility, Lisbon, Portugal, 26–28 October 2015; pp. 97–106. [Google Scholar]
- Holloway, L.; Marriott, K.; Butler, M. Accessible Maps for the Blind: Comparing 3D Printed Models with Tactile Graphics. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, Montreal, QC, Canada, 21–26 April 2018; Association for Computing Machinery: New York, NY, USA, 2018; pp. 1–13. [Google Scholar] [CrossRef]
- Holloway, L.; Marriott, K.; Butler, M.; Reinders, S. 3D Printed Maps and Icons for Inclusion: Testing in the Wild by People who are Blind or have Low Vision. In Proceedings of the 21st International ACM SIGACCESS Conference on Computers and Accessibility, Pittsburgh, PA, USA, 28–30 October 2019; pp. 183–195. [Google Scholar]
- Gual, J.; Puyuelo, M.; Lloveras, J. Three-dimensional tactile symbols produced by 3D Printing: Improving the process of memorizing a tactile map key. Br. J. Vis. Impair. 2014, 32, 263–278. [Google Scholar] [CrossRef]
- Gual, J.; Puyuelo, M.; Lloveras, J. The effect of volumetric (3D) tactile symbols within inclusive tactile maps. Appl. Ergon. 2015, 48, 1–10. [Google Scholar] [CrossRef] [PubMed]
- Touch Graphics. TTPen Stem Binder. 2015. Available online: http://touchgraphics.com/portfolio/ttpen-stem-binder/ (accessed on 9 November 2020).
- D’Agnano, F.; Balletti, C.; Guerra, F.; Vernier, P. Tooteko: A case study of augmented reality for an accessible cultural heritage. Digitization, 3D printing and sensors for an audio-tactile experience. ISPRS Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2020, 40, 207–213. [Google Scholar] [CrossRef][Green Version]
- Axel, E.S.; Levent, N.S. Art Beyond Sight: A Resource Guide to Art, Creativity, and Visual Impairment; American Foundation for the Blind: Arlington, VA, USA, 2003. [Google Scholar]
- Cho, J.D.; Quero, L.C.; Bartolomé, J.I.; Lee, D.W.; Oh, U.; Lee, I. Tactile colour pictogram to improve artwork appreciation of people with visual impairments. Color Res. Appl. 2021, 46, 103–116. [Google Scholar] [CrossRef]
- Volpe, Y.; Furferi, R.; Governi, L.; Tennirelli, G. Computer-based methodologies for semi-automatic 3D model generation from paintings. Int. J. Comp. Aided Eng. Technol. 2014, 6, 88–112. [Google Scholar] [CrossRef]
- Hinton, R. Use of tactile pictures to communicate the work of visual artists to blind people. J. Vis. Impair. Blind. 1991, 85, 174–175. [Google Scholar] [CrossRef]
- Anagnostakis, G.; Antoniou, M.; Kardamitsi, E.; Sachinidis, T.; Koutsabasis, P.; Stavrakis, M.; Vosinakis, S.; Zissis, D. Accessible museum collections for the visually impaired: combining tactile exploration, audio descriptions and mobile gestures. In Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct, Florence, Italy, 6–9 September 2016; pp. 1021–1025. [Google Scholar]
- Vaz, R.; Fernandes, P.O.; Cecília, A. Designing an Interactive Exhibitor for Assisting Blind and Visually Impaired Visitors in Tactile Exploration of Original Museum Pieces. Proc. Comp. Sci. 2018, 138, 561–570. [Google Scholar] [CrossRef]
- Leporini, B.; Rossetti, V.; Furfari, F.; Pelagatti, S.; Quarta, A. Design Guidelines for an Interactive 3D Model as a Supporting Tool for Exploring a Cultural Site by Visually Impaired and Sighted People. ACM Trans. Access. Comp. (TACCESS) 2020, 13, 1–39. [Google Scholar] [CrossRef]
- Reichinger, A.; Fuhrmann, A.; Maierhofer, S.; Purgathofer, W.A.; Fuhrmann, A.; Maierhofer, S.; Purgathofer, W. A Concept for Re-useable Interactive Tactile Reliefs. In Computers Helping People with Special Needs; Miesenberger, K., Bühler, C., Penaz, P., Eds.; Springer: Cham, Switzerland, 2016; pp. 108–115. [Google Scholar]
- Reichinger, A.; Fuhrmann, A.; Maierhofer, S.; Purgathofer, W. Gesture-Based Interactive Audio Guide on Tactile Reliefs. In Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility, Reno, NV, USA, 24–25 October 2016; pp. 91–100. [Google Scholar] [CrossRef]
- Reichinger, A.; Carrizosa, H.G.; Wood, J.; Schröder, S.; Löw, C.; Luidolt, L.R.; Schimkowitsch, M.; Fuhrmann, A.; Maierhofer, S.; Purgathofer, W. Pictures in Your Mind: Using Interactive Gesture-Controlled Reliefs to Explore Art. ACM Trans. Access. Comput. 2018, 11, 1–39. [Google Scholar] [CrossRef][Green Version]
- Elkhuizen, W.S.; Callewaert, T.W.; Leonhardt, E.; Vandivere, A.; Song, Y.; Pont, S.C.; Geraedts, J.M.; Dik, J. Comparison of three 3D scanning techniques for paintings, as applied to Vermeer’s ‘Girl with a Pearl Earring’. Herit. Sci. 2019, 7, 89. [Google Scholar] [CrossRef]
- Zhang, R.; Tsai, P.S.; Cryer, J.E.; Shah, M. Shape-from-shading: A survey. IEEE Trans. Pattern Anal. Mach. Intell. 1999, 21, 690–706. [Google Scholar] [CrossRef][Green Version]
- Prados, E.; Faugeras, O. Shape From Shading. In Handbook of Mathematical Models in Computer Vision; Paragios, N., Chen, Y., Faugeras, O., Eds.; Springer: Boston, MA, USA, 2006; pp. 375–388. [Google Scholar] [CrossRef]
- Furferi, R.; Governi, L.; Volpe, Y.; Puggelli, L.; Vanni, N.; Carfagni, M. From 2D to 2.5D i.e. from painting to tactile model. Graph. Model. 2014, 76, 706–723. [Google Scholar] [CrossRef]
|Halloway et al. ||- Sensing technology: Capacitive sensor board connected to discrete copper interaction points placed on the surface of the model.|
- Input: Double tap and long tap gestures on the surface.
- Tactile presentation: Tactile 3D map model.
- Output: Audio Descriptions.
- Objective: Improve Mobility and Orientation.
|Reichinger et al. [44,45,46]||- Sensing technology: Color and depth mounted camera.|
- Input: Tap gestures on the surface and hand gestures above the surface.
- Tactile presentation: Tactile bas-relif model.
- Output: Audio Descriptions.
- Objective: Improve visual artwork exploration.
|Cavazos et al. *||- Sensing technology: Capacitive sensor connected to conductive ink-based sensors embedded under the surface of the model.|
- Input: Double tap and triple tap gestures on the surface.
- Tactile presentation: Tactile bas-relief model.
- Output: Audio Descriptions, Sound effects, and Background music
- Objective: Improve visual artwork exploration.
|FP1||Female||24||University student||Total vision loss|
|FP2||Male||40||Worker||Near vision loss|
|FP3||Female||42||Worker||Total vision loss|
|FP4||Female||30||Worker||Profound vision loss|
|FP5||Male||27||University student||Near vision loss|
|FP6||Male||24||University student||Total vision loss|
|FP7||Female||23||University student||Total vision loss|
|FP8||Female||23||University student||Total vision loss|
|EP1||Female||16||High school student||Total vision loss|
|EP2||Female||16||High school student||Near vision loss|
|EP3||Female||19||High school student||Profound vision loss|
|EP4||Male||15||High school student||Total vision loss|
|EP5||Male||15||High school student||Total vision loss|
|EP6||Male||18||High school student||Total vision loss|
|EP7||Female||19||High school student||Profound vision loss|
|EP8||Female||16||High school student||Total vision loss|
|EP9||Male||17||High school student||Near vision loss|
|EP10||Male||18||High school student||Profound vision loss|
|EP11||Female||15||High school student||Total vision loss|
|EP12||Female||39||Worker||Total vision loss|
|EP13||Male||38||Worker||Total vision loss|
|EP14||Female||43||Worker||Total vision loss|
|EP15||Male||52||None||Near vision loss|
|EP16||Male||50||Worker||Near vision loss|
|EP17||Female||47||Housewife||Near vision loss|
|EP18||Female||45||Worker||Total vision loss|
|S1. I think that I would like to use this system frequently|
|S2. I found the system unnecessarily complex.|
|S3. I thought the system was easy to use|
|S4. I think that I would need the support of a technical person to be able to use this system.|
|S5. I found the various functions in this system were well integrated.|
|S6. I thought there was too much inconsistency in this system|
|S7. I would imagine that most people would learn to use this system very quickly|
|S8. I found the system very cumbersome to use.|
|S9. I felt very confident using the system|
|S10. I needed to learn a lot of things before I could get going with this system.|
|Interactive Multimodal Guide|
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Cavazos Quero, L.; Iranzo Bartolomé, J.; Cho, J. Accessible Visual Artworks for Blind and Visually Impaired People: Comparing a Multimodal Approach with Tactile Graphics. Electronics 2021, 10, 297. https://doi.org/10.3390/electronics10030297
Cavazos Quero L, Iranzo Bartolomé J, Cho J. Accessible Visual Artworks for Blind and Visually Impaired People: Comparing a Multimodal Approach with Tactile Graphics. Electronics. 2021; 10(3):297. https://doi.org/10.3390/electronics10030297Chicago/Turabian Style
Cavazos Quero, Luis, Jorge Iranzo Bartolomé, and Jundong Cho. 2021. "Accessible Visual Artworks for Blind and Visually Impaired People: Comparing a Multimodal Approach with Tactile Graphics" Electronics 10, no. 3: 297. https://doi.org/10.3390/electronics10030297