ColorWatch: Color Perceptual Spatial Tactile Interface for People with Visual Impairments
Abstract
:1. Introduction
1.1. Tactile Representation of Colors
1.2. Timepiece Watches for PVI
1.3. Review of the Color Systems
1.4. Proposed System
2. Methodology
2.1. Concept Design
2.2. Proposed Color Selection and Tactile Representation Scheme
2.3. Materials
3. Experimental Investigation
3.1. Experiment Design
3.2. Identification Tests
3.3. Workload Assessment
3.4. Usability Test
- I1
- I think that I would like to use this service frequently.
- I2
- I found the service unnecessarily complex.
- I3
- I think the service was easy to use.
- I4
- I think that I would need the support of a technical person to be able to use this service.
- I5
- I found that the various functions in this service were well integrated.
- I6
- I thought there was too much inconsistency in this service.
- I7
- I would imagine that most people would learn to use this service very quickly.
- I8
- I found the service very cumbersome to use.
- I9
- I felt very confident using the service.
- I10
- I needed to learn a lot of things before I could get going with this service.
3.5. Subjective Reasoning
- The square was easier than the round disk.
- Because it was a large prototype and could not be recognized all at once, you have to touch it several times.
- It wasn’t difficult, but I felt confused a couple of times. However, I think there will be no problem if you use it frequently.
- It was easy assuming that the disks were stationary.
- It is divided into a circle and a square shape, so there was no big inconvenience.
- It was easier to distinguish by touching than it was to memorize.
- The Red and Green portions overlapped with the corners of the square and were not easily recognized compared to other positions.
- The distinction between square and circle is good.
- It seems that it was a little difficult to recognize the raised dot.
- It would be helpful if the spectrum of recognizable colors widens.
- I think it will be easier to distinguish if the outer disk is also not round but has edges or angles like the square disk.
- Product miniaturization.
- If voice is added, it will be helpful for recognition.
- Vibration and temperature might also be used.
- I’m not sure if there’s a product like this, but, if there isn’t, it might help.
- It seems to be helpful for the visually impaired in that it can select clothes suitable for TPO (Time Place Occasion) and reflect preferences for fashion.
- It can be needed for scenarios like if someone complimented your shirt, and you associate this emotional response with the shirt color.
- It seems to be helpful because even the visually impaired may have to recognize colors.
- The advantage that people who do not know Braille can easily recognize the color.
- I think it would be good to make it easier to recognize if it is less affected by ambient light.
- I think it will help. Because there is a lot you can find out from color.
- I think it will be helpful enough. You can quickly recognize colors.
- It seems to be helpful in color recognition. It was placed in a circle like a color scheme to make it relevant.
4. Discussion
- The design for ColorWatch is presented which can aide PVIs in color perception, in addition to time recognition.
- The angular tactile pattern for color is proposed with associated angular positions of tactile dot to a range of achromatic colors and basic colors with hue, value, and chroma indication.
- The combination of colors from Goethe’s color triangle and Munsell color system has been integrated with the analog watch design interface. This interface is capable of instantly presenting automatic spatial tactile patterns for any detected reference color from the artworks as well as from real-life objects. This eliminates the need for static color translations of artwork onto tactile relief. The spatial color identification for arbitrary objects outranks the static interpretations for the artworks of existing TCPs.
- The prototype has been developed, and the tests have been performed to investigate effectiveness in terms of accurate identification of color, workload requirements for pattern learning, and usability for color detection.
- The color wheel depicts essentially the visible spectrum of colors enclosed by a circle, and is a useful tool for describing what happens when you mix colors, complementary color relationships, and adjacent colors. Traditional TCP requires embossed surface patterns to represent colors for each artwork. Our proposed system eliminates such embossed surface patterns, and can be used as a reconfigurable platform, providing better mobility, dissemination, and flexibility.
5. Conclusions
Author Contributions
Funding
Institutional Review Board Statement
Informed Consent Statement
Data Availability Statement
Acknowledgments
Conflicts of Interest
References
- Striem-Amit, E.; Wang, X.; Bi, Y.; Caramazza, A. Neural representation of visual concepts in people born blind. Nature communications 2018, 9, 1–12. [Google Scholar] [CrossRef] [PubMed]
- Bedny, M.; Caramazza, A. Perception, action, and word meanings in the human brain: The case from action verbs. Ann. N. Y. Acad. Sci. 2011, 1224, 81–95. [Google Scholar] [CrossRef] [PubMed]
- Cytowic, R.E. Synesthesia: A Union of the Senses; MIT Press: Cambridge, MA, USA, 2002. [Google Scholar]
- Shin, J.; Cho, J.; Lee, S. Please Touch Color: Tactile-Color Texture Design for The Visually Impaired. In Proceedings of the Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems, New York, NY, USA, 25–30 April 2020; pp. 1–7. [Google Scholar]
- Cho, J.D. A Study of Multi-Sensory Experience and Color Recognition in Visual Arts Appreciation of People with Visual Impairment. Electronics 2021, 10, 470. [Google Scholar] [CrossRef]
- Cho, J.D.; Jeong, J.; Kim, J.H.; Lee, H. Sound Coding Color to Improve Artwork Appreciation by People with Visual Impairments. Electronics 2020, 9, 1981. [Google Scholar] [CrossRef]
- Gilbert, A.N.; Martin, R.; Kemp, S.E. Cross-modal correspondence between vision and olfaction: The color of smells. Am. J. Psychol. 1996, 109, 335–351. [Google Scholar] [CrossRef] [PubMed]
- Iranzo Bartolomé, J.; Cho, J.D.; Cavazos Quero, L.; Jo, S.; Cho, G. Thermal Interaction for Improving Tactile Artwork Depth and Color-Depth Appreciation for Visually Impaired People. Electronics 2020, 9, 1939. [Google Scholar] [CrossRef]
- Cavadini, R.; Cerina, L.; Bracco, M.I.; Santambrogio, M.D. A wearable device for blind people to restore color perception. In Proceedings of the 2017 IEEE 3rd International Forum on Research and Technologies for Society and Industry, Modena, Italy, 11–13 September 2017; pp. 1–4. [Google Scholar]
- Sathian, K.; Stilla, R. Cross-modal plasticity of tactile perception in blindness. Restor. Neurol. Neurosci. 2010, 28, 271–281. [Google Scholar] [CrossRef] [PubMed]
- Lawrence, M.A.; Kitada, R.; Klatzky, R.L.; Lederman, S.J. Haptic roughness perception of linear gratings via bare finger or rigid probe. Perception 2007, 36, 547–557. [Google Scholar] [CrossRef] [PubMed]
- Ramsamy-Iranah, S.; Rosunee, S.; Kistamah, N. Application of Assistive Tactile Symbols in a ‘Tactile’ Book on colour and Shapes for Children with Visual Impairments. Int. J. Arts Sci. 2017, 10, 575–590. [Google Scholar]
- Bindi (Decoration). Available online: https://en.wikipedia.org/wiki/Bindi_(decoration) (accessed on 16 February 2021).
- Grant, A.C.; Thiagarajah, M.C.; Sathian, K. Tactile perception in blind Braille readers: A psychophysical study of acuity and hyperacuity using gratings and dot patterns. Percept. Psychophys. 2000, 62, 301–312. [Google Scholar] [CrossRef] [PubMed] [Green Version]
- Kitada, R.; Kito, T.; Saito, D.N.; Kochiyama, T.; Matsumura, M.; Sadato, N.; Lederman, S.J. Multisensory activation of the intraparietal area when classifying grating orientation: A functional magnetic resonance imaging study. J. Neurosci. 2006, 26, 7491–7501. [Google Scholar] [CrossRef] [PubMed]
- Cho, J.D.; Quero, L.C.; Bartolomé, J.I.; Lee, D.W.; Oh, U.; Lee, I. Tactile colour pictogram to improve artwork appreciation of people with visual impairments. Color Res. Appl. 2021, 46, 103–116. [Google Scholar] [CrossRef]
- Taras, C.; Ertl, T. Interaction with Colored Graphical Representations on Braille Devices. In Proceedings of the International Conference on Universal Access in Human-Computer Interaction, San Diego, CA, USA, 19–24 July 2009; pp. 164–173. [Google Scholar]
- Axel, E.; Levent, N. Art beyond sight: Guidelines for making tactile diagrams & standard tactile patterns, lines, and icons. In Art Education for the Blind; AFB Press: Arlington, VA, USA, 2003. [Google Scholar]
- Hetherington, K. Museums and the visually impaired: The spatial politics of access. Sociol. Rev. 2000, 48, 444–463. [Google Scholar] [CrossRef]
- Vaz, R.; Freitas, D.; Coelho, A. Blind and Visually Impaired Visitors’ Experiences in Museums: Increasing Accessibility through Assistive Technologies. Int. J. Incl. Mus. 2020, 13, 57–80. [Google Scholar] [CrossRef]
- Adnan, M.E.; Dastagir, N.M.; Jabin, J.; Chowdhury, A.M.; Islam, M.R. A cost effective electronic braille for visually impaired individuals. In Proceedings of the 2017 IEEE Region 10 Humanitarian Technology Conference (R10-HTC), New York, NY, USA, 21–23 December 2017; pp. 175–178. [Google Scholar]
- Tyler, K. Active Braille Timepiece & Related Methods. U.S. Patent App. 13/180,351, 17 January 2013. [Google Scholar]
- DOT Watch, a Braille Wearable Smartwatch. Available online: https://www.dotincorp.com/ (accessed on 20 January 2021).
- Anderson, N.L.; Puccinelli, J.P.; Singh, C.; Jamar, K.S.J.; Milne, T.A. Braille Watch. U.S. Patent 8,483,018, 9 July 2013. [Google Scholar]
- Twyman, M.; Mullenbach, J.; Shultz, C.; Colgate, J.E.; Piper, A.M. Designing wearable haptic information displays for people with vision impairments. In Proceedings of the Ninth International Conference on Tangible, Embedded, and Embodied Interaction, Stanford, CA, USA, 15–19 January 2015; pp. 341–344. [Google Scholar]
- Velázquez, R. Wearable assistive devices for the blind. In Wearable and Autonomous Biomedical Devices and Systems for Smart Environment; Springer: Berlin/Heidelberg, Germany, 2010; pp. 331–349. [Google Scholar]
- Tyler, J.E.; Hardy, A.C. An analysis of the original Munsell color system. JOSA 1940, 30, 587–590. [Google Scholar] [CrossRef]
- Von Goethe, J.W. Goethe’s Theory of Colours; Murray, J., Ed.; William Clowes and Sons: London, UK, 1840. [Google Scholar]
- Albers, J. Interaction of Color; Yale University Press: New Haven, CT, USA, 2013. [Google Scholar]
- RYB Color Model. Available online: https://en.wikipedia.org/wiki/RYB_color_model (accessed on 20 January 2021).
- Munsell Color System: Explanation. Available online: https://en.wikipedia.org/wiki/Munsell_color_system (accessed on 20 January 2021).
- Lee, C.H.; Jabbar, M.S.; Cho, J.D. ColorWatch: Color-Tactile Interpretation to Improve Color Perception and Accessibility for People with Visual Impairments. In Proceedings of the IEEE 2020 International Conference on Information and Communication Technology Convergence (ICTC), Jeju Island, Korea, 21–23 October 2020; pp. 1632–1637. [Google Scholar]
- Mackay, W.E.; Fayard, A.L. HCI, natural science and design: A framework for triangulation across disciplines. In Proceedings of the 2nd Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques, Amsterdam, The Netherlands, 18–20 August 1997; pp. 223–234. [Google Scholar]
- Hart, S.G.; Staveland, L.E. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research. In Advances in Psychology; Elsevier: North Holland, The Netherlands, 1988; Volume 52, pp. 139–183. [Google Scholar]
- Hart, S.G. NASA-Task Load Index (NASA-TLX); 20 Years Later. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Los Angeles, CA, USA, 16–20 October 2006; Volume 50, pp. 904–908. [Google Scholar]
- Brooke, J. SUS: A “quick and dirty’usability. In Usability Evaluation in Industry; Taylor & Francis: London, UK, 1996; p. 189. [Google Scholar]
- 5 Ways to Interpret a SUS Score. Available online: https://measuringu.com/interpret-sus-score/ (accessed on 20 January 2021).
- Bangor, A.; Kortum, P.; Miller, J. Determining what individual SUS scores mean: Adding an adjective rating scale. J. Usability Stud. 2009, 4, 114–123. [Google Scholar]
- The Braille Literacy Crisis in America: Facing the Truth, Reversing the Trend, Empowering the Blind. In Braille Monitor 52.5; The National Federation of the Blind: Baltimore, MD, USA, 2009; Volume 52.
- Tactile Colours: Scheme for the Tactile Representation of Colours. Available online: http://cpwstonehouse.com/tactile-colours/ (accessed on 20 January 2021).
Angle (Square) | Color Tone | Color Mode |
---|---|---|
0 | Light | Monochromatic |
90 | Saturated | Monochromatic |
180 | Dark | Monochromatic |
270 | Achromatic | Achromatic |
Angle (Round) | Monochromatic Mode Color | Achromatic Mode Color |
---|---|---|
0 | Red | White |
60 | Orange | Gainsboro |
120 | Yellow | Light Gray |
180 | Green | Dark Gray |
240 | Blue | Dim Gray |
300 | Purple | Black |
Monochromatic – Achromatic Colors | Light | Saturated | Dark | Achromatic |
---|---|---|---|---|
Red–White | 15/15 | 15/15 | 15/15 | 15/15 |
Orange–Gainsboro | 15/15 | 15/15 | 15/15 | 15/15 |
Yellow–Light Gray | 14/15 | 15/15 | 14/15 | 15/15 |
Green–Dark Gray | 15/15 | 15/15 | 15/15 | 15/15 |
Blue–Dim Gray | 15/15 | 15/15 | 15/15 | 14/15 |
Purple–Black | 15/15 | 15/15 | 15/15 | 15/15 |
Correct Identifications (%) | 98.89 | 100 | 98.89 | 98.89 |
Average Correct Identifications | 357/360 (99.17%) |
Indicator | S1 | S2 | S3 | S4 | S5 | S6 | S7 | S8 | S9 | S10 | S11 | S12 | S13 | S14 | S15 |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
1. Mental Demand | 7 | 2 | 4 | 2 | 3 | 8 | 2 | 3 | 4 | 1 | 4 | 6 | 8 | 4 | 3 |
How much mental and perceptual activity was required? Was the task easy or demanding, simple, or complex? (memory ability of pattern, color, etc.) | |||||||||||||||
2. Physical Demand | 6 | 1 | 2 | 4 | 3 | 1 | 2 | 5 | 1 | 1 | 2 | 1 | 3 | 2 | 3 |
How much physical activity was required? Was the task easy or demanding, slack or strenuous? (tactile cognition) | |||||||||||||||
3. Temporal Demand | 7 | 3 | 1 | 2 | 3 | 2 | 1 | 3 | 2 | 1 | 3 | 8 | 7 | 3 | 2 |
How much time pressure did you feel due to the pace at which the tasks or task elements occurred? Was the pace slow or rapid? | |||||||||||||||
4. Overall Performance | 8 | 9 | 10 | 9 | 10 | 10 | 10 | 9 | 10 | 10 | 9 | 9 | 3 | 10 | 7 |
How successful were you in performing the task? How satisfied were you with your performance? | |||||||||||||||
5. Effort | 9 | 6 | 1 | 6 | 7 | 2 | 1 | 5 | 3 | 1 | 4 | 3 | 7 | 4 | 4 |
How hard did you have to work (mentally and physically) to accomplish your level of performance? | |||||||||||||||
6. Frustration Level | 6 | 3 | 1 | 1 | 1 | 5 | 1 | 2 | 1 | 1 | 3 | 1 | 1 | 1 | 3 |
How irritated, stressed, and annoyed versus content, relaxed, and complacent did you feel during the task? |
Measured Scores | SUS Indices | ||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|
I1 | I2 | I3 | I4 | I5 | I6 | I7 | I8 | I9 | I10 | ||
Subjects | S1 | 4 | 3 | 2 | 4 | 5 | 1 | 2 | 4 | 3 | 4 |
S2 | 4 | 2 | 4 | 3 | 3 | 2 | 4 | 3 | 4 | 2 | |
S3 | 5 | 2 | 5 | 2 | 4 | 1 | 5 | 1 | 5 | 2 | |
S4 | 4 | 2 | 5 | 2 | 2 | 2 | 5 | 3 | 4 | 1 | |
S5 | 4 | 1 | 4 | 5 | 5 | 1 | 4 | 2 | 5 | 2 | |
S6 | 2 | 2 | 4 | 3 | 3 | 2 | 5 | 4 | 3 | 2 | |
S7 | 5 | 2 | 5 | 2 | 3 | 1 | 4 | 4 | 4 | 2 | |
S8 | 4 | 2 | 4 | 3 | 4 | 2 | 4 | 3 | 4 | 2 | |
S9 | 3 | 2 | 5 | 1 | 4 | 1 | 3 | 2 | 5 | 1 | |
S10 | 5 | 1 | 5 | 5 | 4 | 1 | 5 | 2 | 5 | 1 | |
S11 | 4 | 2 | 4 | 3 | 3 | 2 | 4 | 4 | 4 | 1 | |
S12 | 5 | 1 | 5 | 3 | 5 | 1 | 5 | 1 | 5 | 3 | |
S13 | 4 | 2 | 3 | 1 | 5 | 2 | 5 | 1 | 4 | 4 | |
S14 | 5 | 2 | 4 | 3 | 4 | 1 | 4 | 1 | 5 | 1 | |
S15 | 5 | 1 | 3 | 3 | 3 | 1 | 5 | 1 | 3 | 3 |
Product | Output(s) | Mode | Features | Workload | Learnability |
---|---|---|---|---|---|
Tyler K. [22] | Timepiece | Numeric 4-dot condensed braille code | Time and date | Mental numeric calculation | The Braille literacy crisis |
Dot Inc. [23] | Timepiece | Numeric 6-dot braille code interface | Smartphone wireless connectivity | Mental numeric calculation | The Braille literacy crisis |
Anderson N. L. et al. [24] | Timepiece | Numeric 4-dot condensed braille code | Caller ID and pick/drop for smartphone calls | Mental numeric calculation | The Braille literacy crisis |
This work | Timepiece & Colorperception | Analoginterface | Traditional analog interface, Durable design, Color perception | Intuitive spatial recognition | Intuitive human perceptual design for time and color representation |
TCP | Basic Patterns (Concepts) | Number of Colors Presented | Medium |
---|---|---|---|
Taras et al. [17] | Dots (braille) | 23 (6 hues + 2 levels of lightness for each hues + 5 levels of achromatic) | Braille embossed surface pattern |
Ramsamy-Iranah et al. [12] | Polygons(children’s knowledge) | 14 (6 hues+ 5 other colors + 3 levels of achromatic) | Embossed surface pattern |
Stonehouse [40] | Geometric pattern and texture (traditional conventions) | 11 color hues | Embossed surface pattern |
Shin et al. [4] | Lines, orientation, grating. The first eight colors are divided into 20° angle(rainbow shape) | 90 (8 hues + 4 levels of lightness and 5 levels of saturation for each hues + 9 levels of brown and achromatic) | Embossed surface pattern |
Cho et al. [16] | Dots, lines, and curves (pictograms) | Simplified: 29 (6 hues + 2 levels of lightness and 2 levels of saturation for each hue + 5 levels of achromatic) | Embossed surface pattern |
Extended: 53 (12 hues + 2 levels of lightness and 2 levels of saturation for each hue + 5 levels of achromatic) | |||
This work | Simplified: RYB color wheel model: six colors are divided into 60° angles in the 6 RYB color wheel (watch type) | 24 (6 hues + 3 levels of lightness for each hue + 6 levels of achromatic) | Automatic spatial representation as assistive wearable device, Embossed surface pattern |
Extended: eight colors are divided into 45° angles in the 8 RYB color wheel (watch type). | 32 (8 hues + 3 levels of color tones for each hue + 8 levels of achromatic) |
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations. |
© 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Jabbar, M.S.; Lee, C.-H.; Cho, J.D. ColorWatch: Color Perceptual Spatial Tactile Interface for People with Visual Impairments. Electronics 2021, 10, 596. https://doi.org/10.3390/electronics10050596
Jabbar MS, Lee C-H, Cho JD. ColorWatch: Color Perceptual Spatial Tactile Interface for People with Visual Impairments. Electronics. 2021; 10(5):596. https://doi.org/10.3390/electronics10050596
Chicago/Turabian StyleJabbar, Muhammad Shahid, Chung-Heon Lee, and Jun Dong Cho. 2021. "ColorWatch: Color Perceptual Spatial Tactile Interface for People with Visual Impairments" Electronics 10, no. 5: 596. https://doi.org/10.3390/electronics10050596