Comparing Web Accessibility Evaluation Tools and Evaluating the Accessibility of Webpages: Proposed Frameworks
Abstract
:1. Introduction
2. Background
2.1. WAVE
2.2. SiteImprove
3. Related Works
3.1. Web Accessibility Evaluation Approaches
3.2. Comparing the Performance of Web Accessibility Evaluation Tools
3.3. Metrics for Web Accessibility Evaluation
3.4. Accessibility Evaluation of Saudi Government Websites
4. The Proposed Frameworks for Comparing Tool and Web Accessibility
4.1. Study 1: Framework for Comparing the Performance of Web Accessibility Tools
Algorithm 1: Computing CER score for each tool. |
4.2. Study 2: Framework for Evaluating Webpages in Terms of Web Accessibility
Algorithm 2: Computing WAA score for each webpage. |
5. Experimental Methodology
6. Experimental Results and Discussion
6.1. Study 1: Result of Comparing Web-Accessibility Tools
6.2. Study 2: Result of Evaluating Webpages in Terms of Web Accessibility
6.3. Accessibility Issues in Saudi Universities Homepages
6.4. General Finding
7. Conclusions and Future Works
Funding
Conflicts of Interest
References
- Caldwell, B.; Cooper, M.; Reid, L.G.V.; Erheiden, G. Web Content Accessibility Guidelines (WCAG) 2.0. 2008. WWW Consortium (W3C). Available online: https://www.w3.org/TR/WCAG20/ (accessed on 20 August 2019).
- United States Lawsoverview of the Rehabilitation Act of 1973 (Section 504 And 508). Available online: https://webaim.org/articles/laws/usa/rehab (accessed on 15 September 2019).
- National Government Website Standard. Available online: https://www.yesser.gov.sa/EN/Methodologies/NATIONAL_ICT_DIGITALSTANDARDS_PRINCIPLES/Pages/National-Government-Website-Standard.aspx (accessed on 13 September 2019).
- Brewer, J. Web accessibility highlights and trends. In Proceedings of the 2004 International Cross-disciplinary Workshop on Web Accessibility (W4A ’04), New York, NY, USA, 17–22 May 2004; pp. 51–55. [Google Scholar] [CrossRef]
- Vigo, M.; Brown, J.; Conway, V. Benchmarking web accessibility evaluation tools: measuring the harm of sole reliance on automated tests. In Proceedings of the 10th International Cross-Disciplinary Conference on Web Accessibility, Rio de Janeiro, Brazil, 13–15 May 2013; ACM: New York, NY, USA, 2013; p. 1. [Google Scholar]
- Lundqvist, S.; Ström, J. Web Accessibility in E-Learning: Identifying And Solving Accessibility Issues for Wcag 2.0 Conformance in an E-Learning Application. Student Thesis. 2018. Available online: http://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-148913 (accessed on 25 September 2018).
- Alshamari, M. Accessibility evaluation of arabic e-commerce web sites using automated tools. J. Softw. Eng. Appl. 2016, 9, 439–451. [Google Scholar] [CrossRef] [Green Version]
- Kaur, A.; Dani, D. Banking websites in india: An accessibility evaluation. CSI Trans. ICT 2014, 2, 23–34. [Google Scholar] [CrossRef] [Green Version]
- Akgül, Y.; Vatansever, K. Web accessibility evaluation of government websites for people with disabilities in turkey. J. Adv. Manag. Sci. 2016, 4, 201–210. [Google Scholar] [CrossRef]
- Adepoju, S.A.; Shehu, I.S.; Bake, P. Accessibility evaluation and performance analysis of e-government websites in nigeria. J. Adv. Inf. Technol. 2016, 7. [Google Scholar] [CrossRef]
- AlMourad, B.; Kamoun, F. Accessibility evaluation of dubai e-government websites: Findings and implications. J. -Gov. Stud. Best Pract. 2013, 2013, 978647. [Google Scholar]
- Brajnik, G. Comparing accessibility evaluation tools: A method for tool effectiveness. Univers. Access Inf. Soc. 2004, 3, 252–263. [Google Scholar] [CrossRef]
- Martín, A.; Cechich, A.; Rossi, G. Comparing Approaches to Web Accessibility Assessment. In Handbook of Research on Web Information Systems Quality; IGI Global: Hershey, PA, USA, 2008; pp. 181–205. [Google Scholar]
- Shadi Abou-Zahra, N.S.; Keen, L. Selecting Web Accessibility Evaluation Tools. Available online: https://www.w3.org/WAI/test-evaluate/tools/selecting/ (accessed on 5 May 2018).
- Eggert, E.; Abou-Zahra, S. Web Accessibility Evaluation Tools List. Available online: https://www.w3.org/WAI/ER/tools/index (accessed on 5 July 2018).
- Al-Khalifa, H.S. Wcag 2.0 semi-automatic accessibility evaluation system: Design and implementation. Comput. Inf. Sci. 2012, 5, 73. [Google Scholar] [CrossRef] [Green Version]
- Abuaddous, H.Y.; Jali, M.Z.; Basir, N. Web accessibility challenges. Int. J. Adv. Comput. Sci. Appl. 2016, 7, 172–181. [Google Scholar] [CrossRef] [Green Version]
- Abanumy, A.; Al-Badi, A.; Mayhew, P. e-government website accessibility: In-depth evaluation of Saudi Arabia and oman. Electron. J. Gov. 2005, 3, 99–106. [Google Scholar]
- MasoodRana, M.; Fakrudeen, M.; Rana, U. Evaluating web accessibility of university web sites in the kingdom of Saudi Arabia. Int. J. Technol. Knowl. Soc. 2011, 7, 1–15. [Google Scholar]
- Al-Khalifa, H.S.; Al-Kanhal, M.; Al-Nafisah, H.; Al-soukaih, N.; Al-hussain, E.; Al-onzi, M. A pilot study for evaluating arabic websites using automated wcag 2.0 evaluation tools. In Proceedings of the 2011 International Conference on Innovations in Information Technology (IIT 2011), Abu Dhabi, United Arab, 25–27 April 2011; pp. 293–296. [Google Scholar]
- Alahmadi, T.; Drew, S. An evaluation of the accessibility of top-ranking university websites: Accessibility rates from 2005 to 2015. In Proceedings of the DEANZ Biennial Conference, Hamilton, New Zealand, 17–20 April 2016; pp. 224–233. [Google Scholar]
- Brajnik, G.; Yesilada, Y.; Harper, S. The expertise effect on web accessibility evaluation methods. Hum.–Comput. Interact. 2011, 26, 246–283. [Google Scholar]
- Hong, S.; Katerattanakul, P.; Lee, D. Evaluating government website accessibility: Software tool vs. human experts. Manag. Res. News 2007, 31, 27–40. [Google Scholar] [CrossRef]
- Elkabani, I.; Hamandi, L.; Zantout, R.; Mansi, S. Toward better web accessibility. In Proceedings of the 2015 5th International Conference on Information & Communication Technology and Accessibility (ICTA), Marrakech, Morocco, 21–23 December 2015; pp. 1–6. [Google Scholar] [CrossRef]
- Grantham, J.; Grantham, E.; Powers, D. Website accessibility: An australian view. In Proceedings of the Thirteenth Australasian User Interface Conference (AUIC ’12), Melbourne, Australia, 31 January–3 February 2012; Volume 126, pp. 21–28. Available online: http://dl.acm.org/citation.cfm?id=2512125.2512129 (accessed on 9 March 2019).
- Alayed, A.S. A Framework and Checklist for Localised Web Content Accessibility Guidelines for Arabic University Websites in Saudi Arabia. Ph.D. Thesis, University of Southampton, Southampton, UK, 2018. [Google Scholar]
- Petrie, H.; Kheir, O. The relationship between accessibility and usability of websites. In Proceedings of the SIGCHI Conference on Human Factors In Computing Systems, San Jose, CA, USA, 28 April–3 May 2007; ACM: New York, NY, USA, 2007; pp. 397–406. [Google Scholar]
- Petrie, H.; Bevan, N. The evaluation of accessibility, usability, and user experience. Univers. Access Handb. 2009, 1, 1–16. [Google Scholar]
- Petrie, H.; Hamilton, F.; King, N.; Pavan, P. Remote usability evaluations with disabled people. In Proceedings of the SIGCHI Conference on Human Factors In Computing Systems, Montreal, QC, Canada, 22–27 April 2006; ACM: New York, NY, USA, 2006; pp. 1133–1141. [Google Scholar]
- Kumar, K.L.; Owston, R. Evaluating e-learning accessibility by automated and student-centered methods. Educ. Technol. Res. Dev. 2016, 64, 263–283. [Google Scholar] [CrossRef]
- Latif, M.H.A.; Masrek, M.N. Accessibility evaluation on malaysian e-government websites. J. E-Gov. Stud. Best Pract. 2010, 2010, 935272. [Google Scholar]
- Al-Khalifa, H.S. Exploring the accessibility of Saudi Arabia e-government websites: A preliminary results. In Proceedings of the 4th International Conference on Theory and Practice of Electronic Governance (ICEGOV ’10), Cairo, Egypt, 1–4 December 2008; ACM: New York, NY, USA, 2010; pp. 274–278. [Google Scholar] [CrossRef]
- Al-Khalifa, H.S. The accessibility of Saudi Arabia government web sites: An exploratory study. Univers. Access Inf. Soc. 2012, 11, 201–210. [Google Scholar] [CrossRef]
- Al-Khalifa, H.S.; Baazeem, I.; Alamer, R. Revisiting the accessibility of Saudi Arabia government websites. Univers. Access Inf. Soc. 2017, 16, 1027–1039. [Google Scholar] [CrossRef]
- Khan, M.A.; Buragga, K.A. Effectiveness of accessibility and usability of government websites in Saudi Arabia. Can. J. Pure Appl. Sci. 2010, 4, 1127–1131. [Google Scholar]
- Alotaibi, S.J. Evaluating the blackboard system based on web accessibility and usability guidelines. Int. J. Digit. Soc. 2015, 6, 1094–1101. [Google Scholar] [CrossRef]
- Sullivan, T.; Matson, R. Barriers to use: Usability and content accessibility on the web’s most popular sites. In Proceedings of the 2000 Conference on Universal Usability (CUU ’00)y, Arlington, VA, USA, 16–17 November 2000; ACM: New York, NY, USA, 2000; pp. 139–144. [Google Scholar] [CrossRef]
- Parmanto, B.; Zeng, X. Metric for web accessibility evaluation. J. Am. Soc. Inf. Sci. Technol. 2005, 56, 1394–1404. [Google Scholar] [CrossRef]
- Cluster, W. Unified Web Evaluation Methodology (UWEM 10), 2006. In Proceedings of the 11th International Conference (ICCHP 2008), Linz, Austria, 9–11 July 2008; pp. 394–401. [Google Scholar]
- Bühler, C.; Heck, H.; Perlick, O.; Nietzio, A.; Ulltveit-Moe, N. Interpreting results from large scale automatic evaluation of web accessibility. In Computers Helping People with Special Needs; Miesenberger, K., Klaus, J., Zagler, W.L., Karshmer, A.I., Eds.; Springer: Berlin/Heidelberg, Germany, 2006; pp. 184–191. [Google Scholar]
- Song, S.; Bu, J.; Shen, C.; Artmeier, A.; Yu, Z.; Zhou, Q. Reliability aware web accessibility experience metric. In Proceedings of the Internet of Accessible Things (W4A ’18), Lyon, France, 23–25 April 2018; ACM: New York, NY, USA, 2018; pp. 24:1–24:4. [Google Scholar] [CrossRef]
- Al-Faries, A.; Al-Khalifa, H.S.; Al-Razgan, M.S.; Al-Duwais, M. Evaluating the accessibility and usability of top saudi e-government services. In Proceedings of the 7th International Conference on Theory and Practice of Electronic Governance (ICEGOV ’13), Seoul, Korea, 22–25 October 2013; ACM: New York, NY, USA, 2013; pp. 60–63. [Google Scholar] [CrossRef]
- Alturki, U.T.; Aldraiweesh, A. Evaluating the usability and accessibility of lms “blackboard” at king saud university. Contemp. Issues Educ. Res. 2016, 9, 33–44. [Google Scholar] [CrossRef]
University | Maturity Level Indicator | Category |
---|---|---|
Taibah | 12.82 | red |
Najran | 37.21 | red |
King Fahad | 62.16 | Yellow |
King Khaled | 69.77 | Yellow |
Prince Sattam | 85.71 | Green |
King Saud | 93.75 | Green |
University | Wave | SiteImprove | ||||
---|---|---|---|---|---|---|
Errors Message | Occurrences No. | Corresp. Criteria | Errors Message | Occurrences No. | Corresp. Criteria | |
Taibah University | Empty button | 3 | 1.1.1 2.4.4 | Not Detected | 0 | - |
Not Detected | 0 | - | Image with no alt attribute | 2 | 1.1.1 | |
Linked image missing alternative text | 2 | 1.1.1 2.4.4 | Image link has no alternative text | 2 | 2.4.4 | |
Empty link | 10 | 2.4.4 | Not Detected | 0 | - | |
Not Detected | 0 | - | Non-distinguishable links | 2 | 2.4.4 | |
Not Detected | 0 | - | There is no top level heading (H1) on the page | 1 | 1.3.1 | |
Not Detected | 0 | - | Naming generic landmarks | 2 | 1.3.1 | |
Missing form label | 3 | 1.1.1 1.3.1 2.4.6 3.3.2 | Input field is missing a description | 3 | 1.3.1 3.3.2 4.1.2 |
University | Wave | SiteImprove | ||||
---|---|---|---|---|---|---|
Errors Message | Occurrences No. | Corresp. Criteria | Errors Message | Occurrences No. | Corresp. Criteria | |
Prince Sattam University | Linked image missing alternative text | 1 | 1.1.1 2.4.4 | Image link has no alternative text | 1 | 2.4.4 |
Empty link | 21 | 2.4.4 | Not Detected | 0 | - | |
Broken ARIA reference | 3 | 1.3.1 | Incorrect reference for ’aria-labelledby’ | 3 | 1.1.1 1.3.1 2.4.4 3.3.2 4.1.2 | |
Not Detected | 0 | - | Links should be combined | 8 | 1.1.1 | |
Not Detected | 0 | - | Link refers to a non-existing element | 1 | 1.3.1 | |
Not Detected | 0 | - | Use unique identification for elements | 4 | 1.3.1 4.1.1 | |
Not Detected | 0 | - | Non-distinguishable links | 11 | 2.4.4 | |
Not Detected | 0 | - | Element not highlighted on focus | 87 | 2.4.7 |
University | Wave | SiteImprove | ||||
---|---|---|---|---|---|---|
Errors Message | Occurrences No. | Corresp. Criteria | Errors Message | Occurrences No. | corresp. Criteria | |
King khaled University | Missing alternative text | 9 | 1.1.1 | Image with no alt attribute | 11 | 1.1.1 |
Linked image missing alternative text | 8 | 1.1.1 2.4.4 | Image link has no alternative text | 8 | 2.4.4 | |
Empty heading | 15 | 1.3.1 2.4.6 2.4.1 | The heading is missing text | 15 | 1.3.1 | |
Empty button | 4 | 1.1.1 2.4.4 | Not Detected | 0 | - | |
Empty link | 4 | 2.4.4 | Not Detected | 0 | - | |
Not Detected | 0 | - | Presentational attributes used | 1 | 1.3.1 | |
Not Detected | 0 | - | There is no top level heading (H1) on the page | 1 | 1.3.1 | |
Not Detected | 0 | - | Element not highlighted on focus | 96 | 2.4.7 | |
Not Detected | 0 | - | Non-distinguishable links | 10 | 2.4.4 |
University | Wave | SiteImprove | ||||
---|---|---|---|---|---|---|
Errors Message | Occurrences No. | Corresp. Criteria | Errors Message | Occurrences No. | Corresp. Criteria | |
King Saud University | Linked image missing alternative text | 6 | 1.1.1 2.4.4 | Image link has no alternative text | 6 | 2.4.4 |
Missing alternative text | 1 | 1.1.1 | Image with no alt attribute | 7 | 1.1.1 | |
Empty heading | 1 | 1.3.1 2.4.6 2.4.1 | The heading is missing text | 1 | 1.3.1 | |
Empty link | 13 | 2.4.4 | Not Detected | 0 | - | |
Not Detected | 0 | - | Non-distinguishable links | 9 | 2.4.4 | |
Not Detected | 0 | - | ’Select box’ without a description | 1 | 1.3.1 | |
Not Detected | 0 | - | Presentational attributes used | 1 | 1.3.1 | |
Not Detected | 0 | - | There is no top level heading (H1) on the page | 1 | 1.3.1 | |
Not Detected | 0 | - | The iFrame is missing a description | 1 | 1.3.1 | |
Not Detected | 0 | - | Element not highlighted on focus | 57 | 2.4.7 |
University | Wave | SiteImprove | ||||
---|---|---|---|---|---|---|
Errors Message | Occurrences No. | Corresp. Criteria | Errors Message | Occurrences No. | Corresp. Criteria | |
King Fahad University | Missing alternative text | 4 | 1.1.1 | Image with no alt attribute | 4 | 1.1.1 |
Linked image missing alternative text | 22 | 1.1.1 | Image link has no alternative text | 22 | 2.4.4 | |
Missing form label | 1 | 3.3.2 | Input field is missing a description | 1 | 1.3.1 | |
Empty heading | 1 | 1.3.1 2.4.6 2.4.1 | Not Detected | - | - | |
Empty link | 1 | 2.4.4 | Not Detected | - | - | |
Empty table header | 1 | 1.3.1 | Not Detected | - | - | |
Not Detected | 0 | - | Image with no alt attribute | 20 | 1.1.1 | |
Not Detected | 0 | - | The image does not have the correct alternative | 15 | 1.1.1 | |
Not Detected | 0 | - | Presentational attributes used | 10 | 1.3.1 | |
Not Detected | 0 | - | Non-distinguishable links | 11 | 2.4.4 | |
Not Detected | 0 | - | Element not highlighted on focus | 81 | 2.4.7 |
University | Wave | SiteImprove | ||||
---|---|---|---|---|---|---|
Errors Message | Occurrences No. | Corresp. Criteria | Errors Message | Occurrences No. | Corresp. Criteria | |
Najran university | Missing alternative text | 1 | 1.1.1 | Image with no alt attribute | 1 | 1.1.1 |
Linked image missing alternative text | 3 | 1.1.1 | Image link has no alternative text | 3 | 2.4.4 | |
Missing form label | 10 | 2.4.4 | Not Detected | 0 | - | |
Empty heading | 1 | 1.3.1 2.4.6 2.4.1 | Heading is missing text | 1 | 1.3.1 | |
Empty link | 10 | 2.4.4 | Not Detected | 0 | - | |
not detected | 0 | - | Alternative text for image is identical to link text | 6 | 1.1.1 | |
not detected | 0 | - | Image link is missing alternative text | 1 | 2.4.4 | |
not detected | 0 | - | Link text used for multiple different destinations | 11 | 2.4.4 | |
not detected | 0 | - | Select box has no description | 9 | 3.3.2 | |
not detected | 0 | - | Element ID is not unique | 2 | 4.1.1 | |
not detected | 0 | - | Missing button in form | 2 | 3.3.2 | |
not detected | 0 | - | HTML is used to format content | 3 | 1.3.1 | |
not detected | 0 | - | Input field has no description | 1 | 1.3.1 | |
not detected | 0 | - | Generic landmarks are not named | 4 | 1.3.1 | |
not detected | 0 | - | iFrame is missing a description | 1 | 4.1.2 | |
not detected | 0 | - | Invalid WAI-ARIA role, state or property | 2 | 4.1.2 | |
not detected | 0 | - | Adjacent links used for same destination | 2 | 1.1.1 |
© 2020 by the author. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).
Share and Cite
Alsaeedi, A. Comparing Web Accessibility Evaluation Tools and Evaluating the Accessibility of Webpages: Proposed Frameworks. Information 2020, 11, 40. https://doi.org/10.3390/info11010040
Alsaeedi A. Comparing Web Accessibility Evaluation Tools and Evaluating the Accessibility of Webpages: Proposed Frameworks. Information. 2020; 11(1):40. https://doi.org/10.3390/info11010040
Chicago/Turabian StyleAlsaeedi, Abdullah. 2020. "Comparing Web Accessibility Evaluation Tools and Evaluating the Accessibility of Webpages: Proposed Frameworks" Information 11, no. 1: 40. https://doi.org/10.3390/info11010040