Accessibility of Healthcare Sites: Evaluation by Automated Tools

Full text

Accessibility of Healthcare Sites:
Evaluation by Automated Tools
Kumari Sarita , Parminder Kaur , and Satinder Kaur
Abstract Websites have played a significant role in the evolution of the digital world.
These are the primary sources of communication and dissemination of information all
over the world. The utilization of e-health platforms is getting a universal reputation
in medical field. The most challenging job for the site engineers or developers is
to provide the universal access to all kinds of users having impairments like vision,
hearing, cognitive, physical, literacy with no barrier. So, the prime requirement is to
gauge the accessibility of these websites and same can be ensured by using automated
accessibility evaluation tools. This paper aims to evaluate the accessibility of top
six healthcare websites in India. These sites are selected based on Alexa ranking.
The accessibility of these sites is inspected using three automated tools: AChecker,
WAVE and TAW against the conformance of WCAG 2.0 level AA. A Five-Level
Accessibility Criteria (FLAC) is proposed to determine the level of accessibility of
a particular site depending upon error percentage. The result findings show that all
the three tools produce different evaluation results for the same sites. Therefore, it is
recommended to include the user testing and expert testing with automated testing so
that more accurate and consistent results can be obtained. Furthermore, this work can
be extended by evaluating more sites from this domain as well as from other domains.
In addition, some more commercial tools can be used to enhance the productivity of
evaluation. Further parameters and tools can be increased to gauge the accessibility
under the higher conformance of WCAG 2.1 level AA.
Keywords Accessibility ·Healthcare sites ·Automated accessibility evaluation
tools ·WCAG 2.0 ·AChecker ·Wav e ·TAW ·FLAC
K. Sarita (B)·P. Ka u r
Department of Computer Science, Guru Nanak Dev University, Amritsar, India
e-mail: parminder.dcse@gndu.ac.in
S. Kaur
Department of Computer Engineering and Technology, Guru Nanak Dev University,
Amritsar, India
e-mail: satinder.dcet@gndu.ac.in
© The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd. 2022
M. Saraswat et al. (eds.), Proceedings of International Conference on Data Science
and Applications, Lecture Notes in Networks and Systems 287,
https://doi.org/10.1007/978-981-16- 5348-3_50
625

626 K. Sarita et al.
1 Introduction
Great quality medical services are the most imperative constituent in today’s busy life.
In modern era, the people prefer to use online healthcare sites for their routine health
checkup and for any guidance to health related problem. There are extensive varieties
of healthcare sites for the people with disabilities or old aged people. However, the
barriers in accessing the online services of these sites make it difficult for various
kinds of users. Hence, it is essential to make these sites fully accessible to remove
the barriers faced by different impaired users [13].
Web Accessibility is meant to provide the equality of accessing the web and tech-
nologies to the people with different forms of disabilities [13]. According to W3C,
the site architects should keep in mind all different categories of inabilities during the
design and development of websites [11]. To make the web fully accessible, World
Wide Web Consortium (W3C) has framed some guidelines and standards like Web
Content Accessibility Guidelines: WCAG 1.0, WCAG 2.0 and WCAG 2.1 [25,27,
29]. WCAG 1.0 is the first version published based on the priority levels, checkpoints
and conformance levels, whereas the enhanced versions WCAG 2.0 and WCAG 2.1
have been formulated based on principles, success criteria, checkpoints and con-
formance levels [27,29]. Majority of researchers have worked on the accessibility
evaluation of healthcare sites in terms of parameters like accessibility, readability,
information, communication and electronic services [2,19,21]. The objective of
this paper is to appraise top six healthcare websites in India against the conformance
of WCAG 2.0 with the use of AChecker, WAVE and TAW.
The rest of the paper is organized as follows: Sect. 2throws light on the review
of literature including some limitations on website accessibility assessment by using
automated tools. Section 3portrays the methodology, automated tools, quality factors
and proposed criteria adopted for evaluation. Section 4evaluates the e-learning sites
and discusses the evaluation results produced by automated tools as per FLAC.
Finally, Sect. 5concludes the paper based on major findings and provides some
recommendations for future work.
2 Literature Review
In order to assess the websites in terms of various aspects of accessibility, numerous
researchers have worked in this discipline [3,12,16]. The literature review provides
the overlook of previous studies regarding the accessibility evaluation of healthcare
sites and usability of automated tools.
Llinás et al. [19] compared the sites of Spanish, American and British hospitals,
and after comparison, it was found that not all sites passed the basic requirements
of accessibility guidelines. Kaur et al. [15] conducted a study on the 280 hospital
websites in the metro cities of India. The study was conducted using automated
tools against WCAG 2.0, and it was found that the websites suffered from various

Accessibility of Healthcare Sites: Evaluation by Automated Tools 627
accessibility problems. Salarvand et al. [23] evaluated the quality of public hospital
sites in Tehran. The results showed that the sites found with very low quality score.
The reseachers carried out a nitty-gritty examination on AChecker and found
numerous potential hindrances during the research [10]. Roy et al. [22], Ahmi and
Mohamad [4] implemented AChecker to assess the accessibility of academic web-
sites. The result findings signified that some websites could not pass even the lowest
basic level of accessibility guidelines. Although Kumar and Owston [18] determined
e-learning accessibility by using AChecker. Another evaluation by AChecker was
performed to determine the accessibility of banking websites [20]. In recent studies,
Akgul [5], Barricelli et al. [7] made the use of AChecker to gauge the accessibility of
education websites. The result findings revealed that the websites were not developed
according to the guidelines specified for accessibility.
AlMeraj et al. [6] utilized AChecker and WAVE to evaluate the accessibility of
higher education sites in Kuwait. The results explored that no website was able to
pass even the level A of WCAG 2.0. Furthermore, Ismail and Kuppusami [12] com-
pleted an exploratory examination on the accessibility of university’ homepages by
AChecker and WAVE. The results exhibited that various issues related to accessibil-
ity, speed, navigability and contents were found during the assessment.
Jati and Dominic [14] conducted a study by using TAW to appraise the accessibility
of the aggregate of 90 sites including education, government, business websites
in Malaysia. The outcomes demonstrated that the majority of Malaysian websites
did not meet WCAG standards like HTML code, picture, visual aids, form label,
headings, element, link, navigation bar, so forth.
2.1 Limitations of the Studies
Depending upon the findings from the literature reviewed, certain limitations are dis-
closed. Some studies are single domain oriented and limited to the number of tools.
A single tool is not enough for the complete evaluation of site accessibility as it did
not cover all aspects of evaluation. There is always a discrepency between the same
sites results interpreted by various tools. Additionally, there is a lack of an effective
methodology to determine the efficacy of an automated tool. The current study evalu-
ated the accessibility of top six Indian healthcare sites using the combination of three
automated tools like AChecker, WAVE and TAW. These sites are selected based upon
the popularity of the site determined from Alexa rank. A Five-Level Accessibility
Criteria(FLAC) is proposed in the present study to find the level of accessibility.
3 Methodology
This investigation adopts the method of automated tools for the evaluation of health-
care websites. To start the testing, one has to enter the site url into the address box via
an automated tool and press enter. Once the evaluation process is over, the assess-

628 K. Sarita et al.
Tabl e 1 Healthcare sites with Alexa rank
S. No. Site name Global rank Country rank
1. https://www.practo.com 4794 460
2. https://www.medindia.net 7494 929
3. https://www.healthkart.com 20305 1868
4. https://www.mohfw.gov.in 20966 2216
5. https://www.onlymyhealth.com 44836 6215
6. https://www.drbatras.com 153648 18582
ment report is presented to the user. This paper aims to examine the accessibility of
the six healthcare sites. These sites are selected based on Alexa Traffic Rank: global
as well as country rank as given in Table 1. Based on the ranking, the sitepracto is
found the most popular site and drbatras is found with the lowest rank in popularity.
These sites are evaluated using three automated tools: AChecker, WAVE and TAW.
3.1 Automated Tools
This study encompasses the use of three automated tools: AChecker, WAVE and
TAW to inspect the accessibility of above sites.
AChecker: AChecker is abbreviated as Accessibility Checker, created by the Adap-
tive Technology Research Centre at the University of Toronto and can be found at
http://achecker.ca [10]. Clients can test the websites simply by using site page URL,
HTML file upload, Paste HTML markup. AChecker tests for compliance with WCAG
2.0 (Web Content Accessibility Guideline 2.0), WCAG 1.0 (Web Content Accessi-
bility Guideline 1.0), American Section 508, US Federal Procurement Standard,
German BITV, German Government Standard, Italian Stanca Act, Italian Accessi-
bility Legislation [28]. It presents three types of problems: known problems, likely
problems and potential problems [1].
•Known Problems: These type of problems are easily detectable without any
human involvement.
•Likely Problems: Such kind of problems can be considered as apparent problems
which require human intervention to take decision.
•Potential Problems: Potential problems cannnot be recognized by the tool and
need full human judgement.
WAV E : WAVE stands for Web Accessibility and Versatile Evaluator and is an open,
online tool with Firefox add-on. This tool can be used by entering a site page URL.
The tool inspects the sites with various guidelines such as WCAG 2.1 (Web Content
Accessibility Guideline), WCAG 2.0 (Web Content Accessibility Guideline), WCAG

Accessibility of Healthcare Sites: Evaluation by Automated Tools 629
1.0 (Web Content Accessibility Guideline), American Section 508, US Federal Pro-
curement Standard [17,28]. These guidelines are not visible during the assessment
by WAVE tool [9]. The assessment results are shown via different coloured icons.
Red icons mean accessibility errors, yellow icons are meant to alerts, green icons
show accessibility features, light blue icons present basic, semantic, or navigational
elements. The current study carried out the evaluation only on the basis of errors, con-
trast errors and features. Like AChecker, WAVE has no choices of HTML validator
and CSS validator.
TAW: TAW stands for Test de Accessibilidad Web and is available at https://www.
tawdisnet/ [8]. It is made by the Spanish Foundation Centre for the Development
of Information and Communication Technologies in Asturias (CTIC) [9,24]. TAW
assesses against WCAG 2.1 (Web Content Accessibility Guideline), WCAG 2.0 (Web
Content Accessibility Guideline), WCAG 1.0 (Web Content Accessibility Guide-
line) [17,28]. It creates the assessment report with a depiction of obstacles (problems,
warnings, and not reviewed) [9].
•Problems: These obstacles can easily be identified without any human involve-
ment.
•Warnings: Warnings can be resolved with the help of human review .
•Not Reviewed: These require totally expert intervention for decision making.
3.2 Principles to Evaluate Site Accessibility
The websites accessibility can be evaluated on the basis of four WCAG 2.0 principles:
Perceivable, operable, understandable and robust.
Perceivable: If the content, interface and its components are identifiable in sensing
way such as visually, hearing or touch, then the web content is supposed to be
perceivable [26].
Operable: Operability implies that all controls, buttons, navigation and other com-
ponents are effectively usable by all kinds of poeple [26].
Understandable: If the users are able to understand, use and learn the interface, then
the web content is understandable [26].
Robust: The users ought to have the option to select the technology through which
they communicate with the sites and web content [26].
3.3 Proposed Criteria
The study executed three automated tools to evaluate the accesibility of six health-
care sites. The authors proposed a Five-Level Accessibility Criteria(FLAC) as in

630 K. Sarita et al.
Tabl e 2 Five level accessibility criteria (FLAC)
Level of accessibility Error percentage (%)
Maximal 0–10
High 11–30
Moderate 31–60
Low 61–90
Minimal 91–100
Table 2to determine the level of accessibility based on error percentage identified in
a particular site. This criteria will be helpful for the site makers to get an idea about
the accessibility level of their sites for further improvement.
4 Results and Discussions
Table 3shows the evaluation results generated by AChecker. The sites practo and
onlymyhealth gives zero values except some HTML and CSS validation problems
when evaluated with AChecker. So, based on HTML and CSS validation, the site
practo achieves the highest level of accessibility followed by the site onlymyhealth.
Overall, the website medindia contains the greatest number of problems (Fig. 1).
Operable factor is found most violated while robust the least violated. The most
experienced problems are absence of alt attribute for non-text content, information,
structure and relationship in text, and also the errors related to contrast, resize, links,
headings, label, language, etc.
Table 4presents the results obtained from WAVE. The results are evaluated based
on errors, contrast errors and features. WAVE tools records the site medindia with
the largest number of issues, whereas the site mohfw is recorded with the smallest
number of issues (Fig. 2). WAVE tool found several errors such as missing alternative
text, linked image missing, missing and empty form label, very low contrast, etc.
Table 5frames the results produced by TAW. According to findings, the site
healthkart shows the highest number of errors and the site mohfw shows the low-
est number of errors (Fig. 3). TAW finds perceivable factor the most violated and
understandable the least violated factor. A lot of errors include text alternatives, info
and relationship, colour, contrast, resize, images, keyboard, seizures, focus, link,
heading, label, language, parsing, etc. are found during evaluation by TAW.

Accessibility of Healthcare Sites: Evaluation by Automated Tools 631
Tabl e 3 Results by AChecker
Problem Principle practo medindia healthkart mohfw onlymyhealth drbatras
Known problems Perceivable 0 59 035 0295
Operable 0 194 0 1 0 1
Understandable 021 0 3 0 4
Robust 0 1 0 0 0 1
Tota l 0275 039 0301
Likely problems Perceivable 0 3 0 0 0 11
Operable 0 1 0 0 0 3
Understandable 0 3 0 0 0 1
Robust 0 0 0 0 0 0
Tota l 0 7 0 0 0 15
Potential
problems
Perceivable 0 586 10 35 0229
Operable 0 1092 16 122 0741
Understandable 0100 3 2 0 38
Robust 0 0 0 0 0 0
Tota l 01778 29 159 00 1008

632 K. Sarita et al.
Tabl e 4 Results by WAVE
Type practo medindia healthkart mohfw onlymyhealth drbatras
Errors 112 4 3 45 45 9
Contrast
errors
36 11 0 5 16 12
Features 123 39 103 33 728
Tot al 271 54 106 83 68 49
Tabl e 5 Results by TAW in detail
Problem Principle practo medindia healthkart mohfw onlymyhealth drbatras
Problems Perceivable 25 80 121 645 59
Operable 40 96 3 7 8 5
Understandable 431 3 2 10 6
Robust 42 40 269 918 20
Total 111 247 396 24 81 90
Warnings Perceivable 57 146 469 150 131 124
Operable 27 122 83 24 58 110
Understandable 048 12 0 6 24
Robust 42 13 2 0 20 0
Total 126 329 566 174 215 258
Not
reviewed
Perceivable 4 3 4 4 4 4
Operable 8 5 7 6 5 7
Understandable 5 4 4 5 4 4
Robust 0 0 0 0 0 0
Total 17 12 15 15 13 15
Fig. 1 Violations by AChecker

Accessibility of Healthcare Sites: Evaluation by Automated Tools 633
Fig. 2 Violations by WAVE
Fig. 3 Violations by TAW
Fig. 4 Combined results by AChecker, WAVE, TAW

634 K. Sarita et al.
Tabl e 6 FLAC results
S. No. Site name Achecker TAW WAV E
1. https://www.practo.com Maximal Maximal Moderate
2. https://www.medindia.net Moderate Moderate Maximal
3. https://www.healthkart.com Maximal Moderate Maximal
4. https://www.mohfw.gov.in Maximal Maximal Maximal
5. https://www.onlymyhealth.
com
Maximal Maximal Maximal
6. https://www.drbatras.com High Maximal Maximal
4.1 Accessibility Evaluation as per FLAC
Each site is assigned the level of accessibility based on the error percentage in the
range by a Five-Level Accessibility Criteria (FLAC) as proposed in Table 2. As per
FLAC, the comparative analysis of the six sites by the toolsis shown in Table 6.
From the evaluation results framed in Table 6, it is observed that all the three tools
produce different results for different sites except mohfw and onlymyhealth. These
two sites are found with maximal level of accessibility after the evaluation by all
three tools. In addition, AChecker evaluates the site medindia as moderately acces-
sible while TAW finds two sites medindia and healthkart as moderately accessible.
Further, according to WAVE tool, all the sites except practo achieve the maximal level
of accessibility. Although, Achecker finds Operable factor with the largest number of
violations whereas robu st factor with the least number of violations when evaluated
by AChecker. However, evaluation by TAW indicates that the most violated factor is
Perceivable while Understandable is the least violated factor. Furthermore, none of
the websites fully qualified the guidelines specified in WCAG 2.0 level AA. More-
over, based on error findings, AChecker may be the more appropriate tool followed
by TAW and WAVE (Fig. 4). As no study is free from limitations so current study
also suffers from certain limitations. The present study relies only on the automated
evaluation methodology. The results need to be reviewed manually as no tool can
perform the complete automated evaluation by itself only due to subjective nature of
some guidelines. It requires human intervention to make decisions about the errors
not analysed by the tools. In addition, commercial tools are not used for current
evaluation as these are highly expensive. Further, in case of WAVE tool, alerts, struc-
tural elements, ARIA related problems are excluded during inspection. Moreover,
automated testing alone is not enough for the complete evaluation of the websites
accessibility so it is recommended to include the user testing and expert testing along
with automated testing. Future work can be extended by in-depth evaluation of more
sites from this domain as well as from other domains. Further parameters and tools
can be increased for accessibility evaluation under the higher conformance of WCAG
2.1 level AA.

Accessibility of Healthcare Sites: Evaluation by Automated Tools 635
5 Conclusion and Future Scope
Healthcare sites are becoming progressively significant for health conscious people.
Subsequently, the engineers of these sites need to plan and create accessible sites
to make the information effectively available to all individuals. Current study has
evaluated the accessibility of top six healthcare sites in India against the conformance
of WCAG 2.0 level AA. The appraisal technique used for assessment is the auto-
mated tools. Overall evaluation results differ with respect to different tools. Although,
Achecker finds Operable factor with the largest number of violations whereas robus t
factor with the least number of violations when evaluated by AChecker. However,
evaluation by TAW indicates that the most violated factor is Perceivable while Under-
standable is the least violated factor. The limitation of this study is that only free and
online tools are used for evaluation as commercial tools are not used due to their high
cost. Moreover, automated testing alone is not enough for the complete evaluation
of the websites accessibility so it is recommended to include the user testing and
expert testing with automated testing so that more accurate and consistent results
can be obtained. Future work can be extended by in-depth evaluation of more sites
from this domain as well as from other domains. Further parameters and tools can be
increased for accessibility evaluation under the higher conformance of WCAG 2.1
level AA.
References
1. AChecker: Web accessibility checker. https://achecker.ca/checker/index.php
2. Acosta-Vargas, P., Hidalgo, P., Acosta-Vargas, G., Gonzalez, M., Guaña-Moya, J., Salvador-
Acosta, B.: Challenges and improvements in website accessibility for health services. In: Inter-
national Conference on Intelligent Human Systems Integration, pp. 875–881. Springer (2020)
3. Adepoju, S.A., Ekundayo, A., Ojerinde, A.O., Ahmed, R.: Usability and accessibility evaluation
of Nigerian mobile network operators’ websites. In: 2019 2nd International Conference of the
IEEE Nigeria Computer Chapter (NigeriaComputConf), pp. 1–7. IEEE (2019)
4. Ahmi, A., Mohamad, R.: Evaluating accessibility of Malaysian public universities websites
using Achecker and wave. J. Inf. Commun. Technol. (2016)
5. Akgül, Y.: Accessibility, usability, quality performance, and readability evaluation of university
websites of turkey: a comparative study of state and private universities. Universal Access Inf
Soc. 1–14 (2020)
6. AlMeraj, Z., Boujarwah, F., Alhuwail, D., Qadri, R.: Evaluating the accessibility of higher
education institution websites in the state of Kuwait: empirical evidence. Universal Access
iInf. Soc. 1–18 (2020)
7. Barricelli, B.R., Casiraghi, E., Dattolo, A., Rizzi, A.: 15 years of Stanca act: are Italian public
universities websites accessible? Universal Access Inf. Soc. 1–16 (2020)
8. CTIC: Taw web accessibility test. https://www.tawdisnet/ (2018)
9. Dongaonkar, S., Vadali, R., Dhutadmal, C.: Content accessibility evaluation of government
website using WCAG (web content accessibility guidelines). In: RICE, pp. 27–31 (2017)
10. Gay, G., Li, C.Q.: Achecker: open, interactive, customizable, web accessibility checking. In:
Proceedings of the 2010 International Cross Disciplinary Conference on Web Accessibility
(W4A), pp. 1–2 (2010)
11. Henry, S.: Evaluating web accessibility overview. https:// www.w3.org/WAI/eval/Overview

636 K. Sarita et al.
12. Ismail, A., Kuppusamy, K.: Accessibility of indian universities’ homepages: an exploratory
study. J. King Saud Univ. Comput. Inf. Sci. 30(2), 268–278 (2018)
13. Ismail, A., Kuppusamy, K.: Web accessibility investigation and identification of major issues
of higher education websites with statistical measures: a case study of college websites. J. King
Saud Univ. Comput. Inf. Sci. (2019)
14. Jati, H., Dominic, D.D.: Website accessibility performance evaluation in Malaysia. In: 2008
International Symposium on Information Technology, vol. 1, pp. 1–3. IEEE (2008)
15. Kaur, A., Dani, D., Agrawal, G.: Evaluating the accessibility, usability and security of hospitals
websites: an exploratory study. In: 2017 7th International Conference on Cloud Computing,
Data Science & Engineering-Confluence, pp. 674–680. IEEE (2017)
16. Kaur, S., Gupta, S.K.: Key aspects to evaluate the performance of a commercial website. Int.
J. Comput. Appl. 1(1), 1–5 (2014)
17. Kirkpatrick, A., Connor, J., Campbell, A., Cooper, M.: Web content accessibility guidelines
(WCAG) 2.1 w3c recommendation. World Wide Web Consortium. https://www.w3.org/TR/
WCAG21 (2018)
18. Kumar, K.L., Owston, R.: Evaluating e-learning accessibility by automated and student-
centered methods. Educ. Technol. Res. Dev. 64(2), 263–283 (2016)
19. Llinás, G., Rodríguez-Iñesta, D., Mira, J.J., Lorenzo, S., Aibar, C.: A comparison of websites
from Spanish, American and British hospitals. Methods Inf. Med. 47(02), 124–130 (2008)
20. Nazar, H., Sarfraz, M.S., Shoaib, U.: Web accessibility evaluation of banking website in Pak-
istan. Int. J. Comput. Sci. Inf. Secur. 15(1), 642 (2017)
21. Patsioura, F., Kitsiou, S., Markos, A.I.: Evaluation of Greek public hospital websites. In: ICE-B,
pp. 223–229 (2009)
22. Roy, S., Pattnaik, P.K., Mall, R.: A quantitative approach to evaluate usability of academic
websites based on human perception. Egypt. Inform. J. 15(3), 159–167 (2014)
23. Salarvand, S., Samadbeik, M., Tarrahi, M.J., Salarvand,H.: Quality of public hospitals websites:
a cross-sectional analytical study in Iran. Acta Informatica Medica 24(2), 130 (2016)
24. Tashtoush, Y.M., Ala’F, D., Al-Sarhan, H.N.: The Arabian e-government websites accessibility:
a case study.In: 2016 7th International Conference on Information and Communication Systems
(ICICS), pp. 276–281. IEEE (2016)
25. W3C: WCAG1.0. https://www.w3.org/tr/wcag10/guidelines (1999)
26. W3C: Pour principles. https:// webaim.org/standards/wcag/checklist (2008a)
27. W3C: WCAG 2.0. https://www.w3.org/tr/wcag20/guidelines (2008b)
28. W3C: Web accessibility evaluation tools list. https://www.w3.org/WAI/ER/tools/ (2016)
29. W3C: WCAG 2.1. https://www.w3.org/tr/wcag21/new-features-in-wcag-2-1 (2018)