Web Accessibility Evaluation of Government Websites for People with Disabilities in Turkey

Abstract

The Web is a progressively more important resource in many aspects of life: government, commerce and more. As governments to continue to provide businesses and citizens with new value-added e-services, citizens with disabilities are still being deprived from taking full advantage of these services. While the proportion of people with disabilities (visual impairment, hearing impairment, cognitive disability etc.) in society has been rapidly increasing due to the demographic trends long documented by many researchers, governmental leaders have paid little attention to their needs when planning and implementing Web projects. Therefore, it is essential that all citizens must have equal accessible opportunities to all e-government recourses. This research evaluates the accessibility of each of the 25 e-Government websites in Turkey by people disabilities based on the Web Content Accessibility Guidelines (WCAG) 1.0 and 2.0 and using automated testing tools. The results of study indicate that the prevalent priority-1 accessibility barriers identified in this study were related to the absence of text equivalents for non-text elements, and the failure of the static equivalents for dynamic content to get updated when the dynamic content changes.

Full text

Web Accessibility Evaluation of Government
Websites for People with Disabilities in Turkey
Yakup AKGÜL
Dumlupınar University, Simav Vocational School
Email: yakupakgul@gmail.com
Kemal VATANSEVER
Pamukkale University, Applied Science School
Email: kvatansever@pau.edu.tr
Abstract—The Web is a progressively more important
resource in many aspects of life: government, commerce
and more. As governments to continue to provide businesses
and citizens with new value-added e-services, citizens with
disabilities are still being deprived from taking full
advantage of these services. While the proportion of people
with disabilities (visual impairment, hearing impairment,
cognitive disability etc.) in society has been rapidly
increasing due to the demographic trends long documented
by many researchers, governmental leaders have paid little
attention to their needs when planning and implementing
Web projects. Therefore, it is essential that all citizens must
have equal accessible opportunities to all e-government
recourses. This research evaluates the accessibility of each
of the 25 e-Government websites in Turkey by people
disabilities based on the Web Content Accessibility
Guidelines (WCAG) 1.0 and 2.0 and using automated
testing tools. The results of study indicate that the prevalent
priority-1 accessibility barriers identified in this study were
related to the absence of text equivalents for non-text
elements, and the failure of the static equivalents for
dynamic content to get updated when the dynamic content
changes.
Index Terms—web accessibility, disability, e-government
I. INTRODUCTION
The rapid growth of new Information and
Communication Technologies (ICTs) imposes the
adoption of these technologies in different parts of the
modern life, including the governmental side.
EGovernment can be defined as the process by which the
government can deliver services and information to its
citizens via the internet [1]. Access to government
through Web interfaces has become commonplace in
recent times as a consequence of pervasive use of the
Internet for access to information and services [2]. The
use of information and communication technology (lCT)
has been playing a vital role in the 21st century due to
globalization and the governments of the countries are
Manuscript received September 1, 2014; revised December 21,
2014.
being encouraged to adapting with the coming future.
Turkey has declared the “Vision 2023”, which targets
establishment of a resourceful and modern country by
2023 through effective use of information and
communication technology [3]. The government of
Turkey has realized the importance of ICT to improve the
delivery of information and services to disabled citizens.
And now the changes are being seen in government
initiatives [4]. The websites of all the ministries and
divisions are developed under the technical assistance of
Republic of Turkey E-Transformation Turkey Prime
Ministry State Planning Organization Interoperability
Principles Project Guide and are in working for the last
five years [5].
A considerable number of users of the Web have
various types of disabilities such as vision, hearing,
motor and cognitive impairments [6]. Studies show that
presently most of the government websites are
inaccessible for the impaired users [7]. However, more
than one billion people in the world are disable and this
number is increasing day by day as the population
increases [8]. Turkey has an estimated population of 76
million, out of which about 8, 5 million are disable [9].
The accessibility of these web sites, especially by the
people with disabilities, has not been evaluated to date.
This has motivated us to assess the accessibility of e-
government web sites for people with disabilities using
automatic testing tools for checking of target websites.
The purpose of this study is limited to the accessibility
assessment of the central government websites and to
find out whether the web based public services are
provided in equitable manner to all the citizens.
The rest of the paper is organized in six sections: In
Section 2 presents we review the relevant works. Section
3 presents W3C standards and guidelines. Section 4
describes web accessibility evaluation tools. Section 5
describes the adopted methodology to make the complete
analysis of selected websites of government. Section 6
presents the results and their detailed description. Section
7 presents limitations and future work. Section 8
concludes the paper with recommendation.
201
©2016 Engineering and Technology Publishing
Journal
of Advanced Management Science Vol. 4, No. 3, May 2016
doi: 10.12720/joams.4.3.201-210

II. RELATED WORKS
Many studies have been carried out in the field of Web
accessibility. These studies used different techniques and
different measures for assessing the accessibility of
different Websites, especially the government ones. Also
these studies found that large percentage of Websites
have serious problems in their accessibility. In this
section we briefly mention some work that has been done
in the field of Web accessibility. Definition of
accessibility is “making web content available to all
individuals, regardless of any disabilities or
environmental constraints they experience” [10]. The
provision of physical access to appropriate hardware and
software to enable access to the web; it can mean the
provision of add-on technologies to widen access to the
web, for example through the use of assistive
technologies such as screen reading software, screen
magnification, alternative mouse and keyboard devices,
alternative pointing devices, refreshable Braille displays
and voice input [11], [6].
Abdul Latif and Masrek [12] undertaken with the
purpose of identifying the accessibility of Malaysian e-
government websites based on the World Wide Web
Consortium (W3C). The result of the analysis indicated
that there were no single Malaysian e-government
websites that passed the W3C Priority 1 accessibility
checkpoints. Dominic et al. [13] have used diagnostic to
evaluate the Asian egovernment websites in terms of
technical aspects such as loading time, page rank,
frequency of update, traffic, mark-up validation,
accessibility errors, etc. Baowaly and Bhuiyan [14]
concentrated on mainly two things; firstly, it briefly
examined accessibility guidelines, evaluation methods
and analysis tools. Secondly, it analyzed and evaluated
the web accessibility of e-Government websites of
Bangladesh according to the “W3C Web Content
Accessibility Guidelines”. Baowaly et al. [15] analyzed
and evaluated accessibility of government websites’ in
perspective of developing countries. They taked
Bangladesh as a case study. Bakhsh and Mehmood [16]
evaluated the websites of central government in Pakistan
including all ministries and divisionsusing accessibility
evaluation tools based on World Wide Web Consortium's
(W3C) web accessibility standards. The results showed
that most of the web sites were not developed according
to the accessibility standards for disabled persons.
Kuzma [17] assessed the accessibility of e-Government
websites for 12 developing and developed countries. She
identified serious accessibility issues for the tested e-
Government sites, even for websites belonging to
governments who stated adherence to W3C accessibility
standards and UN legislations. Goodwin et al. [18]
conducted a global web accessibility analysis of e-
Government websites from the United Nations member
states. The study revealed that, with few exceptions,
government websites of developed countries are more
accessible than those of developing countries. The study
also found that e-Government websites that are
recognized as mature and of high quality are more likely
to be accessible. Isa et al. [19] who used several
automated testing tools and identified many usability and
accessibility issues related to Malaysia e-Government
websites. Al-Radaideh et al. [20] evaluated the
accessibility of major E-government Websites in Jordan
by people with disabilities with conformance to Web
Content Accessibility Guidelines (WCAG) 1.0. Results
showed that all tested Websites did not address the issue
of disability-accessibility and they have many Web
accessibility problems. Abu-Doush et al. [21] evaluated a
set of Jordan e-government websites using 20 blind and
visually impaired volunteers and at the same time
conducted a survey on e-government websites developers.
Al Mourad and Kamoun [22] evaluated the accessibility
of each of the 21 Dubai e-Government websites based on
the Web Content Accessibility Guidelines (WCAG) 1.0
and using automated testing tools. The results of research
revealed that many Dubai e-Government sites did not
meet the minimum W 3C accessibility conformance level.
AbuAli et al. [23] evaluated Jordan E-Government
Website from the accessibility perspective. The results
from the evaluation process showed that Jordan E-
Government Website lacks accessibility and needs
further improvements to improve its quality. Luj´an-
Mora et al. [24] analysed the accessibility of a group of
e-government websites of all South American countries
and Spain. The results of research showed that the
majority of e-government websites do not provide
adequate levels of web accessibility.
Some similar studies on accessibility of web sites and
web contents were also conducted by Mankoff et.al [10],
Lazar et.al [6], Venter et.al [25], Choudrie et al. [26], Shi
[27], Potter [28], Abanumy et al. [29], Rowena Cullen
and Caroline Houghton [30], Salon et al. [31], Kuzma
[32], Kuzma et al. [33], Kuzma et al. [34], Hong et al.
[35], Hong et al.[36], Basdekis et al.[37], Kurniawan and
Zaphiris [38], Choi et al.[39], Johnson and Kent [40],
Evans-Cowley [41], Freire et al. [42], Paris [43], Goette
et al. [44], Jaeger [45], Jaeger [46], Shi [47], Rabaiah and
Vandijck [48], Huang [49], Jati and Dominic [50],
Loiacono et al. [51], Mehmood [52], Baguma ve Lubega
[53], Baguma et al. [54] and give suggestions for
improvements.
III. WEB ACCESSIBILITY AND GUIDELINES
Web accessibility can be defined as the degree to
which a site is accessible to the largest possible range of
people. The more people are able to access a website, the
more accessible is the site. At its core, Web accessibility
emphasizes making website accessible to persons with
disabilities and involves removing potential barriers to
access caused by inconsiderate website designs [55].
Web accessibility primarily benefits people with
disabilities. However, as an accessible website is
designed to meet different user needs, preferences, skills
and situations, this flexibility can also benefit people
without disabilities in certain situations, “such as people
using a slow Internet connection, people with temporary
disabilities such as a broken arm, and people with
changing abilities due to aging” [56]. The World Wide
Web Consortium (W3C) is an international organization
202
©2016 Engineering and Technology Publishing
Journal
of Advanced Management Science Vol. 4, No. 3, May 2016

dedicated to the standardization of the World Wide Web
(WC3). In 1996, W3C established the Web Accessibility
Initiative (WAI) campaigning for a more accessible Web
for persons with disabilities. For the consortium, Web
accessibility was defined as “access to the Web by
everyone, regardless of disability” [55].
In 1999, the Web Accessibility Initiative (WAI), a
Project by the World Wide Web Consortium (W3C)
published the Web Content Accessibility Guidelines
(WCAG) version 1.0 [56]. These guidelines were widely
accepted in many countries around the world as the
definitive guidelines on creating accessible websites. The
WAI approach to Web accessibility revolves around
three interrelated fronts: (i) the content accessibility of
websites for persons with disabilities to perceive,
understand, and use; (ii) making Web browsers and
media players usable for persons with disabilities by
making them operable through assistive technologies and
(iii) Web authoring tools and technologies to support
production of accessible Web content and sites, so that
persons with disabilities can use them effectively. An
accessible web site is very similar to an accessible
building. An accessible building offers curb cuts, ramps,
and elevators to allow a person with disabilities to enter
and navigate through the building with ease. Hence, an
accessible web site offers similar functionality [12].
However, on 11 December 2008, the WAI released the
WCAG version 2.0 to be up to date while being more
technology neutral [57].
Currently, there are a number of guidelines and tools
Web designers and webmasters can use to make their
websites accessible to people with disabilities. Such
guidelines include the Web Content Accessibility
guidelines (WCAG) developed by the World Wide Web
Consortium (W3C), the US government’s Section 508
Initiative, Americans with Disabilities Act (ADA),
Australians with Disabilities Act and the National
Institute on Ageing Guidelines (NIA). Similar guidelines
exist in Canada, UK and Portugal. In addition to the
guidelines, automated software tools that help in finding
accessibility flaws in websites before the sites are
publicly posted, are available. Such tools include bobby,
ramp, infocus and a-prompt. More so new versions of
web development tools such as dream weaver and front
page include tools that assist developers with
accessibility related issues [6]. The most common
standards Based website Design and development are
W3C Web Content Accessibility Guidelines 1.0 or 2.0
(WCAG 1.0 or WCAG 2.0). WCAG 2.0 was approved as
an ISO/IEC 40500 International accessibility standard in
October 2012 [58]. The meaning of that more countries
can formally adopt WCAG 2.0 and many countries are
updating their laws to the new version. The W3C Web
Accessibility Initiative (WAI) defines three possible
accessibility conformance levels, as illustrated in Table I.
TABLE I. WAI CONFORMANCE CLAIMS
Conformance Level
Website Accessibility Checkpoint
WAI-A
(basic accessibility)
All priority 1 checkpoints are met. This is the
minimum (basic) W3C requirement. Otherwise
one or more groups of people will find it
impossible to access information from the
website. This is the minimum requirement and
must be met.
WAI-AA
(intermediate
accessibility)
All priority 1 and 2 checkpoints are satisfied;
otherwise one or more groups of people will
find it difficult to access information from the
website. This conformance level status should
be met, as it will remove significant barriers to
accessing Web documents.
WAI-AAA
(high accessibility)
All priority 1, 2 and 3 checkpoints are
satisfied; otherwise one or more groups of
people will find it somehow difficult to access
information from the website. This
conformance level status may be addressed by
Web developers to improve access to Website
documents.
IV. WEB ACCESSIBILITY EVALUATION TOOLS
After Web accessibility evaluation tools are software
programs or online services that are used to check your
website's accessibility level under web accessibility
guidelines. There is a huge number of accessibility tools
for commercial purposes or freely available on the web
such as Watch Fire Bobby, AChecker, Cynthia Says,
EvalAccess, Accessibility Valet Demonstrator
(WebThing), AccMonitor Online (HiSoftware),
Torquemada (WebxTutti), Wave 3.5 (WebAIM) and
Tawdis etc. Some good free web-based website
accessibility evaluation tools are linked in [59], [60], [61],
[62]. A complete list of accessibility evaluation tools is in
W3C [63]. These tools are very useful for programmers
and designers to determine whether or not their sites
follow WCAG. During the design, implementation, and
maintenance phases of Web development if these tools
are used carefully, it can help the targeted users in
preventing accessibility barriers, repairing encountered
barriers, and improving the overall quality of Web sites
[64].
This study will use Automatic evaluation tools such as
AChecker, eXaminator, TAW, Total Validator, WAVE,
Web AccessibilityAssessment Tool, EvalAccess 2.0,
Cynthia Says, MAGENTA, HERA, Amp and Sort Site
which is considered as the web accessibility test tool
which able to provide relatively complete analysis of
website accessibility and have been the pioneers and are
the most well-known, due to their usability, ease of use
and its quick results.
V. METHODOLOGY
In this study, the 25 the official website of the
governments have been analysed. The home page of each
one of the websites has been analysed from three points
of view: HTML and CSS validity; web accessibility; and,
203
©2016 Engineering and Technology Publishing
Journal
of Advanced Management Science Vol. 4, No. 3, May 2016

current use of HTML5 and ARIA. The home page of a
website is the first contact a user has with the website. If
the home page shows problems or is not accessible, it
would be very difficult that a disabled user can access
other pages of the website. Therefore, it is essential to
ensure the accessibility of the home page of a website.
All the tests of a web page were conducted during the
same day in order to avoid changes in its content.
A. HTML and CSS Validity
Two automatic evaluation tools have been used to
evaluate the validity of the HTML and CSS of the
websites. The first automatic tool is the Markup
Validation Service, a free service by the W3C [65]. This
automatic tool checks the markup validity of web pages
in HTML, XHTML, SMIL, MathML, etc. According to
the W3C [66], “Validating web documents is an
important step which can dramatically help improving
and ensuring their quality, and it can save a lot of time
and money”. The result of the Markup Validation Service
is summarized in the number of errors and warnings in a
web page. The second tool is the CSS Validator Service,
another free service by the W3C [67]. Not only, this tool
evaluates the style sheets of a web page its conformance
with W3C open standards and the CSS specifications. It
can also detect when CSS poses some risks in terms of
usability. It can find errors, typos, or incorrect uses of
CSS.
B. Web Accessibility
Thirteen automatic evaluation tools have been used to
evaluate the accessibility of the websites analysed in this
study: AChecker, eXaminator, TAW, Total Validator,
WAVE, Web AccessibilityAssessment Tool, EvalAccess,
Cynthia Says, MAGENTA, HERA, Amp and Sort Site
AChecker [68] is an online free service that produces a
report of accessibility problems according to different
guidelines (Section 508, WCAG 1.0 and WCAG 2.0).
AChecker classifies accessibility problems into three
categories: known problems (problems that have been
identified with certainty as accessibility barriers), likely
problems (problems that have been identified as probable
barriers, but require a human to make a decision) and
potential problems (problems that AChecker cannot
identify, that require a human decision). AChecker also
provides an Application Programming Interface (API)
that allows remote accessibility analysis. eXaminator is
an online free service to check the accessibility of a web
page developed by Carlos Benav´ıdez [69]. eXaminator
checks the application of the WCAG 2.0 [57] on the
HTML and CSS contents in a web page and summarizes
the results in an overall score from 1 to 10 that is quite
easy to understand by everybody. Of course, the score
calculated by eXaminator is a fast check of accessibility,
but automatic evaluation does not cover all of the success
criteria in WCAG 2.0. TAW is a limited online free
service to check the web accessibility against WCAG 1.0
and 2.0 [70]. TAW classifies accessibility problems into
automatic problems, those in which the tool is certain
that the problem violates the guidelines and manual
problems, those that need to be reviewed by an expert.
Total Validator is an HTML validator, an accessibility
validator, a spell checker, and a broken links checker, all
included into one tool [71]. This tool is provided in two
versions: the basic tool for free and the professional tool
that must be purchased. Finally, WAVE is an online
automatic evaluation tool that helps web developers to
make their web content more accessible [72]. However,
WAVE cannot completely state if a web page is
accessible, only a human can determine true accessibility.
WAVE detects HTML5 and Accessible Rich Internet
Applications (ARIA) features, such as header, footer,
ARIA landmarks and roles, and so on. Besides, WAVE
also provides an API that allows automated and remote
accessibility analysis of web pages using the WAVE
processing engine. Web Accessibility Assessment Tool is
a Java application developed by the EU FP7
ACCESSIBLE project [73]. It evaluates a website
according to WCAG 2.0 (level A, level AA, and level
AAA). Providing an option for the users to select among
the success criteria they want to check. Another option is
saving a report on the user computer as a PDF file. This
tool can evaluate more than one page; users can define
the number of pages for evaluation. The results of the
evaluation are categorized into: errors and warnings.
EvalAccess is being developed by the Laboratory of HCI
for Special Needs at the University of the Basque
Country (UPV-EHU). EvalAccess web service checks
web pages accessibility, based on the WAI's WCAG 1.0
guidelines. It has been implemented as a web service to
allow any other application to use it [74]. Cynthia Says
tests your page against predefined checkpoint groups to
validate it against the US Access Board’s Section 508 or
the W3C’s WCAG 2.0 A-AAA Accessibility Guidelines
[75]. M.A.G.EN.T.A. 2.0 (Multi-Analysis of Guidelines
by an Enhanced Tool for Accessibility) is a system to
evaluate accessibility of Web sites by checking their
HTML and CSS code through guidelines, which are to be
specified through an XML-compliant specification
language called G.A.L. (Guideline Abstract Language)
that maintains the guidelines separated from the
underlying logic. M.A.G.EN.T.A. 2. 0 is able to validate
the accessbility of web pages in relation to the following
guidelines: WCAG 2.0 (Level A, AA, AAA), Stanca Act,
Visually Impaired [76]. HERA is a tool to check the
accessibility of Web pages accoridng to the specification
Web Content Accessibility Guidelines (WCAG 1.0).
HERA performs a preliminary set of tests on the page
and identifies any automatically detectable errors or
checkpoints met, and which checkpoints need further
manual verification [77]. The Accessibility Management
Platform (AMP) provides the infrastructure to facilitate
all aspects of a successful accessibility compliance
program. AMP’s powerful testing engine and work flow,
reporting support, accessible development best practices,
and extensive training course library allow organizations
to quickly and efficiently incorporate accessibility
compliance into existing development processes. This
ensures that organizations have the infrastructure to
rapidly conform to Section 508, the Web Content
204
©2016 Engineering and Technology Publishing
Journal
of Advanced Management Science Vol. 4, No. 3, May 2016

Accessibility Guidelines (WCAG), the Americans with
Disabilities Act (ADA), and other leading accessibility
requirements [78].SortSite checks sites against the W3C
WCAG 1.0 and 2.0 accessibility standards, and
compliance with Section 508 of the Rehabilitation Act
[79].
C. Current Use of HTML5 and ARIA
HTML5 is the latest standard and which is an example
of modern technology. The first draft of HTML5, was
published by the W3C in January 2008 [80], HTML5 is
expected to be completed and published at the end of
2014. Unfortunately, six years later the use of the new
version of the markup language of the Web is not very
common yet. HTML5 updates the specification to
include latest advances and best practices in web
development. HTML5 also includes new accessibility
features that will improve the accessibility of websites.
ARIA, another standard of the W3C, addresses the lack
of accessibility of many web pages. Complex web
applications become inaccessible when assistive
technologies cannot determine the semantics behind
portions of a document or when the user is unable to
effectively navigate to all parts of it in a usable
way. WAI-ARIA divides the semantics into roles (the
type defining a user interface element)
and states and properties supported by the roles. ARIA
defines ways to make Web content and Web applications
(especially those developed with Ajax and JavaScript)
more accessible to people with disabilities. It especially
helps with dynamic content and advanced user interface
controls developed with Ajax, HTML, JavaScript, and
related technologies. ARIA enhances accessibility of
interactive controls (such as tree menus, drag and drop,
sliders, sort controls, etc.), provides content roles for
identifying page structure (navigation, search, main
content, etc.), areas that can be dynamically updated
(called "live regions" in ARIA), better support for
keyboard accessibility and interactivity, and much more.
ARIA is a set of special accessibility attributes which can
be added to any markup, but is especially suited to
HTML. The role attribute defines what the general type
of object is (such as an article, alert, or slider). ARIA is
supported by most up-to-date browsers and screen
readers. It is also supported by many scripting libraries.
Usage of HTML 5 and ARIA together make web content
and web applications more accessible to people with
disabilities [81], [82].
VI. RESULTS
A. HTML and CSS Validity
Fig 1 shows the HTML and CSS validity results.
Acolour code is used to clarify the results. An anomalous
situation detected during the analysis: one website could
not be analysed, the website of the Prime Ministry of
Turkey. Only the website of the Republic of Turkey
Ministry of Justice website had 0 validation errors. The
following websites with the less number of HTML errors
were: Republic of Turkey Ministry of Family and Social
Policy with 2 errors, Republic of turkey ministry of
defense with 4 errors. The worst results were obtained
with the website of the Republic of Turkey Ministry of
Youth and Sports, with 525 errors, and the Republic of
Turkey Ministry of Economy, with 306 errors. Regarding
the CSS validation, thebest results were the website of
the Presidency of the Republic of Turkey, Republic of
Turkey Ministry of Justice, Republic of turkey ministry
of defense, Republic of Turkey Ministry of Interior,
Republic of Turkey Ministry of Transport, Maritime
Affairs and Communications with 0 errors. On the
opposite side, Republic of Turkey ministry of
development web site presented the highest number of
errors with 353.
B. Web Accessibility
Due to the lack of space, we cannot include the whole
results of the web accessibility analysis. Therefore, Fig. 2
summarizes the number of problems detected with
automatic evaluation tools and some information has to
be discarded. Unfortunately, the home pages of all the
websites have accessibility issues. In Fig. 2, column
“AChecker” represents the number of “known problems”
that have been detected. According to AChecker, these
problems should be fixed. “Likely” and “potential errors”
have not been included in the figure. Column
“eXaminator” shows the global score provided by this
tool, a value from 1 to 10: the higher the value, the better
the accessibility of the web page. Column “TAW 1.0 P1”
indicates the number of issues to pass the WCAG 1.0
priority 1 (A level) requirement that can be automatically
detected. The manual errors have been discarded because
they required additional human intervention. Column
“TAW 2.0 Problems” provides the number of problems
that should be corrected because there is a certainty about
them. “Warnings” and “Not verified problems” have also
been discarded and they are not showed in the figure.
Column “TV” Errors WCAG 2.0 A” shows the number
of errors of WCAG 2.0 priority 1 (A level) detected by
Total Validator. The other errors have been discarded.
Column “WAVE Errors” provides the number of errors
detected by WAVE. “Alerts” have also been discarded.
Column “Web Accessibility Assessment Tool” shows the
number of errors of WCAG 2.0 priority 1 (A level)
requirement that can be automatically detected.
“Warnings” have been discarded. Column “EvalAccess
2.0” indicates the number of errors to pass the WCAG
1.0 priority 1 requirement that can be automatically
detected. “Warnings” have not been included in the
figure. Column “Cynthia Says” provides the number of
failures of WCAG 2.0 priority 1 (A level) requirement
that can be automatically detected. “Warnings” have been
discarded. Column “MAGENTA” shows the number of
errors of WCAG 2.0 priority 1 (A level) requirement that
can be automatically detected. Requirement that can be
automatically detected. Column “HERA” indicates the
number of errors to pass the WCAG 1.0 priority 1
requirement that can be automatically detected. Column
“Amp” provides the global percentage provided by this
tool, a value from %1 to %100: the higher the value, the
205
©2016 Engineering and Technology Publishing
Journal
of Advanced Management Science Vol. 4, No. 3, May 2016

better the accessibility of the web page. Finally, Column
“Sort Site” shows the number of issues WCAG 2.0
priority 1 requirement that can be automatically detected.
In general, the worst results regarding web
accessibility were obtained with the websites of Republic
of Turkey Ministry of Culture and Tourism, Republic of
Turkey Ministry of Economy and Republic of Turkey
Ministry of Youth and Sports. On the other side, the best
results were obtained with the websites of Turkish
Armed Forces/Turkish General Staff, Republic of Turkey
Ministry of Justice and Republic of turkey ministry of
defense.
Website
HTML
Errors
HTML
Warns
CSS
Errors
CSS
Warns
Presidency of the
Republic of Turkey
55
49
0
0
The Grand
National Assembly
of Turkey
97
26
17
64
Prime Ministry of
Turkey
Turkish Armed
Forces/Turkish
General Staff
97
16
16 0
Ministry of Justice
0
0
0 1
Ministry of
Defense
4
4
0 0
Ministry of Interior
20
28
0 33
Ministry of Foreign
Affairs
33
8
79 7
Ministry of
Finance
297
193
57 720
Ministry of
National Education
134
75
51 58
Ministry of
Environment and
Planning
36
12
11 46
Ministry of Health
39
0
5
0
Transport,
Maritime Affairs
and Comm.
18
0
0
0
Ministry of Food,
Agriculture and
Livestock
165
5
64
740
Ministry of labor
and social security
212
72
95
55
Ministry of
Science, Industry
and Technology
212
171
10
41
Ministry of Energy
and Natural
Resources
283
424
2
4
Ministry of Culture
and Tourism
14
3
54
13
Ministry of forest
and water
13
2
16
5
Ministry of Family
and Social Policy
2
2
4
18
Ministry for EU
Affairs
76
112
2
0
Ministry of
Economy
306
173
123
49
Ministry of Youth
and Sports
525
0
130
89
Ministry of
Customs and Trade
81
2
33
57
Ministry of
Development
13
1
353
973
Figure 1 . HTML and CSS validation results
Website
AChecker
eXaminato
r
TAW 1.0
TAW 2.0
TV
WAVE
Web
A.A.T.
EvalAccess
2.0
Cynthia
Says
MAGENT
A
HERA
Amp
Sort Site
Presidency of the
Republic of Turkey
27
5.7
18
57
28
9
36
23
3
31
1
%59
18
The Grand National
Assembly of Turkey
38
4.6
20
87
59
27
102
20
3
1
2
%58
17
Prime Ministry of Turkey
15
4.6
15
22
10
8
26
3
1
3
%50
13
Turkish Armed
Forces/Turkish General
Staff
26
2.9
0
26
59
21
22
18
8
46
1
%66
20
Ministry of Justice
2
3.8
6
12
8
1
18
0
27
9
0
%70
12
Ministry of Defense
3
4.7
0
5
2
3
0
0
9
1
0
%68
2
Ministry of Interior
3
4.3
1
24
5
3
6
1
4
19
1
%59
8
Ministry of Foreign
Affairs
16
5.1
1
28
7
3
27
1
4
29
2
%57
13
Ministry of Finance
85
4.5
73
8
153
3
73
10
143
12
%52
15
Ministry of National
Education
54
4.7
49
124
89
61
177
49
3
78
1
%59
16
Ministry of Environment
and Planning
13
4.6
0
47
29
8
18
4
4
12
1
%65
10
Ministry of Health
30
3.4
27
8
66
33
111
27
11
69
1
12
Transport,Maritime
Affairs and Comm.
2
6.0
1
18
1
1
24
0
16
2
0
%71
17
Ministry of Food,
Agriculture and Livestock
40
4.1
33
47
52
34
31
14
54
3
%59
17
Ministry of labor and
social security
60
3.9
56
164
129
60
125
54
6
111
3
%69
14
Ministry of Science,
50
3.8
55
110
77
44
37
10
101
3
%48
206
©2016 Engineering and Technology Publishing
Journal
of Advanced Management Science Vol. 4, No. 3, May 2016

Industry and Technology
Ministry of Energy and
Natural Resources
16
3.1
0
41
54
14
53
10
22
59
3
%53
16
Ministry of Culture and
Tourism
10
3.9
0
46
29
6
829
0
7
28
0
%63
14
Ministry of forest and
water
25
5.1
11
37
36
23
81
11
7
37
1
%59
11
Ministry of Family and
Social Policy
5
5.8
0
12
26
2
68
0
6
26
1
%58
6
Ministry for EU Affairs
25
4.5
6
35
27
16
27
16
6
27
1
%58
17
Ministry of Economy
64
4.1
51
227
95
58
51
9
159
1
17
Ministry of Youth and
Sports
63
4.7
50
213
160
60
41
9
169
3
%57
17
Ministry of Customs and
Trade
73
3.9
67
113
118
69
156
67
8
210
1
%59
9
Ministry of Development
34
4.9
0
128
115
30
142
28
4
123
3
%53
14
Figure 2. Accessibility results
C. Current Use of HTML5 and ARIA
The DOCTYPE is a declaration that always has to
appear at the very top of HTML documents. This
declaration defines the type of document, tells the
browser what element to expect as the top-level element,
and identifies the version of the type of document.
According to the results of W3C’s Markup Validation
Service [65], only 4 web pages (16%) have the HTML5
DOCTYPE: Republic of Turkey Ministry of
Environment and Planning, Republic of Turkey Ministry
of Family and Social Policy, Republic of Turkey ministry
of customs and trade, Republic of Turkey ministry of
development.
Regarding the use of ARIA, WAVE [72] has been
used to detect ARIA features in the analysed websites.
Only 5 web sites (20%) present some use of ARIA:
Republic of Turkey Ministry of Family and Social Policy,
Republic of Turkey Ministry of Food, Agriculture and
Livestock, Republic of Turkey Ministry of Foreign
Affairs and Prime Ministry of Turkey. For example,
Republic of Turkey Ministry of Foreign Affairs website
makes use of: one header (header), one footer (footer),
one navigation sections (nav)
VII. LIMITATIONS AND FUTURE WORK
Two main limitations have been found in this study.
The first limitation is related to the exclusive reliance of
our accessibility analysis on automated testing results.
Web accessibility evaluation tools and expert inspections
cannot substitute user testing, because the difficulties of
understanding all the interactions between web content
and assistive technology. Automatic tools generally
verify the presence of a valid element or attribute, such as
the alt attribute (alternative text) or the label element
(description of a form control). However, human
judgment is also needed, because some questions are
very relevant, such as whether or not the value of the alt
attribute clearly and effectively conveys the function of
the image. For example, there is a big difference between
the alternative text that an active or inactive image needs.
Indeed, in some cases an image may not need an
alternative text (null alt text). Vigo et al. [83] tested and
compared the capabilities of six automatic current web
accessibility evaluation tools, by analysing their coverage,
completeness and correctness with regard to WCAG 2.0
conformance. The conclusion was that relying on only
one automatic evaluation tool was an error because none
of the analysed tools obtained the best scores in all the
dimensions studied. For example, some tools exhibited
high completeness scores and low correctness scores at
the same time. Therefore, a web accessibility analysis
based only on automatic evaluation tools should include
the results of different tools in order to achieve reliable
results.
Another limitation is the restriction of our automated
accessibility testing on the home page of each tested
website. In order to achieve a more accurate view of the
accessibility of each website, this study is going to be
extended to study hundreds or thousands of web pages in
each website to have a more precise view of the
accessibility. We also note that the accessibility metric,
derived from an automatic accessibility evaluation
approach, is a proxy indicator of Website accessibility
and not a real assessment of accessibility as experienced
by a person with disability. Therefore, our results may
not capture all the accessibility issues that disabled
individuals might encounter in real-life. However, they
do pinpoint to some major accessibility issues that need
to be resolved.
Throughout the whole investigation to determine the
conformance level of accessibility, the researchers
adopted the various evaluation tools (AChecker,
eXaminator, TAW, Total Validator, WAVE, etc.), all of
them were open source applications. However they are
widely used and to ensure the scalability of the result we
followed W3C Evaluating Accessibility [64]. Although
the commercial tools (e.g. Bobby) are not freely available
and expensive, we will try to apply both commercial
evaluation tools and also open source and commercial
assistive Technologies (NVDA, JAWS, etc) them in our
next study. In addition to, in order to obtain more
conclusive results, we plan to compare the results across
countries and across different government websites.
207
©2016 Engineering and Technology Publishing
Journal
of Advanced Management Science Vol. 4, No. 3, May 2016

Finally, another future work we plan to address is to
detect the most common problems that recur in the same
site and between different sites.
VIII. CONCLUSION AND RECOMMENDATIONS
In this paper we addressed Turkish e-government
websites accessibility for people with disabilities. 25
websites were tested using automatic tools for checking
of websites. Results showed that all Turkish e-
government websites do not address the issue of
disability-accessibility, and it is clear that the vast
majority of Turkish government websites do not meet
minimum levels of web accessibility requirements. In our
study it is difficult to obtain conclusive results because
each automatic evaluation tool detects different types of
errors. Because of this, it is difficult to say which one of
the analysed websites presents the best and the worst
level of web accessibility.
The most common detected accessibility issues were
related to the absence of text equivalents for non-text
elements and the failure of the static equivalents for
dynamic content to get updated when the dynamic
content changes.
As a recommendation, websites designers are
encouraged to consider the w3c guidelines because of the
increasing number of people with disabilities and in order
to give them their right in accessing websites information
equally with other. As a future work, different tools
might be used to check governmental websites to see
whether any differences in the accessibility degree will
be captured.
Based on the work described in this paper, the authors
would like to recommend the following issues as critical
initial steps forwards: Government should either adapt
the existing web accessibility guidelines or develop its
own guidelines that are appropriate for their context.
Also, government should set a policy for web
accessibility together with an enforcement procedure e.g.
making the accessibility of government websites a
compulsory requirement. An incentive or reward for
those who accommodate website accessibility may
promote good web accessibility.
Finally, organizations caring for disabled people have
a responsibility to spread the awareness amongst
government organizations for making e-Government
websites accessible. The successful implementation of e-
Government websiteaccessibility would enable disabled
peoples to get involved directly in the community thus
making it better for all.
REFERENCES
[1] http://www.lao.ca.gov/2001/012401_egovernment.html
24.07.2014
[2] M. R. Middleton, “Approaches to evaluation of websites for
public sector services ,” in Kommers, Piet, Eds. in Proc. IADIS
Conference on e-Society, Lisbon, Portugal, 2007, pp. 279-284.
[3] TÜRKİYE BİLİMSEL ve TEKNOLOJİK ARAŞTIRMA
KURUMU. [Online]. Available:
http://www.tubitak.gov.tr/tr/kurumsal/politikalar/icerik-vizyon-
2023#bit 24.07.2014
[4] http://kakis.gov.tr/erisilebilirlik 24.07.2014
[5] http://www.kakis.gov.tr/rehberi-indir 24.07.2014
[6] J. Lazar, A. Dudley-Sponaugle, and K. Greenidge, “Improving
web accessibility: A study of webmaster perceptions”, Journal of
Computers in Human Behavior, 20, pp.269–288, 2004.
[7] R. Baguma, T. Wanyama, P. V. Bommel, and P. Ogao, “Web
Accessibility in Uganda: A study of Webmaster perceptions”, In
Proc. 3rd Annual International Conference on Computing & ICT
Research (SREC'07), 2007, pp. 183-197.
[8] United Nations Enable. [Online]. Available:
http://www.un.org/disabilities/default.asp?id=18 24.07.2014.
[9] http://www.bianet.org/bianet/toplum/26969-turkiyede-ozurlu-
orani-yuzde-12-29 24.07.2014.
[10] J. Mankoff, H. Fait, and T. Tran, “Is your web page accessible? A
comparative study of methods for assessing web page
accessibility for the blind,” in Proc. SIGCHI Conference on
Human Factors in Computing Systems (CHI’05), 2005, pp. 41-50.
[11] J. Craven and A. Nietzio, “A task-based approach to assessing the
accessibility of web sites,” Performance Measurement and
Metrics, vol. 8, no. 2, pp. 98–109, 2007.
[12] M. A. Latif and M. N. Masrek, “Accessibility evaluation on
Malaysian E-government website,” Journal of e-Government
Studies and Best Practices, pp. 1-11, 2010.
[13] P. D. D. Dominic, H. Jati, P. Sellappan, and G. K. Nee, “A
comparison of Asian e-government websites quality: Using a non-
parametric test,” International Journal of Business Information
Systems, vol. 7, no. 2, pp. 220-245, 2011.
[14] M. K.Baowaly and M. Bhuiyan, “Accessibility analysis and
evaluation of Bangladesh government websites,” in Proc.
International Conference on Informatics, Electronics & Vision,
18-19 May 2012, pp. 46–51.
[15] M. K. Baowaly, J. Hossain Md., and M. Bhuiyan, “Accessibility
analysis and evaluation of government-websites’ in developing
countries: Case study Bangladesh,” Computer Engineering and
Intelligent Systems, vol. 3, no. 4, pp. 1-9, 2012.
[16] M. Bakhsh and A. Mehmood, “Web accessibility for disabled: A
case study of government websites in Pakistan,” in Proc. 2012
10th International Conference on Frontiers of Information
Technology (FIT), 17-19 Dec. 2012, pp. 342–347.
[17] J. Kuzma, “Global E-government web accessibility: “A case
study,” in Proc. British Academy of Management 2010
Conference, University of Sheffield, UK., 14-16, September 2010.
[18] M. Goodwin, D. Susar, A. Nietzio, M. Snaprud, and C. S. Jensen,
“Global web accessibility analysis of national government portals
and ministry web sites,” Journal of Information Technology and
Politics, vol. 8, no. 1, pp. 4167, 2011.
[19] W. Isa, M. Suhami, N. Safie, and S. Semsudin, “Assessing the
usability and accessibility of Malayisa E government website,”
American Journal of Economics and Business Administration, vol.
3, no. 1, pp. 40-46, 2011.
[20] M. S. M. Al-radaideh and A. Wahbeh, “Evaluating accessibility of
Jordanian E-government websites for people with disabilities,” in
Proc. International Conference on Information and
Communication Systems, Irbid, Jordan. May 22-24 2011.
[21] I. Abu-Doush, E. A. Ashraf Bany-Mohammed, and M. A. Al-
Betar, “Towards a more accessible e-government in Jordan: An
evaluation study of visually impaired users and Web developers,”
Behaviour and Information Technology, vol. 32, no. 3, pp. 273-
293, 2013.
[22] B. Al Mourad and F. Kamoun, “Accessibility evaluation of Dubai
e-government websites: Findings and implications,” Journal of E-
Government Studies and Best Practices, vol. 2013, pp. 1-15, 2013.
[23] A. N. Abu Ali, A. Obedidat, and H. Y. Abu-Addose,
“Accessibility as an indicator of Jordanian E-government website
quality,” in Proc. Fourth International Conference on e-Learning
Best Practices in Management, Design and Development of e-
Courses: Standards of Excellence and Creativity, 7-9 May 2013,
pp. 156-160.
[24] S. Luj´an-Mora, R. Navarrete, and M. Penafiel, “Egovernment
and web accessibility in SouthAmerica, ” in Proc. 2014 First
International Conference on eDemocracy and eGovernment
(ICEDEG), 24-25 April 2014, pp. 77–82.
[25] S. Venter and H. Lotriet “Accessibility of South African websites
to visually disabled users,” South African Journal of Information
Management, vol. 7, no. 2, June 2005.
[26] J. Choudrie, G. Ghinea, and V. Weerakkody, “Evaluating global
e-government sites: A view using web diagnostic tools,”
208
©2016 Engineering and Technology Publishing
Journal
of Advanced Management Science Vol. 4, No. 3, May 2016

Electronic Journal of e-government, vol. 2, no. 2, pp. 105-114,
2004.
[27] S. Yuquan, “The accessibility of Chinese local government
Websites: An exploratory study,” Government Information
Quarterly, vol. 24, pp. 377–403, 2007.
[28] A. Potter, “Accessibility of Alabama government websites,”
Journal of Government Information, pp. 303–317, 2002.
[29] A. M. Abanumy, A. Al-Badi, and P. Mayhew, “E-government
website accessibility: In-depth evaluation of Saudi Arabia and
Oman,” The Electronic Journal of e-Government, vol. 3, no. 3, pp.
99-106, 2005.
[30] R. Cullen and C. Houghton, “Democracy online: An assessment
of New Zealand government Websites,” Government Information
Quarterly, vol. 17, no. 3, pp. 243-267, 2000.
[31] L. Gibson, D. Salon, P. Gregor, and P. Booth, “Auditing
accessibility of UK higher education Website,” Interacting with
Computer, pp. 12313-12325, 2002.
[32] J. Kuzma, “Regulatory compliance and web accessibility of UK
parliament sites,” Journal of Information Law & Technology, vol.
2, pp. 1-15, 2009.
[33] J. Kuzma, G. Weisenborn, T. Philippe, A. Gabel, and R. Dolechek,
“Analysis of U.S senate web sites for disability accessibility,”
International Journal of Business Research, vol. 9, no. 6, pp. 174-
181, 2009.
[34] J. Kuzma, Y. Dorothy, and K.Oestreicher, “Global e-government
web accessibility: An empirical examination of EU, Asian and
African sites,” Paper presented at the ICTA’09, Hammamet,
Tunisia, 2009.
[35] S. Hong, P. Katerattanakul, and D. Lee, “Evaluating government
website accessibility: Software tool vs human experts,”
Management Research News, vol. 31, no. 1, pp. 27-40, 2007.
[36] S. Hong, P. Katerattanakul, and S. J. Joo, “Evaluating government
website accessibility: A comparative study,” International Journal
of Information Technology & Decision Making, vol. 7, no. 3, pp.
491–515, 2008.
[37] I. Basdekis, L. Klironomos, I. Metaxas, and C. Stephanidis, “An
overview of web accessibility in Greece: A comparative study
2004–2008,” Universal Access in the Information Society, vol. 9,
no. 2, pp. 185-190, 2009.
[38] S. H. Kurniawan and P. Zaphiris, “Usability and accessibility
comparison of governmental, organizational, educational and
commercial aging/health-related websites,” in Proc. the 9th
International Conference on Human Computer Interaction, New
Orleans, LA: Abridged, 2001, pp. 34-36.
[39] S. Choi, S. Kim, and S. Kim “Korean web site usability for
disabled people,” Lecture Notes in Computer Science, vol.
5068, pp. 405-412, 2008.
[40] R. Johnson and S. Kent, “Designing universal access: Web-
applications for the elderly and disabled. Cognition,” Technology
and Work, vol. 9, no. 4, pp. 209–218, 2007.
[41] J. Evans-Cowley, “The accessibility of municipal government
websites,” Journal of E-Government, vol. 2, no. 2, pp. 75-90,
2006.
[42] A. P. Freire, C. M. Russo, and R. P. M. Fortes, “The perception of
accessibility in web development by academy, industry and
government: A survey of the Brazilian scenario,” New Review of
Hypermedia and Multimedia, vol. 14, no. 2, pp. 149–175, 2008.
[43] M. Paris, “Website accessibility: A survey of locale-government
websites and legislation in Northern Ireland,” Universal Access in
the Information Society, vol. 4, pp. 292–299, 2006.
[44] T. Goette, C. Collier, and J. D. White, “An exploratory study of
the accessibility of state government websites,” Universal Access
in the Information Society, vol. 5, pp. 41–50, 2006.
[45] P. T. Jaeger, “Assessing section 508 compliance on federale-
government websites: A multi-method, user-centered evaluation
of accessibility for persons with disabilities,” Government
Information Quarterly, vol. 23, pp. 169–190, 2006.
[46] P. Jaeger, “User-centered policy evaluations of section 508 of the
rehabilitation act,” Journal of Disability Policy Studies, vol. 19,
no. 1, pp. 24-33, June 2008.
[47] Y. Shi, “E-government web site accessibility in Australia and
China: A longitudinal study,” Social Science Computer Review,
vol. 24, no. 3, pp. 378–385, 2006.
[48] A. Rabaiah and E. Vandijck, “A strategic framework of e-
government: generic and best practice,” Electronic Journal of e-
Government, vol. 7, no. 3, pp. 241–258, 2009.
[49] C. J. Huang, “Usability of E-government web-sites for people
with disabilities,” in Proc. 36th Annual Hawaii International
Conference on System Sciences, 6-9 Jan. 2003.
[50] H. Jati and D. D. Dominic, “Website accessibility performance
evaluation in Malaysia,” in Proc. International Symposium
on Information Technology, 2008. IT Sim 2008. (Volume:1 ), 26-
28 Aug. 2008, pp. 1–3.
[51] E. Loiacono, and S. McCoy, “Web site accessibility: An online
sector analysis,” Information Technology and People, vol. 17, pp.
87-101, 2004.
[52] M. A. Awan, “E-government: Assessment of GCC (Gulf
Cooperating Council) countries and services provided,” Lecture
Notes in Computer Science, vol. 2739, 2003.
[53] R. Baguma and J. T. Lubega, “A web design framework for
improved accessibility for people with disabilities (WDFAD),” Y.
Yesilada & D. Sloan (eds.), W4A (p./pp. 134-140), ACM.
[54] R. Baguma, P. Bommel, T. Wanyama, and P. Ogao, “Web
accessibility in Uganda: A study of webmaster perceptions,”
Kizza, M. J., Muheirwe, J., Aisbett, J., Gitao, K., Mbarika, V. W.,
Patel, D. and Rodrigues, A.T., eds. Strengthening the Role of ICT
in Development, vol. III, pp. 183-197.
[55] X. Zeng, “Evaluation and enhancement of web content
accessibility for persons with disabilities,” Ph.D. Thesis.
University of Pittsburgh, 2004.
[56] World Wide Web Consortium (W3C). Web Content Accessibility
Guidelines 1.0. 1999. Internet. [Online]. Available:
http://www.w3.org/TR/WCAG10/
[57] World Wide Web Consortium (W3C). Web Content Accessibility
Guidelines (WCAG) 2.0. 2008. Internet. [Online]. Available:
http://www.w3.org/TR/WCAG/
[58] International Standard Organization. ISO/IEC 40500:2012
Information technology–W3C Web Content Accessibility
Guidelines (WCAG) 2.0. 2012.
[59] (2014, August). Free Web-Based Web Site Accessibility
Evaluation Tools. [Online]. Available: http://usabilitygeek.com/ 1
0-free-web-based-web-siteaccessibility-evaluation-tools/
[60] Recommended Web Accessibility Validators. [Online]. Available:
http://www.indiana.edu/~iuadapts/webaccessibility/access
validators.html
[61] 100 Killer Web Accessibility Resources: Blogs, Forums and
Tutorials. [Online]. Available:
http://whdb.com/blog/2008/100-killer-web-accessibility-
resources-blogs-forums-and-tutorials/
[62] Accessibility Testing Tools and Techniques. [Online]. Available:
http://www.3pillarglobal.com/insights/accessibility-
testing-tools-and-techniques
[63] Complete List of Web Accessibility Evaluation Tools. [Online].
Available: http: //www.w3.org/WAI/RC/tools/complete
[64] Evaluating Accessibility. [Online]. Available: http://www.w3.
org/WAIIeval/Overview.html
[65] World Wide Web Consortium (W3C). The W3C Markup
Validation Service. [Online]. Available: http://validator.w3.org/
[66] World Wide Web Consortium (W3C). About The W3C Markup
Validation Service. [Online]. Available:
http://validator.w3.org/about.html
[67] World Wide Web Consortium (W3C). The CSS Validation
Service.[Online]. Available: http://jigsaw.w3.org/css-validator/
[68] Inclusive Design Institute. AChecker. [Online]. Available:
Internet: http://achecker.ca/
[69] C. Benav´ıdez. eXaminator. [Online]. Available:
http://examinator.ws/
[70] Fundaci´on CTIC. TAW. [Online]. Available:
http://www.tawdis.net/
[71] Total Validator. Total Validator. [Online]. Available:
http://www.totalvalidator.com/
[72] WebAIM, WAVE. [Online]. Available: http://wave.webaim.org/
[73] Chalkia Eleni, Evangelos Bekiaris, “Accessible EU project user
cases,” in Proc. AEGIS Project Conference, Seville, Spain, 7-8
October 2010.
[74] Eval Access 2.0. [Online]. Available:
http://sipt07.si.ehu.es/evalaccess2/howto.html
[75] HiSoftware Cynthia Says. (2012). [Online]. Available:
http://www.cynthiasays.com/
http://www.cynthiasays.com/Pages/About.aspx
[76] http://giove.isti.cnr.it:8080/MagentaWeb2/credits.jsp
209
©2016 Engineering and Technology Publishing
Journal
of Advanced Management Science Vol. 4, No. 3, May 2016

[77] What is HERA. [Online]. Available:
http://www.sidar.org/hera/index.php.en
[78] AMP. [Online]. Available:
https://amp.ssbbartgroup.com/index.php
[79] SortSite-Accessibility Checker and Validator. [Online]. Available:
http://www.powermapper.com/products/sortsite/checks/ac
cessibility-checks.htm
[80] World Wide Web Consortium (W3C). HTML 5-A vocabulary and
associated APIs for HTML and XHTML-W3C Working Draft.
(22 January 2008). [Online]. Available:
http://www.w3.org/TR/2008/WD-html5-20080122/
[81] WAI-ARIA Overview. [Online]. Available:
http://www.w3.org/WAI/intro/aria
[82] Accessibility of Rich Internet Applications. [Online]. Available:
http://webaim.org/techniques/aria/
[83] M. Vigo, J. Brown, and V. Conway, “Benchmarking web
accessibility evaluation tools: measuring the harm of sole reliance
on automated tests,” in Proc. the 10th International Cross-
Disciplinary Conference on Web Accessibility (W4A ’13), 2013,
pp. 1-10.
Yakup Akgül was born in Simav on March 22,
1977 and grew up into Kütahya (Turkey). He
studied Department of Information
Management at the university of Hacettepe,
Ankara (Turkey), from which he graduated in
2001. He received Master (2010) and Ph.D.
student in Business Administration at Süleyman
Demirel University, Isparta, Turkey. He works
as a lecturer at the Dumlupınar University,
Kütahya, Turkey. His researches focuses on
knowledge managemet, knowledge managemet strategy, strategy, web
usability and accessibility.
Kemal Vatansever was born in Burgaz on
August 29, 1979 and grew up into Bursa
(Turkey). He studied Business Administration
at the university of Osmangazi, Eskişehir
(Turkey), from which he graduated in 2002.
He received Master (2005) and Ph.D. (2010)
degree in Business Administration at
Dumlupınar University, Kütahya, Turkey. He
works as an Assistant Professor at the
Department of Capital Markets and as a head
of department at the same Department, Pamukkale University, Denizli,
Turkey. His researches focuses on quantitative decision methods, multi
criteria decision making, statistics and operational research.
210
©2016 Engineering and Technology Publishing
Journal
of Advanced Management Science Vol. 4, No. 3, May 2016