Conference PaperPDF Available

Gesture-based Children Computer Interaction for Inclusive Education: A Systematic Literature Review

Authors:

Abstract

Gestural interfaces are closely related with cognition and physical activity, and can be powerful tools for cognitive training and motor skills. Their use has been proposed by researchers in various areas, including education, and within this field, inclusive education. In this study, a systematic literature review about chil-dren computer gestural interactions (touch, body, face and motion) and on its ap-plication to digital educational resources for learning disabilities has been con-ducted. Applying the Torres-Carrión method, a "conceptual mindfact" and re-search problem has been structured, as a basis to build the search script, to be ap-plied in the selected scientific databases (Scopus, WoS and Google Scholar). Five research questions are proposed, which involves standards of gesture-based computer interaction for children, design guides, methods and instruments, non-invasive interaction environments and personalization of didactic resources for children with special needs, in particular children with Down’ syndrome. As a fi-nal product, a list of relevant magazines and databases of the area has been ob-tained; 47 valid papers were analyzed to answer the research questions, and they are organized in a structured way, allowing the researcher to establish a valid con-text from which to focus future research.
Gesture-based Children Computer Interaction for
Inclusive Education: A Systematic Literature Review
Pablo Torres-Carrión1 [0000-0002-7606-0582], Carina González-González2, César Bernal-
Bravo3 and Alfonso Infante-Moro 4
1 Universidad Técnica Particular de Loja, San Cayetano Alto 1101608, Loja, Ecuador
2 Universidad de la Laguna, La Laguna, Santa Cruz de Tenerife, Spain
3 Universidad Rey Juan Carlos, Madrid, Spain
4 Universidad de Huelva, Huelva, Spain
pvtorres@utpl.edu.ec
Abstract. Gestural interfaces are closely related with cognition and physical ac-
tivity, and can be powerful tools for cognitive training and motor skills. Their use
has been proposed by researchers in various areas, including education, and
within this field, inclusive education. In this study, a systematic literature review
about children computer gestural interactions (touch, body, face and motion) and
on its application to digital educational resources for learning disabilities has
been conducted. Applying the Torres-Carrión method, a "conceptual mindfact"
and research problem has been structured, as a basis to build the search script, to
be applied in the selected scientific databases (Scopus, WoS and Google
Scholar). Five research questions are proposed, which involves standards of ges-
ture-based computer interaction for children, design guides, methods and instru-
ments, non-invasive interaction environments and personalization of didactic re-
sources for children with special needs, in particular children with Down’ syn-
drome. As a final product, a list of relevant magazines and databases of the area
has been obtained; 47 valid papers were analyzed to answer the research ques-
tions, and they are organized in a structured way, allowing the researcher to es-
tablish a valid context from which to focus future research.
Keywords: Gestural Computer Interaction, Children Computer Interaction, in-
clusive education, systematic review.
1 Introduction
Tangible interfaces and tangible interaction approaches specializes on interfaces or sys-
tems that are physically embodied (be it in physical artifacts or in environments), and
they include the tangibility and materiality of the interface, the whole body interaction,
and the users interaction in real spaces and contexts. This involves a more natural in-
teraction with information and a greater sense of control over it, while improving cog-
nitive abilities and assimilation of information. The creation of such interfaces involves
the development of sensors and their encapsulation in a variety of objects that can be
of daily use. Tangible objects and gestural interfaces are closely related with cognition
2
and physical activity, and can be powerful tools for cognitive training and motor
skills[13]. So, one of the objectives of our work is to contribute to the knowledge
about the research and applications of gestural interfaces in education, in particular,
with children with learning disabilities, and more specifically, with children with Down'
syndrome. For this reason, a systematic review of the scientific literature on these spe-
cific topics was conducted, in order to identifying research questions, as well as for
justifying future research [4, 5].
In this systematic literature review we used an adaptation of the method proposed
by Kitchenham [6] and Bacca [7] and adapted in a new methodology by Torres-Carrión
[5], which divides the review process into three sub-parts: planning, conducting and
reporting results. In our case we found 75 studies on Human-Computer Interaction,
three of which specifically referenced gestural interactions in educational contexts. No
studies were found for inclusive education. With this information in hand, we continued
with the process by proposing five research questions involving standards for gesture-
based learning, design guidelines, methods and instruments for educating persons with
Down syndrome (DS), evaluating results in non-invasive interaction environments, and
personalizing educational resources based on physical and cognitive needs.
As part of planning the search process, several general and specific inclusion and
exclusion criteria were defined, along with some complementary inclusion and exclu-
sion parameters. Variables were set up involving theoretical research, international
standards and research methods adaptable to each item in order to steer the replies to
the five research questions. Applying the search process to scientific articles yielded
forty-three studies, which were properly sorted and coded with the aid of the Mendeley
bibliographic management tool.
The report on the results provides tables and graphs that explain the answers to each
of the research questions posed. A comparative analysis is then conducted of the results
and the prior studies, as well as of the listing of studies selected and the potential re-
search proposals. Finally, the findings of the study are presented, complemented with
suggestions for possible applications of this methodological adaptation to subsequent
systematic reviews of the scientific literature and the state of the art in new areas of
research.
2 Method
We used the method for a systematic review of the literature by Torres-Carrión [5]
adapted from Kitchenham [6] and Bacca [7], which divides the process into three main
phases, as shown in the outline below:
Planning
Identification of the need for review
o Current State of Natural Interaction
o Research Questions
o Mentefacto Conceptual
o Semantic Search Structure
o Related Systematic Reviews
3
o Selection of Journals
Development of a review protocol
o Definition of inclusion and exclusion criteria
o Definition of analysis categories
o Preparing a data extraction form
Conducting the review
Identification of research
Selection of primary studies
Study quality assessment
Data extraction and monitoring
Data synthesis & monitoring
Reporting the review
2.1 Planning
2.1.1 Current State of Natural Interaction
Studies on Human-Computer Interaction (HCI) are becoming increasingly relevant to
technology designers and manufacturers, as well as to groups of people with some kind
of disability and who require personalized equipment and sensors to enable them to
interact with computers. This field of research is growing, primarily due to the expan-
sion of mobile technology and the lower prices of sensors and devices used to carry out
everyday activities, which often allows people to issue instructions to computers using
common gestures or voice commands, a process known as Natural Interaction (NI).
2.1.2 Research Questions
CCI environments are becoming more prevalent thanks to the market availability of
increasingly cheap and efficient sensors developed for the leisure, entertainment and
health industries, and in particular for videogames and fitness. Their use has been pro-
posed by researchers in various areas, including education, and within this field, in in-
clusive education [8]. Since we are interested in CCI based on natural interactions (ges-
tures, touch, voice and motion) and on its application to digital educational resources
that are customized to children’s needs, we considered the following research ques-
tions:
RQ1 Of the standards that describe the gesture-based CCI, which are being applied
in inclusive educational environments?
RQ2 How are design guides applied to inclusive educational gestural interfaces for
children?
RQ3 What methods/instruments are considered in gestural inclusive interactions
for children with Down syndrome in educational environments?
RQ4 How were the research results in non-invasive interaction environments eval-
uated?
RQ5 What processes were adapted to personalize interaction resources, consider-
ing each child’s educational needs and disabilities?
4
2.1.3 Mentefacto Conceptual
Fig. 1. Mentefacto Conceptual about Gestural Interaction.
In the Torres-Carrión methodology [5], the mentefacto conceptual allows the re-
searcher to focus his attention on the real theoretical context of the investigation. Mov-
ing vertically can make the central concept more specific or general, as it moves down
or up. In the right part the concepts that differ from the central one are detailed, in such
a way that the researcher can discriminate with reference articles. On the left, the char-
acteristics of the concept are located, as an input to locate the key words, part of the
scientific thesaurus.
2.1.4 Semantic Search Structure
This data is the input of the consultations to the various databases that the researcher
considers. The information is organized in five layers, being the first an abstraction of
the mentefacto conceptual; the second a filtering to the specific population; the third
and fourth refer to the subfield of application, which is education inclusive; and, the
fifth layer is subdivided according to the five research questions. The semantic search
structure is the input to perform a structure review, valid for two moments: a) during
the search for systematic reviews or related meta-searches; b) in the specific search for
documents related to each research question, which in this case are five.
Table 1. Semantic structure from thesaurus for searching specific papers
L1
Gestural
Computer In-
teraction
(gestur* OR hand* OR body OR leg OR mov* OR
motor* OR motion OR fac* OR eye OR mobile OR
touch*) AND (comput* OR automat*) AND (interact*
OR interfac* OR recognit* OR track*)
L2
+ Child *
AND (child* OR boy OR kid OR infant*)
5
L3
+ Education
AND (educa* OR learn* or train*)
L4
+ Special Edu-
cation
AND (syndrome OR disabilit* OR inclusiv* OR spe-
cial* OR disorder)
L5
Question
Q1: (Standard Interaction)
Q2: (Design Guide)
Q3: (Down syndrome)
Q4: (Assessment AND Noninvasive environment)
Q5: (Digital Learning Literacy)
2.1.5 Related Systematic Reviews
Initially, it is important to identify previous studies on systematic reviews of the litera-
ture in our field of research so that our contribution can be original and useful to the
scientific community. We conducted a systematic general search of the Web of Science
(WoS), Scopus and Google Scholar databases, using a search syntax as similar as pos-
sible in all three platforms and adhering to the rules in place for each one. We were
unable to find reviews of the literature that allow us to provide an answer to the research
questions posed, thus requiring us to undertake this work in order to achieve our goal.
Table 2. Recent studies on reviews of the literature involving natural Interaction in children for
educational purposes
Study
Analysis
Papers
revie-
wed
Sheu, 2014
[9]
Analyzes research focused on “gesture-based computing in educa-
tion” using an empirical approach that searched five academic da-
tabases, with a manual selection of papers (published between
2001 and 2013) that are then analyzed.
59
Boucenna,
2014 [10]
Although only one goal focuses on education (exclusive for autistic
children), it provides a set of emerging resources and tools and
strategies for enhancing their use.
>100
Benton,
2015 [11]
Considers the roles, responsibilities and activities in the design of
technology projects adapted to inclusive educational needs,
adapted by both students and teachers.
46
2.1.6 Selection of Journals and Databases
The platform used for this initial filtering was “Primo de Ex Libris” (licensed to the
library of the Universidad de La Laguna), specifically through its search engine, “Punto
q”. The platform automatically generates lists that are arranged into two groups, one
for journals and another for databases.
6
Table 3. List of databases with the highest number of articles returned by the search
Number
of papers
364
234
210
190
134
Additionally, in the case of the DBs an initial filter was applied that left only the five
databases with the most number of papers, as shown in Table3. The final sum exceeds
the resulting amount (508) because several papers are indexed in more than one DB.
Identifying the databases used to index the scientific papers with the highest impact
consolidates the secondary sources of research, facilitating future work.
Table 4. List of journals arranged by category based on JCR 2016
Ord
Journal Name
N° of
papers
JCR
SJR
Google
Aca-
demic
h5
Ord
IF
Quar-
tile
JCR Science Edition
1
Pediatrics
10
5,473
Q1
2,894
116
4593,27
2
Computers & Education
23
2,556
Q1
2,578
88
3334,22
3
PLoS ONE
9
3,224
Q1
1,300
161
1518,26
8
IEEE Transactions On Neural
Systems And Rehabilitation
Engineering
4
3,188
Q1
1,042
45
149,49
10
Physical Therapy
2
2,526
Q1
1,270
52
83,41
JCR Social Science Edition
4
Journal Of Autism And Devel-
opmental Disorders
9
3,665
Q1
1,696
61
853,12
5
Child Development
3
4,061
Q1
3,065
64
597,45
6
Research in Developmental
Disabilities
22
1,887
Q1
0,986
47
480,96
7
Journal of Learning Disabilities
7
1,901
Q1
1,596
34
180,52
9
Journal of Intellectual Disabil-
ity Research
9
1,778
Q1
0,935
33
123,44
11
Computers In Human Behavior
1
2,694
Q1
1,582
75
79,91
12
Journal of Computer Assisted
Learning
2
1,370
Q1
2,048
41
57,52
7
2.2 Development of a review Protocol
2.2.1 Definition of inclusion and exclusion criteria
For research purposes, it is necessary to define criteria for selecting journals related
with our objectives and with the research questions posed.
General Criteria:
Studies involving gestural interactions by children with technology devices and
whose main purpose is inclusive education processes.
Studies published in the last ten years, that is, between 2008 and 2017.
Specific Criteria:
The studies must comply with one or more of the following specifications:
Studies on standards that include an analysis of gestures in child-computer interac-
tions.
Studies that present design guides for gesture-based, inclusive educational inter-
faces.
Studies that share methods/instruments used in research on populations with Down
syndrome and gestural interfaces.
Studies that explain studies for validating research processes in non-invasive envi-
ronments involving gestural interactions.
Also considered is whether the studies present a methodology for designing a ges-
tural interaction to make it more effective for individuals with learning disabilities.
Additional parameters were defined to exclude papers from consideration:
Papers involving gestural interaction in environments not pertaining to HCI or for
non-educational purposes.
Journals that are not catalogued as scientific papers: editorials, book reviews, technical re-
ports, data sets, etc.
2.2.2 Definition of analysis categories
In keeping with the methodology in Torres-Carrión[5] in this sub-stage we define a
series of analysis categories, the criteria for which are based on the research questions
posed at the start of the study. These categories will allow us to group studies depending
on the criteria that enable a systematic response to the research questions (RQ).
RQ1 Of the standards that describe the gesture-based CCI, which are being applied
in inclusive educational environments? In this section we consider variables from
ISO 9241 (Ergonomics of human-system interaction) and all its parts.
ISO Title: based on ISO 9241 - Ergonomics of human-system interaction and IEC
[12].
8
Categories of the standard: User performance/satisfaction, product, development
process, life cycle processes [12].
Type of gestural interaction: ISO/DIS 9241-960 Framework and guidance for ges-
ture interactions; ISO 9241-210 Human-centered design for interactive systems
[13, 14].
Report new experimental standard of gestural analysis [12].
RQ2 How are design guides applied to inclusive educational gestural interfaces for
children?
Phases and procedures: based on ISO 9241-210 Human-centered design for in-
teractive systems [14]
Human factors: senses, memory and cognition.
Types of senses: light, sound, smell, movement, speech, touch and biological var-
iables.
Emotions or feelings in design: based on EMODIANA [15]
RQ3 - What methods/instruments are considered in gestural inclusive interactions
for children with Down syndrome in educational environments?
Educational target group: based on International Standard Classification of Edu-
cation - UNESCO [16]
Research method.
Data collection method.
Range of mental age / natural age.
Technology for interaction: details on instruments and tools.
Type of interface: based on ISO 9241-210 Human-centered design for interactive
systems [14].
Type of gestural interaction: ISO/DIS 9241-960 Framework and guidance for ges-
ture interactions [13].
Report results from research.
RQ4 - How were the research results in non-invasive interaction environments eval-
uated?
These variables apply only to research using non-invasive technologies.
Research method.
Assessment tools.
Report results from research.
Special need addressed: name and percent.
RQ5 What processes were adapted to personalize interaction resources, consider-
ing each child’s needs and disabilities?
Detail of adaptation process
Special need addressed: name and percent.
Detail of digital learning strategy (qualitative item) if the scope of the research
is pedagogical.
2.3 Conducting the review
This process relies on the results from the previous phase: the inclusion and exclu-
sion criteria and the list of journals (listJournal_2) given in Table 5. We followed the
9
“Knowledge Discovery in Databases” (KDD) process [17] by conducting a continuous
search in each of the journals and arranging the results based on the structure of the
variables in the research questions. The five steps in this section follow the method by
Torres Carrión[5], using Mendeley as software for the administration of the resulting
scientific articles.
3 Reporting the review
3.1 RQ1 Of the standards that describe gesture-based CCI, which are
being applied in inclusive educational environments?
Table 5. Papers that apply the ISO 9241-960 and ISO 9241-210 standards
Of those that apply the ISO 9241 standard (Ergonomics of human-system interaction),
those applicable to our study were selected: [12]
f
o ISO 9241-9: Environment
[1826]
8
o ISO 9241-10: Interface
[20, 23, 24, 2737]
14
o ISO 9241-11: Usability.
[24, 30, 32, 33, 35]
5
o ISO 9241-17: Interaction
[19, 23, 28, 30, 31, 35, 3747]
17
Type of gestural interaction: [13, 14]
o Body
[18, 23, 25, 27, 29, 30, 33, 34, 37, 3941, 45, 4850]
16
o Speech
[21, 36, 40, 51, 52]
5
o Touch
[20, 26, 3538, 40, 45, 47, 5256]
14
o Hand
[22, 28, 53, 57, 58]
5
o Face
[46, 53, 57, 59]
4
o Eye
[30]
1
The standard that best references multi-touch interactions is ISO/IEC 14754, which de-
fines the commands for the basic gestures for select, delete, insert space and line, move,
copy, paste, scroll and undo actions, and also extends these actions to pen interfaces
[60]. One of the new work standards in HCI is that associated with the muscle-com-
puter interface (MCI), which despite having many elements of natural interaction, is
mainly studied in the area of augmented reality.
3.2 RQ2 How are design guides applied to inclusive educational
natural interfaces for children?
This question is closely related to the first. To answer it, we considered the standards
presented, which were complemented with the design phases and processes, the human
factors that are of interest in our field of research, and an additional consideration in-
volving an emerging topic, namely the user’s emotional response during the interaction.
Table 9 shows part of the classification, relating the human factors and type of sense to
the study needs and research questions.
10
Table 6. Papers that apply sub-categories of the design guide as per the ISO 9241-210 Standard
Human factors:
f
o Senses
Every paper in the next sub-category (type of sense).
35
o Memory
[31, 43, 49, 61]
4
o Cognition.
[18, 19, 31, 32, 3436, 38, 39, 42, 43, 45, 20, 48, 49, 52, 54, 55,
57, 58, 61, 62, 21, 22, 24, 2628, 30]
29
Type of Sense:
o Light
---
0
o Sound
[21, 36, 40, 51, 52]
5
o Smell
---
0
o Movement
[18, 23, 40, 41, 45, 4850, 25, 27, 29, 30, 33, 34, 37, 39][46, 53,
57, 59] [22, 28, 53, 57, 58]
24
o Speech
[21, 36, 40, 51, 52]
5
o Touch
[20, 26, 5356, 3538, 40, 45, 47, 52]
14
o Biological variables
---
0
3.3 RQ3 - What methods/instruments are considered in gestural
inclusive interactions for children with Down syndrome in
educational environments?
Knowing the specifics of the literacy level, learning style and physiological conditions
of a person with Down syndrome (DS) is vital when planning their education, preparing
learning strategies and the resources for their interaction in the classroom [63]. Of the
papers studied, 16.28% involve specific studies carried out on this population, and one
on fragile X syndrome [38], which also considers this sub-group. The study by Tabata-
baei [57] does not consider a formal academic activity, but rather uses images of young
children to establish differentiation patterns in images based on specific facial features.
Table 7. Articles based on Target Group of Education (UNESCO sub-group) and age.
Target Group of Education: based on International Standard Classification of Edu-
cation - UNESCO [16] (subcategory)
f
o Early childhood education
[58, 61]
2
o Primary education
[29, 44, 61, 64]
4
o Lower secondary or higher
0
Range of mental age / natural age.
o 0 3
[58]
1
o 4 6
[58]
1
o 7 9
[29]
1
o 10 -12
[29]
1
o > 12
0
o No specific
[44, 64]
2
11
3.4 RQ4 - How were the research results in non-invasive interaction
environments evaluated?
Of the list of papers studied, there were none of an experimental nature that considered
maintaining the everyday interaction and working setting. Parés [23] makes an effort
by studying the interactive design for children with autism and visual impairments, pre-
senting as a result a protocol for a multi-sensory space that evaluates visual, aural and
vibrotactile stimuli. Although the author’s design proposes non-invasive interaction as-
pects, there is no method for letting the student maintain his/her interaction space; in-
stead, a removable space resembling a small room has to be installed to allow for per-
sonalized interaction.
3.5 RQ5 What processes were adapted to personalize interaction
resources, considering each child’s educational needs and
disabilities?
Of the articles reviewed, 48.83% considered the personalization of resources in some
way, at least theoretically and conceptually. As Graph 5 shows, of this group it was
mostly the studies on autism (28.57%) that emphasized the personalization of re-
sources, followed by general SEN studies. We did not find any experimental studies in
this group of papers that formally or informally describe the application of a platform
for personalizing the educational resources of students in the classroom. Mahmoud [58]
and Tabatabaei [57] share a model for an Intelligent Tutoring System applicable to
Down syndrome (Mahmoud) and to speech disorders (Tabatabaei).
4 Conclusions
The methodological adapted to the method of Torres-Carrión[5], Kitchenham [6]
and Bacca [7] can be used to confidently select scientific studies in a way that is
organized and focused on the user’s needs. The results of this systematic review
confirm its validity in the field of scientific research, allowing researchers to locate
databases, important scientific journals and leading and relevant researchers working
in their area of study.
The ISO 9241-960 (Ergonomics of Human-System Interaction) and ISO 9241-210
(Human-Centered Design for Interactive Systems) standards are the most widely ap-
plied in the area of child-computer gesture interaction, with ISO 9241-17: Interaction
being prominent in the former and types of motor and touch interaction in the latter.
The design guides for natural interfaces are partially applied in some studies, and
underscore the cognition and sensory human factors. The most widely used sensors
are for motion (motor) and touch. None of the studies considered an emotional as-
sessment, either as a subjective or objective measure.
The few studies on subjects with Down syndrome involved children 0 to 12 years of
age, and do not consider subjects beyond primary education. The main technology
12
used is virtual reality through Wii games, and artificial intelligence applications for
early intervention in mathematics learning.
No experimental studies were found that take place specifically in non-invasive in-
teraction environments. Proposals for designing work protocols in multi-sensory
spaces were proposed, but these require taking the student to this new interactive
environment.
Any experimental study describing the personalization of gesture-based interaction
in inclusive educational environments for students with special needs has been
found. Neither, about the use of gestural interaction platforms to personalize the ed-
ucational resources for Down's syndrome students. Two models for Intelligent Tu-
toring Systems are proposed that rely on expert systems and artificial intelligence
algorithms.
References
1. Torres-Carrion, P., González-González, C.S., Mora Carreño, A.: Methodology of emotional evaluation
in education and rehabilitation activities for people with Down syndrome. XV Int. Conf. Hum. Comput.
Interact. 1215 (2014).
2. Torres-Carrión, P.: Evaluación de Estrategias de Aprendizaje con HCI Kinect en alumnos con Síndrome
de Down, http://e-spacio.uned.es/fez/view/tesisuned:ED_Pg_CyEED-Pvtorres, (2017).
3. Torres-Carrión, P., Gonzalez-Gonzalez, C.: Usability study of didactical resources to children with
Down Syndrome. In: Guerrero-Garcia, J., González-Calleros, J.M., Muñoz-Arteaga, J., and Collazos,
C.A. (eds.) HumanComputer Interaction Series. pp. 127148. Springer International Publishing (2017).
4. Swartz, M.K.: The PRISMA statement: a guideline for systematic reviews and meta-analyses. J. Pediatr.
Health Care. 25, 12 (2011).
5. Torres-Carrión, P., González-González, C., Aciar, S., Rodríguez-Morales, G.: Methodology for
Systematic Literature Review applied to Engineering and Education. In: EDUCON2018 IEEE Global
Engineering Education Conference. IEEE Xplore Digital Library, Santa Cruz de Tenerife - España
(2018).
6. Kitchenham, B.: Procedures for Performing Systematic Reviews. Joint Technical Report. , Keele, UK
(2004).
7. Bacca, J., Baldiris, S., Fabregat, R.: Augmented Reality Trends in Education: A Systematic Review of
Research and Applications. J. Educ. …. 14, 133–149 (2014).
8. Yarosh, S., Radu, I., Hunter, S., Rosenbaum, E.: Examining values: an analysis of nine years of IDC
research. In: Proceedings of the 10th International Conference on interaction design and children. pp.
136144. ACM (2011).
9. Sheu, F.R., Chen, N.S.: Taking a signal: A review of gesture-based computing research in education.
Comput. Educ. 78, 268277 (2014).
10. Boucenna, S., Narzisi, A., Tilmont, E., Muratori, F., Pioggia, G., Cohen, D., Chetouani, M.: Interactive
Technologies for Autistic Children: A Review. Cognit. Comput. 6, 722740 (2014).
11. Benton, L., Johnson, H.: Widening participation in technology design: A review of the involvement of
children with special educational needs and disabilities. Int. J. Child-Computer Interact. 34, 2340
(2015).
12. Bevan, N.: International standards for HCI and usability. Int. J. Hum. - Comput. Stud. 55, 533552
13
(2001).
13. ISO: ISO/DIS 9241-960. Ergonomics of human-system interaction - Part 960: Framework and guidance
for gesture interactions. , Geneva, Switzerland (2015).
14. ISO: ISO 9241-210:2010. Ergonomics of human-system interaction - Part 210: Human-centred design
for interactive systems. , Geneva, Switzerland (2015).
15. González-González, C.S., Cairós-González, M., Navarro-Adelantado, V.: EMODIANA: Un instrumento
para la evaluación subjetiva de emociones en niños y niñas. Actas del XIV Congr. Int. Interacción Pers.
(2013).
16. UNESCO: The International Standard Classification of Education 2011. UNESCO Institute for
Statistics, Montreal, Quebec, Canada (2012).
17. Dhiman, A.K.: Knowledge Discovery in Databases and Libraries. DESIDOC J. Libr. Inf. Technol. 31,
446451 (2011).
18. Jarus, T., Ghanouni, P., Abel, R.L., Fomenoff, S.L., Lundberg, J., Davidson, S., Caswell, S., Bickerton,
L., Zwicker, J.G.: Effect of internal versus external focus of attention on implicit motor learning in
children with developmental coordination disorder. Res. Dev. Disabil. 37, 11926 (2015).
19. Cheng, Y., Huang, R.: Using virtual reality environment to improve joint attention associated with
pervasive developmental disorder. Res. Dev. Disabil. 33, 214152 (2012).
20. Fernández-López, Á., Rodríguez-Fórtiz, M.J., Rodríguez-Almendros, M.L., Martínez-Segura, M.J.:
Mobile learning technology based on iOS devices to support students with special education needs.
Comput. Educ. 61, 7790 (2013).
21. Toki, E.I., Pange, J.: The design of an expert system for the e-assessment and treatment plan of
preschoolers’ speech and language disorders. Procedia - Soc. Behav. Sci. 9, 815819 (2010).
22. Cano, M.-D., Sanchez-Iborra, R.: On the use of a multimedia platform for music education with
handicapped children: a case study. Comput. Educ. 87, 254276 (2015).
23. Parés, N., Carreras, A., Durany, J., Ferrer, J., Freixa, P., Gómez, D., Kruglanski, O., Parés, R., Ribas,
J.I., Soler, M., Sanjurjo, À.: Starting Research in Interaction Design with Visuals for Low-Functioning
Children in the Autistic Spectrum: A Protocol. CyberPsychology Behav. 9, 218223 (2006).
24. Mich, O., Pianta, E., Mana, N.: Interactive stories and exercises with dynamic feedback for improving
reading comprehension skills in deaf children. Comput. Educ. 65, 3444 (2013).
25. Shih, C.-H., Shih, C.-T., Chu, C.-L.: Assisting people with multiple disabilities actively correct abnormal
standing posture with a Nintendo Wii balance board through controlling environmental stimulation. Res.
Dev. Disabil. 31, 93642 (2010).
26. Ahmed, S., Parsons, D.: Abductive science inquiry using mobile devices in the classroom. Comput.
Educ. 63, 6272 (2013).
27. Crisco, J.J., Schwartz, J.B., Wilcox, B., Costa, L., Kerman, K.: Design and Kinematic Evaluation of a
Novel Joint-Specific Play Controller: Application for Wrist and Forearm Therapy. Phys. Ther. 95, 1061
6 (2015).
28. Cai, Y., Chia, N.K.H., Thalmann, D., Kee, N.K.N., Zheng, J., Thalmann, N.M.: Design and development
of a Virtual Dolphinarium for children with autism. IEEE Trans. Neural Syst. Rehabil. Eng. 21, 20817
(2013).
29. Wuang, Y.-P., Chiang, C.-S., Su, C.-Y., Wang, C.-C.: Effectiveness of virtual reality using Wii gaming
technology in children with Down syndrome. Res. Dev. Disabil. 32, 31221 (2011).
30. Lahiri, U., Bekele, E., Dohrmann, E., Warren, Z., Sarkar, N.: Design of a virtual reality based adaptive
response technology for children with autism. IEEE Trans. Neural Syst. Rehabil. Eng. 21, 5564 (2013).
31. de la Guía, E., Lozano, M.D., Penichet, V.M.R.: Educational games based on distributed and tangible
14
user interfaces to stimulate cognitive abilities in children with ADHD. Br. J. Educ. Technol. 46, 664
678 (2015).
32. Seo, Y.-J., Woo, H.: The identification, implementation, and evaluation of critical user interface design
features of computer-assisted instruction programs in mathematics for students with learning disabilities.
Comput. Educ. 55, 363377 (2010).
33. Torrente, J., Freire, M., Moreno-Ger, P., Fernández-Manjón, B.: Evaluation of semi-automatically
generated accessible interfaces for educational games. Comput. Educ. 83, 103117 (2015).
34. Stasolla, F., Damiani, R., Perilli, V., D’Amico, F., Caffò, A.O., Stella, A., Albano, V., Damato, C.,
Leone, A. Di: Computer and microswitch-based programs to improve academic activities by six children
with cerebral palsy. Res. Dev. Disabil. 4546, 113 (2015).
35. Marco, J., Cerezo, E., Baldassarri, S.: Bringing tabletop technology to all: evaluating a tangible farm
game with kindergarten and special needs children. Pers. Ubiquitous Comput. 17, 15771591 (2012).
36. Shahin, M., Ahmed, B., Parnandi, A., Karappa, V., McKechnie, J., Ballard, K.J., Gutierrez-Osuna, R.:
Tabby Talks: An automated tool for the assessment of childhood apraxia of speech. Speech Commun.
70, 4964 (2015).
37. Cantón, P., González, Á.L., Mariscal, G., Ruiz, C.: Applying New Interaction Paradigms to the
Education of Children with Special Educational Needs. In: Miesenberger, K., Karshmer, A., Penaz, P.,
and Zagler, W. (eds.) Computers Helping People with Special Needs. pp. 6572. Springer Berlin
Heidelberg, Berlin, Heidelberg (2012).
38. Hall, S.S., Hammond, J.L., Hirt, M., Reiss, A.L.: A “learning platform” approach to outcome
measurement in fragile X syndrome: a preliminary psychometric study. J. Intellect. Disabil. Res. 56,
947960 14p (2012).
39. Marti, P., Giusti, L.: A robot companion for inclusive games: A user-centred design perspective. In: 2010
IEEE International Conference on Robotics and Automation. pp. 43484353. IEEE (2010).
40. Keskinen, T., Heimonen, T., Turunen, M., Rajaniemi, J.-P., Kauppinen, S.: SymbolChat: A flexible
picture-based communication platform for users with intellectual disabilities. Interact. Comput. 24, 374
386 (2012).
41. Mombarg, R., Jelsma, D., Hartman, E.: Effect of Wii-intervention on balance of children with poor motor
performance. Res. Dev. Disabil. 34, 29963003 (2013).
42. Doyle, T., Arnedillo-Sánchez, I.: Using multimedia to reveal the hidden code of everyday behaviour to
children with autistic spectrum disorders (ASDs). Comput. Educ. 56, 357369 (2011).
43. Dovis, S., Van der Oord, S., Wiers, R.W., Prins, P.J.M.: Improving Executive Functioning in Children
with ADHD: Training Multiple Executive Functions within the Context of a Computer Game. A
Randomized Double-Blind Placebo Controlled Trial. PLoS One. 10, e0121651 (2015).
44. Feng, J., Lazar, J., Kumin, L., Ozok, A.: Computer Usage by Children with Down Syndrome: Challenges
and Future Research. ACM Trans. Access. Comput. 2, 144 (2010).
45. Jong, J.-T., Hong, J.-C., Yen, C.-Y.: Persistence temperament associated with children playing math
games between touch panel and embodied interaction. J. Comput. Assist. Learn. 29, 569578 (2013).
46. Hamzah, M.S.J., Shamsuddin, S., Miskam, M.A., Yussof, H., Hashim, K.S.: Development of Interaction
Scenarios Based on Pre-school Curriculum in Robotic Intervention for Children with Autism. Procedia
Comput. Sci. 42, 214221 (2014).
47. Fletcher-Watson, S., Pain, H., Hammond, S., Humphry, A., McConachie, H.: Designing for young
children with autism spectrum disorder: A case study of an iPad app. Int. J. Child-Computer Interact.
(2016).
48. Levac, D., Rivard, L., Missiuna, C.: Defining the active ingredients of interactive computer play
15
interventions for children with neuromotor impairments: A scoping review. Res. Dev. Disabil. 33, 214
223 (2012).
49. Jelsma, D., Ferguson, G.D., Smits-Engelsman, B.C.M., Geuze, R.H.: Short-term motor learning of
dynamic balance control in children with probable Developmental Coordination Disorder. Res. Dev.
Disabil. 38, 21322 (2015).
50. Sanna, A., Lamberti, F., Paravati, G., Rocha, F.D.: A kinect-based interface to animate virtual characters.
J. Multimodal User Interfaces. 7, 269279 (2013).
51. Wagner, A., Rudraraju, R., Datla, S., Banerjee, A., Sudame, M., Gray, J.: Programming by voice. In:
Proceedings of the 2012 ACM annual conference extended abstracts on Human Factors in Computing
Systems Extended Abstracts - CHI EA ’12. p. 2087. ACM Press, New York, New York, USA (2012).
52. Tressoldi, P., Vio, C., Iozzino, R.: Efficacy of an intervention to improve fluency in children with
developmental dyslexia in a regular orthography. J. Learn. Disabil. 40, 203209 7p (2007).
53. Rosenblum, S., Regev, N.: Timing abilities among children with developmental coordination disorders
(DCD) in comparison to children with typical development. Res. Dev. Disabil. 34, 218227 (2013).
54. Campigotto, R., McEwen, R., Demmans Epp, C.: Especially social: Exploring the use of an iOS
application in special needs classrooms. Comput. Educ. 60, 7486 (2013).
55. Jones, A.C., Scanlon, E., Clough, G.: Mobile learning: Two case studies of supporting inquiry learning
in informal and semiformal settings. Comput. Educ. 61, 2132 (2013).
56. Hassan, A., Danish, Z.: Multi-touch User Interfaces to Treat Social, Communicative and Collaborative
Impairments in Children with Autism: A Review. Int. J. Comput. Appl. 117, 3539 (2015).
57. Tabatabaei, S.M., Chalechale, A.: Using DLBP texture descriptors and SVM for Down syndrome
recognition. In: 2014 4th International Conference on Computer and Knowledge Engineering (ICCKE).
pp. 554558. IEEE (2014).
58. F.A Mahmoud, A., A.F.Belal, M., M.K. Helmy, Y.: Towards an Intelligent Tutoring System to Down
Syndrome. Int. J. Comput. Sci. Inf. Technol. 6, 129137 (2014).
59. Gordon, I., Pierce, M.D., Bartlett, M.S., Tanaka, J.W.: Training facial expression production in children
on the autism spectrum. J. Autism Dev. Disord. 44, 248698 (2014).
60. Cantón, P., González, Á.L., Mariscal, G., Ruiz, C.: Applying New Interaction Paradigms to the
Education of Children with Special Educational Needs. In: Miesenberger, K., Karshmer, A., Penaz, P.,
and Zagler, W. (eds.) Computers Helping People with Special Needs. pp. 6572. Springer Berlin
Heidelberg, Berlin, Heidelberg (2012).
61. Visu-Petra, L., Benga, O., Tincas, I., Miclea, M.: Visual-spatial processing in children and adolescents
with Down’s syndrome: a computerized assessment of memory skills. J. Intellect. Disabil. Res. 51, 942–
952 11p (2007).
62. Imhof, B., Scheiter, K., Edelmann, J., Gerjets, P.: Learning about locomotion patterns: Effective use of
multiple pictures and motion-indicating arrows. Comput. Educ. 65, 4555 (2013).
63. Ratz, C.: Do students with Down syndrome have a specific learning profile for reading? Res. Dev.
Disabil. 34, 450414 (2013).
64. Næss, K.-A.B., Melby-Lervåg, M., Hulme, C., Lyster, S.-A.H.: Reading skills in children with Down
syndrome: a meta-analytic review. Res. Dev. Disabil. 33, 73747 (2012).
... The incorporation of electronic devices is at a high level in all areas of society [6][7][8][9][10][11], including education [12][13][14][15][16][17][18], both in their number and in their use by citizens [19][20][21][22][23][24]. Teaching is one area where great technological advances have been made, and these tools were already in use before the pandemic: massive open online courses (MOOCs) [25,26], online social networks [27][28][29], multimedia tools [30,31], virtual reality [32], robotics [33] and mobile phones [34][35][36][37]. ...
Article
Full-text available
Many of the tools used for virtual teaching during the pandemic had not been used previously, but they could continue to be used when traditional teaching returns. For this reason, this study focused on locating the key motivational factors for the possible continuation of the use of one of these tools, videoconferencing, to carry out tutorials in Spanish universities as a complement to face-to-face tutorials. For this, a literary review was conducted to obtain a list of motivational factors that may influence teachers to continuing using it, and a causal study was performed with university professors (through fuzzy cognitive maps) to identify the causal relationships among these factors and classify them by their relevance in making a decision. The most influential factors are intention, attitude and perceived compatibility with how tutorials are given, and the negative factors include quality management and trust.
Article
Full-text available
English has become important in the 4.0 era since numerous electronic devices, such as medical equipment for doctors or salespeople who interact with foreigners in malls, have English conditions. Because of the various cultures that continue to enter Indonesia, ordinary as well as oustanding school systems must learn more about English by expanding their English vocabulary, especially for students special needs, and sign language is employed as a teaching tool in exceptional schools. The purpose of this research is to learn gestures for vocabulary learning for students special needs (deaf), specifically eleventh-grade students at SLB-B YPPLB Makassar. The research method used was qualitative. The responses from observations, questionnaire sheets, and interview sessions have been reviewed by the researcher. According to the data, the first, SIBI, was used in vocabulary learning for students at the Makassar YPPLB SLB-B school because it was less complicated and better organized. Second, it could be concluded that the teacher's influence was indeed important for students, but student interaction was actually very influential on the mood and enthusiasm of students when learned in class, and the third was that using video was ineffective for them to learn because it was difficult for them to understand.
Article
Full-text available
El uso como sistema de seguridad del Internet de las Cosas en el sector hotelero conlleva una ventaja competitiva a los hoteles gracias a la seguridad transmitida a los clientes, además de un ahorro en costes y un beneficio extra por la gestión optimizada y automatizada de muchos de sus servicios. Aún así, sólo en la mitad de los hoteles se están tomando iniciativas en la implementación de esta tecnología. Por este motivo y por el auge del Internet de las Cosas en las empresas, se realiza un estudio bibliográfico que permite obtener un listado de factores motivacionales que pueden afectar a los hoteles a la hora de aceptar y decidir la inserción de iniciativas basadas en Internet de las Cosas como herramienta de seguridad, y un estudio causal a profesores universitarios (expertos en tecnologías y hoteles) y a responsables del departamento de tecnologías en hoteles que permite confirmar la relación de estos factores en esta decisión y clasificarlos por su relevancia. El listado obtenido está formado por los siguientes factores motivacionales: gestión de calidad, disposición de información, condicionantes externos, confianza, compatibilidad percibida, utilidad percibida, actitud e intención, y entre los factores más decisivos en todo este sistema (a parte de la utilidad y la actitud) se encuentra la confianza, que sería el grado de seguridad y privacidad que los hoteles tienen en el uso de esta herramienta.
Poster
Full-text available
La gestión de proyectos de desarrollo de software implica métodos, herramientas, técnicas, entre otras habilidades para garantizar el cumplimiento del proyecto, desde su inicio con el levantamiento de requerimientos, hasta la entrega de un producto de calidad. Por otro lado, el enfoque de desarrollo de software orientado a modelos (MDD), busca optimizar los recursos mediante el empleo de modelos en todas las fases de desarrollo, separando las especificaciones de negocio de las especificaciones técnicas del proyecto. Permitiendo trabajar de forma independiente de la plataforma y responder eficientemente a los cambios del negocio y de tecnología. Contar con un modelo eficaz de gestión de proyectos de desarrollo de software orientado a modelos, permitirá explotar las bondades de una metodología madura y probada con las ventajas que ofrece el MDD. En este trabajo se presenta una revisión sistemática de literatura sobre la gestión de proyectos orientado al desarrollo de MDD.
Poster
Full-text available
La revisión bibliográfica que fue realizada tuvo el motivo de conocer más acerca de nuevas tecnologías como son el internet de las cosas y los sistemas inteligentes embebidos en circuitos electrónicos, así mismo, sus aplicaciones en otros temas aparte de los habituales. Sabemos que ya han sido aplicadas en muchas tareas desde inicios de este siglo, obteniendo muy buenos resultados. En este caso en concreto se buscó las revisiones sobre la aplicación de estas tecnologías para la prevención de riesgos en ambientes cerrados, tales como un laboratorio o otros ambientes en los cuáles existen riesgos más considerables. Como resultado de esta revisión obtuvimos que actualmente esta rama de aplicación aún no es muy estudiada, por lo cuál se obtuvo un total de cinco documentos que responden concretamente a nuestra pregunta los mismos que se detallan a continuación.
Conference Paper
Full-text available
A systematic review of the scientific literature in a specific area is important for identifying research questions, as well as for justifying future research in said area. This process is complex for beginners in scientific research, especially if you have not developed skills for searching and filtering information, and do not know which high-level databases are relevant in their field of study. The method proposed leads the researcher from "My" to "The" current state of the problem; we propose an adaptation of the method by Kitchenham and Bacca, which divides the process into three sub-parts: planning, conducting and reporting results. From the approach of the research problem in the preliminary phase research questions (recommended between 3 to 5) and “mentefacto conceptual” is drawn; this last one gives originality to the method and facilitates the development of the thesaurus for searches and inclusion and exclusion criteria. Early research requires doing a basic systematic study to identify work done to review the literature in the area and, if any is found, to verify if those results yield an answer to our research questions. As part of planning the search process, general and specific inclusion and exclusion criteria were defined, along with some complementary inclusion and exclusion parameters. The method followed with rigor, returns to the researcher a list of impact journals in the study area, and a detail of articles that are related to each category of the research questions. A study case has been considered as a guide to expose each of the phases of the methodology in a practical way, with results that support the proposal.
Chapter
Full-text available
The techniques to know the visual attentions in screens by the user are used with great efficiency in the field of design to make more efficient its objective. In the educational field are few efforts made, being one of the challenges of engineering and Sciences of education to improve learning outcomes. This study aims is to know the usability of personalized, conventional and gestural interaction resources based in the patterns of visual attention of children with Down syndrome, in teaching resources based on eye-tracker validated visual attention in a classroom environment for children with Down Syndrome. Selected 6 students of the Down Tenerife Association (CG = 3, GE = 3), of which GE has previously interacted with the HCI Tango:H platform with teaching resources customized to their requirements cognitive and applied in a playful environment of teaching and learning, and the remaining with common resources in the classroom, designed by the teachers of the institution. Learning objects were presented to students by way of films on a screen, by tracking their visual attention with the help of an eye tracker and following the protocol think-aloud, focused on the attributes of ISO 9141-11: usability for Visual-Based gesture recognition: efficacy, effectiveness, learnability and satisfaction. The results were evaluated with the help of Tobbi Studio platform and subjective attributes through systematic observation, being the resources designed for the Tango:H plaque better valued than those of everyday use in the conventional classroom.
Article
Full-text available
In recent years, there has been an increasing interest in applying Augmented Reality (AR) to create unique educational settings. So far, however, there is a lack of review studies with focus on investigating factors such as: the uses, advantages, limitations, effectiveness, challenges and features of augmented reality in educational settings. Personalization for promoting an inclusive learning using AR is also a growing area of interest. This paper reports a systematic review of literature on augmented reality in educational settings considering the factors mentioned before. In total, 32 studies published between 2003 and 2013 in 6 indexed journals were analyzed. The main findings from this review provide the current state of the art on research in AR in education. Furthermore, the paper discusses trends and the vision towards the future and opportunities for further research in augmented reality for educational settings.
Thesis
Down Syndrome causes a reduction in cognitive abilities in the person, being the viso-motor one of the most sensitive, and from which better learning has been achieved. The general research of this report focuses on this skill, and from it has been able to stimulate better learning. The proposal is based on the visual-spatial stimulation of children with DS through didactic resources with support in the gestural interaction platform Tango: H. For this, the research question has been posed: is the psycho-cognitive development with stimuli through playful teaching strategies with gestural interaction in children with Down Syndrome greater than that achieved with conventional didactic resources? and for their response, three independent studies have been proposed in terms of form and method but related continuously to the objectives and study population. They refer to: a) stimulation of working memory, validating the memory of learning objects b) stimulation of reading skills and their learning, validating specific knowledge errors and response times; c) visuo-motor cognitive aptitudes, validating the visuo-motor memory capacity from the ITPA test. In all three cases, the results have been favorable to stimuli from the platform of gestural interaction Tango: H and with personalized didactic resources. In addition, the existing relationships between the study carried out and the previous results obtained by the international scientific community have been established. In each study the limitations of the study are also commented, as well as the possible future lines of work that arise from all this research
Article
Individuals with autism spectrum disorders often benefit from technology-based intervention. Technologies being marketed to the autism community, and relevant published research, are proliferating. However, in the context of research in health and education, the requirement for an effective design process is not necessarily recognised. Understanding this process is necessary to facilitate recommendations about best practice in technology design and implementation where the end result is being applied in a health or education context. This report describes the development of an iPad app designed for very young children with autism. We describe methods for user-centred design with relevant stakeholders, expert evaluation and pilot testing of demo versions of the app, and their consequences for the finished game. In a final evaluation with 41 pre-schoolers with autism, average game play over a 2 month period was 11 minutes per day, with no evidence of obsessive behaviour. We discuss how this approach permits individual studies to inform the design of multiple technologies, contributing to dissemination of high standards in how therapeutic and educational technologies for specific populations are designed, pilot-tested and reported.
Conference Paper
Down syndrome, the most prevalent chromosome disorder in mankind, occurs approximately in one per thousand infants born per a year. Also, life expectancy of people suffering from this irregularity has increased from 25 to 59 in the last decades. Recognizing such patients in critical and high security places like security gates could assist responsible people to make proper decisions. This irregularity causes a private facial view which differentiates regular people from patients. In this study, we have proposed a novel framework, which uses first and second order directional derivative local binary pattern (LBP) histograms for texture description then applies the support vector machine for classification, in order to distinguish Down syndrome population from healthy one. We have investigated and compared two methods for texture description: one method utilizes only first order directional derivative LBP and the other benefits from both first and second order directional derivative LBPs. The histogram bins values obtained from the mentioned descriptors have been used for training the support vector machine to classify Down and not Down population. The proposed approach has been implemented using a custom database collected from free web resources. Experimental results show PPV, NPV, sensitivity and specificity factors equal to 92.35%, 96.50%, 96.66% and 92% in the best case, respectively.
Article
The use of Information and Communication Technologies (ICT) for students with special learning needs is being particularly studied by the research community. Likewise, music has gained recognition through the years in the field of learning disabilities, playing lately a key role in providing a better quality of life to people with special needs. However, there is still a gap on the combination of ICT and music teaching tools. In this work, we present the results of a case study where students with disabilities took a 3-month introductory music course using the multimedia tool called PLAIME (PLAtform for the Integration of handicapped children in Music Education) developed by the same authors. The aims of this work are three-fold: i) to explore the capability of disabled students to acquire musical knowledge using PLAIME, ii) to measure the improvement in their ICT skills, and iii) to observe their behavior along the study. At the end of the program, students showed an advance in their music knowledge and a positive improvement in behavioral development, being able to perfectly manage the computer platform.
Article
This article presents a review of the design methods and techniques that have been used to involve children with special educational needs and/or disabilities (SEND) in the technology design process. Situating the work within the established child-computer interaction research sub-field of participatory design, we examine the progress that has been made in relation to the participation of this specific child population. An extensive review of the literature in this area has been undertaken and we describe the different roles, responsibilities and activities that have been undertaken by both the child and adult participants within previous technology design projects. We also highlight the different types of outcome from this previous work involving children with SEND, exploring the impact the children’s participation has had on both the resulting technology as well as the impact on the child participants themselves. Finally we conclude this review with a set of reporting recommendations for technology designers and researchers aiming to involve this population in future technology design projects.