Content uploaded by Wanda Benesova
Author content
All content in this area was uploaded by Wanda Benesova on Apr 02, 2023
Content may be subject to copyright.
Augmented reality to promote collaborative and autonomous learning
in higher education
Jorge Martín-Gutiérrez
a,
⇑
, Peña Fabiani
a
, Wanda Benesova
b
, María Dolores Meneses
c
, Carlos E. Mora
a
a
University of La Laguna, Faculty of Engineering and Technology, Avda. Astrofísico Francisco Sánchez sn, 38200, La Laguna, Spain
b
Slovak University of Technology in Bratislava, Faculty of Informatics and Information Technologies, Ilkovic
ˇova, 2, SK-84216, Bratislava 4, Slovak Republic
c
University of La Laguna, Faculty Communication Sciences, Avda. Cesar Manrique sn, 38071, La Laguna, Spain
article info
Article history:
Available online 18 December 2014
Keywords:
Augmented reality
Engineering education
Collaborative learning
Interactive learning environments
abstract
The learning scenarios described in this work reach further than any previous approach. The connections
between augmented reality (AR) and traditional learning based on textbooks through the well-known
augmented books also known as ‘‘magic books,’’ are already there. However, they are restricted to just
a few isolated uses that commonly take place on a PC showing 3D information with few actions in higher
education. In a collaborative and autonomous way, this work combines every learning process from the
electrical machines course in the electrical engineering degree. It allows interactive and autonomous
studying as well as collaborative performance of laboratory practices with other students and without
a teacher’s assistance. Tools presented in this work achieve a connection between the theoretical expla-
nations and the laboratory practices using augmented reality as a nexus. Students feel comfortable about
it and consider that tools are nice, easy, and useful, according to the goal of learning contents, training on
performance, and design of installations and machines.
Ó2014 Elsevier Ltd. All rights reserved.
1. Introduction
This past decade has been the time when all information and
communication technologies (ICTs) have been extended to every
field of our society, and of course in the learning field where there
have been abrupt changes in teaching methodologies, as well as in
teaching resources used in the learning process. ICTs are presented
as a tool associated with the actual social context where the need
of access to information anytime and everywhere, the quick tech-
nological changes, deeper social knowledge, and demands of a high
level education, which is constantly up to date, becomes a perma-
nent demand.
Right now, education and teaching institutions try to avoid tra-
ditional teaching methods despite their validity and successful
results, as the interest now focuses on more productive methods
that may improve the learning experience and the students’ intel-
lectual level. Computer technologies have provided a strong
improvement according to educational tools, allowing develop-
ment of new teaching methodologies. During the last few years,
the educational institutions from all levels have tried to evolve
by integrating and using ICTs in teaching methodologies for
improving the teaching–learning processes. Many universities
have adopted virtual learning environments (VLEs) for helping in
the teaching process. Following this trend, Pan, Cheok, Yang, Zhu,
and Shi (2006) have already demonstrated that virtual learning
applications may provide the adequate tools that allow users to
learn in a quick and efficient way, interacting with virtual environ-
ments. Both learning environments and computer tools have
enjoyed good feedback from students and teaching staff. Those
students may be considered as digital natives, because in their
ordinary life they are constantly interacting with a lot of graphic
information provided by videogames, the Internet, or 3D movies.
This fact causes many researchers, teachers, and pedagogues to
focus eagerly on new visualization methods for improving the cur-
rent teaching models. One of the most promising technologies that
currently exist is augmented reality (AR), which allows a combina-
tion of real world elements captured through a camera with multi-
media elements such as text, images, video, or 3D models and
animations. Computer Supported Collaborative Learning is a peda-
gogical approach that can be used for deploying educational apps
based on augmented reality in higher education. Collaborative
learning is a method applied to learners for performing common
tasks in small groups in order to reach shared goals or learning
results (Heejeon, 2011), which is introduced as a learning strategy
http://dx.doi.org/10.1016/j.chb.2014.11.093
0747-5632/Ó2014 Elsevier Ltd. All rights reserved.
⇑
Corresponding author at: Av. Angel Guimerá Jorge, sn, CP. 38202, La Laguna,
Tenerife, Spain. Tel.: +34 657282142.
E-mail addresses: jmargu@ull.edu.es (J. Martín-Gutiérrez), mfabiani@ull.edu.es
(P. Fabiani), vanda_benesova@stuba.sk (W. Benesova), dmeneses@ull.edu.es
(M.D. Meneses), carmora@ull.edu.es (C.E. Mora).
Computers in Human Behavior 51 (2015) 752–761
Contents lists available at ScienceDirect
Computers in Human Behavior
journal homepage: www.elsevier.com/locate/comphumbeh
that supplements issues found in the traditional learning environ-
ment since it presents an opportunity for students to experience
enriching interactions and to participate in active learning.
Researchers have pointed out that collaborative learning is where
the greatest potential of the AR (Billinghurst & Kato, 2002;
Kaufmann, 2003). Collaboration occurs when learners are involved
with social interactions, which would result in improved learning
capabilities. In the AR where the virtual and physical worlds coex-
ist, users learn while communicating with others in the same
space. This naturally leads to collaborative interactions (Park,
Jung, & You, 2015). Mobile devices, particularly smartphones, are
an ideal platform for the collaborative AR. Billinghurst and Kato
(2002) describe the main characteristics of collaborative AR.
Azuma (1997) defines AR as a variation of virtual environments
(VR). VR technology completely immerses the user in a synthetic
environment, which can interact with obtaining answers, while
not seeing the outer real world. However, an augmented reality
environment allows the user to see the real world with virtual
computer-generated objects superimposed or merged with real
surroundings. In terms of used technology, AR can be said to
require the following three characteristics: it combines the real
and virtual, it is interactive in real time, and registered in 3D.
Among the most innovative tools for virtual education used in
higher education have been the development tools from virtual
worlds on education (Lucke & Zender, 2011). The virtual environ-
ments allow students to create an avatar and train, learn or manip-
ulate virtual objects. The virtual world experience emulates the
experiences and items from real life, but AR technology allows
the coexistence of virtual elements in real environments, so inter-
action between objects is completely real (Saleeb & Dafoulas, 2011;
Schiller, Mennecke, Nah, & Luse, 2014).
According to The New Media Consortium’s 2011 Horizon Report
(Johnson, Smith, Willis, Levine, & Haywood, 2011) augmented real-
ity is becoming a technical trend in higher education for making
technology blend virtual and real worlds, and is expected to reach
mainstream use in education through augmented reality textbooks
(augmented book).
Today, one of most relevant changes in our society is augmented
reality, which is a technology that is being developed in several
fields and applied to medicine, architecture, marketing, advertising,
military, archeology, leisure, etc. (Craig, 2013). The versatility
offered by AR technology has allowed the development of applica-
tions for several knowledge areas of education like mathematics,
mechanic, physics, and town planning, among many others. The
work of Ibáñez, Serio, Villarán, and Delgado (2014) is an experience
based on AR learning about the basic principles of electromagne-
tism. Although it was a close approach to our contribution, our
work context goes even further, promoting real learning in both
collaborative and autonomous learning. Physics Playground is an
interesting tool developed by Kaufmann and Meyer (2008) for
explaining physical experiments and concepts through animations,
where the student has the chance to interact with virtual objects
and practice with them to learn in a fun and entertaining way. Over
time there have been more teaching tools with augmented reality
technology, such as training of spatial abilities by Martín-
Gutiérrez et al. (2010), and the training for future anesthetists using
operating theater material through an AR simulation (Quarles,
Lampotang, Fischler, & Fishwich, 2009).
According to Bujak et al. (2013) this technology creates possibil-
ities for collaborative learning around virtual content in non-
traditional environments. Besides, the authors of this work provide
guidelines for future AR learning experiences from the analysis of
existing AR applications, considering its pragmatic and technologi-
cal concerns facing the widespread implementation of augmented
reality inside and outside the classroom. Cuendet, Bonnard,
Do-Lenh, and Dillenbourg (2013) starting from the premise that
classroom usability increases if the learning environment satisfies
all classroom constraints, proposes a design of material and
resources adapted to classroom based on AR for performing teach-
ing duties following collaborative learning guidelines. Authors such
as Kerawalla, Luckin, Seljeflot, and Woolard (2006) made reference
to the ‘AR for learning’ term proposing several design requirements
which may be considered: (1) AR systems should be flexible enough
for the teacher to adapt to the needs of their students; (2) the con-
tent should be taken from the curriculum and delivered in periods
as short as other lessons; and (3) the system should take into
account any constraints of the context. Goals of our contribution
include considering these three requirements, applying it to three
different teaching contexts: use of electrical machines at the labo-
ratory, professional use (reading and interpreting diagrams for
inspecting installations) and autonomous study of contents.
In higher education, some AR experiences have been performed
already but they have not generated any didactic material for con-
tinued use. We can just mention a few collaborative learning stud-
ies about land and town planning fields (Chen & Wang, 2008;
Fonseca, Martí, Redondo, Navarro, & Sánchez, 2014). These experi-
ences concluded that AR technology may improve the design of
tasks performed by students and their academic performance.
In the context of collaborative AR learning is worth mentioning
the experiences from the chemistry and molecular biology teach-
ing field (Gillet, Sanner, Stoffler, Goodsell, & Olson, 2004; Cai,
Wang, & Chiang, 2014).
Luckily, university classrooms have been updated, giving them
the infrastructure needed for using the most suitable teaching
technologies such as internet networks, computers, electronic
blackboards, projectors, and videoconference systems. Every one
of these technologies could allow the integration of augmented
reality inside the classrooms; in fact, research has shown that
learning does occur in virtual environments (Harrington, 2006).
One of the earliest works in this area, applying AR to an educa-
tional context, is the ‘‘Classroom of the Future’’ (Cooperstock,
2001), which conceptualizes how it could be possible to enhance
interaction between instructor and students to interact through
various interactive scenarios in a collaborative environment.
Augmented reality can also be used to enhance collaborative
tasks. A good example is this work, as it allows several users to per-
form tasks together. In this paper, actions described have been per-
formed with 50 engineering students from the electrical machines
course. Through augmented reality technology, the students were
able to perform training of the use of dangerous machines in a safe
way, checking the virtual information associated with the symbols
on the diagrams and electrical installations, and study with notes
upgraded by virtual information provided by the teacher. Besides,
it should be noted, that the use of new technologies increases stu-
dent motivation, although that is not the goal of this work. Any
methodology that captures the interest and enthusiasm of the stu-
dents improves their performance.
In the following section, we present recent didactic materials and
experiences about the application of augmented reality in higher
education contexts. After that, the observed results and feedback
surveys are analyzed. Finally, conclusions are presented as well.
2. Augmented reality applied for training and education in
electrical engineering
Since the creation of the European Higher Education Area in
2010, university education across Europe, especially in Spain, has
undergone a deep transformation within the new European frame-
work, regarding the structure, methodology, and the philosophy of
technical education. In this context of renewal and convergence,
there is a new concept of the European Credit Transfer System,
which measures not only classroom teaching hours, but also the
J. Martín-Gutiérrez et al. / Computers in Human Behavior 51 (2015) 752–761 753
total amount of work needed to meet the goals of the program of
study (EHEA, 2010). The changes clearly show a new philosophy
aimed at customized teaching and encouraging autonomous work
of the students. It implies a deep change in the Spanish educational
system and requires a new teaching methodology based on new
strategies and objectives. The reform promotes ‘‘tutored’’ work
rather than ‘‘master’’ teaching. At the same time, there has been a
large increase of students in Spanish universities due to the existing
economic crisis in the country and the high unemployment rate.
This situation is particularly severe in technological careers where
the teacher/student ratio has not been increased (as would be
expected from the reform), but reduced due to the growing demand
for these sorts of degrees.
This fact has a negative influence over the attention paid to stu-
dents and the quality of teaching. The problem is especially notice-
able in the practical teaching of these subjects, where student
mentoring and supervision is much more necessary, and learning
must be completely personal and manipulative. Therefore, we are
aware of the need to seek new ways of teaching in electrical engi-
neering practice, and are looking for more efficient alternatives to
minimize the problems we face in the current context.
Given this new challenge, teachers have to take advantage of
the tools that new technologies may provide, such as Virtual
Teaching and augmented reality (AR). The teaching of engineering
is particularly suitable for the use of these technologies. We have
experienced some Virtual training in the area of engineering teach-
ing. We have used Virtual Simulators for the study of electrical
machines. Virtual reality is a good way to increase the knowledge
acquired by students; but for a future engineer, real practical learn-
ing is essential when dealing with their future professional chal-
lenges. In that sense, the AR teaching opens new possibilities
because it allows us to combine the real and virtual worlds,
increasing the autonomy of students while maximizing the time
and resources available.
Focusing on the electrical engineering branch, we have imple-
mented different applications of AR in order to improve several
aspects and needs of the students in this field. We understand that
engineering education should include both theoretical and practi-
cal aspects, and in the following sections, three applications that
have been developed and used to achieve this goal are described.
3. Material
3.1. ElectARmanual: AR training for installations and electrical
machines practice
The authors have developed an educational augmented reality
application called ElectARmanual (Martín-Gutiérrez, Fabiani,
Meneses-Fernández, & Pérez-López, 2012), which is supposed to
support students in a practice laboratory and then in training for
use of electrical machines (Fig. 1).
The application is an assistant, which guides the student step by
step through the tasks that he may perform in order to understand
instructions and explanations of the practice’s manual provided by
the teacher in the laboratory. An animation of 3D models is superim-
posed over the main panels at the workplace indicating how to con-
nect the wires, and place several components (coils, magnets, rotor,
wide pole pieces, etc.) for creating installations of several sorts, cre-
ating configurations of electrical machines with different purposes.
For visualizing each sequence, the user will press a key from the lap-
top or press the ‘‘next’’ button on tablet or smartphone’s screen.
The Electrical Machine Laboratory has four independent work-
place ranging from protection systems, analysis, construction and
operation of different types of electrical machines, to industrial
Fig. 1. User visualizing the training she is about to perform on AR.
Fig. 2. Workplaces in the electrical laboratory.
754 J. Martín-Gutiérrez et al. / Computers in Human Behavior 51 (2015) 752–761
electrical equipment—training in electrical machines’ automatic
control (Fig. 2).
Electrical protection (Masterlab-5000, by 3E Electronic didactic
material).
Construction and study of electrical machines (TPS 2.5, by Ley-
bold Didactic GmbH).
Performance and operating characteristics of electrical
machines (Lucas-Nülle GmbH).
Coach of industrial electrical installations – training in electrical
machines automatic control (Masterlab-3000, by 3E-Equipos
Electrónicos Educativos, S.L.).
ElectARmanual has been implemented for performing practice
or training over the mentioned equipment. These practices are
devoted to students in the university degree’s first courses, and
who are without any previous experience in this subject. Due to
the laboratory’s structure, it is hard for the supervisor to perform
all teaching and control duties because different groups should
develop quite different practices at the same time. The traditional
practices model is not efficient, since a single teacher has to guide
and teach 25 students, conducting very different practices simulta-
neously. As a result, direct mentor time spent by the teacher for
each student is greatly reduced. There is also the danger of this
type of practice, where students must manipulate actual pieces
of an engine with high working voltages. Use of ElectARmanual pro-
vides the students with a positive attitude and autonomy while
training, and it also reduces the teacher
´s dedication to every stu-
dent, thus improving safety in the laboratory.
3.1.1. ElectARmanual description
To develop this app, we have worked with researchers from
Labhuman institute (www.labhuman.com) to create a software
library called HUMANAR (Martín-Gutiérrez et al., 2010). It uses
computer vision techniques for calculating the real camera view-
point relative to a real world marker, which calculates integration
of three-dimensional objects codified by the camera and captured
by itself in real time. When the marker enters the scene picked up
by the camera, the fusion of the real world with the virtual object is
shown on the screen. This requires the application to relate the two
worlds (real and virtual) in a single system of coordinates. The key
technical issues for the development of the AR practical guide have
been marker detection, camera calibration, calculation of marker
position and orientation, augmentation of virtual object.
Our AR software identifies fiducial marks. Each mark is a black
frame image containing a white surface divided into 4 rows and 4.
We have 16 squares where black and white colors will define the
individual mark. Using the binary base of each cell, the mark can
be converted into a hexadecimal base digit that will be read by
the software. Our augmented reality software identifies the marker
through a decimal association, and that is why this hexadecimal
code of 4 digits should be transferred to decimal notation. Finally,
we associate the decimal code belonging to a mark with a sequen-
tial description of a practice, including some 3D model files in any
of these formats: FBX, MDL, VRML, and OBJ.
The application is programmed so a different 3D model’s
sequence is superimposed over the real machine to show the
instructions and explanations in the practice’s manual. For visual-
izing each sequence the user will just press a key from the laptop.
While executing the application, a menu is shown where student
may choose between eight different training sessions (two train-
ings per workplace). Each panel front of workplace has a different
mark, so each mark defines two different trainings. The fiducial
mark is the activator of the virtual elements from the chosen train-
ing in the menu (Fig. 2).
The trainings of electrical machines performed on ElectARman-
ual are based on alternating current (AC) and direct current (DC)
generators with permanent magnets and separate exits as well as
single-phase motors (synchronous and asynchronous). The main
panel allows assembly of different kinds of electrical machines.
3.1.2. Virtual information
The 3D Studio software has been used to create the models and
3D animations that we want to incorporate into the real scene
(Fig. 3). The information has been saved in MDL format, as this for-
mat is compatible with the graphic engine of the GAME STUDIO A8
software, which is the one over that our augmented reality graphic
viewer will work.
Fig. 3. Some 3D models of pieces for training with electrical machines.
J. Martín-Gutiérrez et al. / Computers in Human Behavior 51 (2015) 752–761 755
3.1.3. Interface devices
The code uses Windows platforms. The students can use PC
devices or a tablet for visualizing the virtual objects in the practice.
The application has also been tested by using a head mounted dis-
play (HMD) (Martín-Gutiérrez et al., 2012). The use of this HMD
device is optimum for this kind of work, where the students need
to manipulate objects. However, it is not the best choice for great
numbers of students given that is quite expensive equipment (Fig. 4).
3.2. ELECT3D: App for electrical plans reading
This AR application allows the students a more realistic and
useful learning of electrical engineering and its symbols. Future
engineers must be able to elaborate and understand engineering
projects, which implies the comprehension and understanding of
electrical symbols. To achieve this goal, the application must inter-
pret both complex and realistic symbols and images, then show the
corresponding virtual information.
The application we have developed is both realistic and useful;
it can be used on any electrical diagram, professional or academic,
because of the normalized standard symbols used. On the other
hand, it uses an extended library of symbols and objects, so this
application can be used not only in the electrical engineering field,
but also in any other field that uses electrical normalized symbols,
thus allowing its proper understanding and comprehension via AR.
3.2.1. Marker-less system
Unlike the previously described applications, ELECT3D does not
need fiducial markers to connect the real and the virtual worlds.
The system can understand any symbol or image predefined with
a cloud of points, e.g., normalized symbols, which are widely used
in the industry. Our application can use standard electrical plan
drawings or complex electrical circuit diagrams.
Fiducial markers-based AR applications are limited to previ-
ously defined and controlled environments (where defined marks
have been introduced). However, a marker-less system can be used
in any environment as well as in real situations where normalized
symbols are used (which is usually the case). Students can use not
only the former academic material, but also real work develop-
ments (Fig. 5).
3.2.2. ELECT3D Software description
Free and open source software (Metaio’s SDK -software devel-
opment kit) has been used for developing this AR app. The main
advantages of this software are the following:
Any (⁄.png) reference images can be used, not only predefined
ones.
Detection method is performed through a cloud of points (not
looking for symmetries), allowing customized tracking configu-
ration. Therefore, the level of sensitivity can be adjusted to our
application’s needs.
The speed and stability are quite good, depending on the size
and characteristic of the library.
Software is easy-to-learn, and good tutorials are provided, mak-
ing it a suitable choice for students with no previous AR experi-
ence. Students can even develop the application by themselves.
3.2.3. Virtual information
With this project, we have tried to make the included virtual
information as realistic as possible. To this end:
We have selected elements commonly used in electrical appli-
cations, with preference for those that are used in the labs
and those most widely used in low voltage electrical
installations.
We have made realistic models of the most usual electrical
elements.
We have associated the above with the electrical symbols avail-
able to the students in the workplace. As already mentioned,
only elements with standard symbols were selected.
For the simplest elements we use 2D images (⁄.obj), whereas for
more complex elements we perform and incorporate 3D images,
whose rotation allows more detailed information about the object.
This reduces the memory requirements, because we only use ‘‘vol-
ume elements’’ when it is necessary.
The elements selected for their complexity are video recorded.
From these videos we create the 3D models library with ‘‘Video-
Trace software’’ (Van den Hengel, Dick, Thormählen, Ward, &
Torr, 2014). The video recording and modeling was performed by
students who participate in a ‘‘Teaching Innovation’’ project at
the University of La Laguna (Fig. 6).
Fig. 4. User undertaking training with HMD.
Fig. 5. Users making use of ELECT3D. Reading of plan drawings.
756 J. Martín-Gutiérrez et al. / Computers in Human Behavior 51 (2015) 752–761
We have complemented the 3D images by a text file, which
includes the object name and a short description of its operation
and usual application. Finally, this ‘‘image + text/symbol’’ library
is integrated onto the AR software so the user can choose between
just watching the image and increasing information; by observing
a particular symbol you get an image, a text, or both by just touch-
ing the device
´s screen.
3.2.4. Interface devices
The philosophy for ELECT3D is easily accessible through porta-
ble devices, such as smartphones and tablets. We have selected
Android as the most widespread operative system in these devices.
The Elect3D users can download the application to their own
devices, where it is accessible at all times.
3.3. ElectAR_notes: Theoretical electricity notes enriched with AR
contents
For complementing actions developed through previous sec-
tions, we have included AR in the notes of several electrical engi-
neering topics (app ElectAR_notes). With the inclusion of 3D
players, videos, explanations and animations, our main goal was
to improve the spatial perception and theoretical comprehension
of difficult-to-understand concepts, such as the generation and
behavior of electrical and magnetic fields inside the electrical
engines. AR also contributes by dismissing one of the weaknesses
detected during the validation study of the practical app previously
described. We get a better connection between theoretical and
practical teaching. It allows the students to relate the laboratory
learning with the theoretical knowledge (Fig. 7).
3.3.1. Notes book description
The name of the developed notes book is Basic Electrical
Machines, and includes basic theoretical contents on electromagne-
tism, ferromagnetism, and operating principles of electrical
machines. We have added AR contents on both text and images.
This ‘‘classical/AR’’ combination is aimed, on one hand, at improv-
ing the understanding of the contents with more realistic and
detailed explanations, and on the other, at increasing the interest
level of the students with the use of more interactive notes.
We have chosen topics that are usually not clearly reflected in
written text or 2D images included on notes. First, when working
with vector quantities, concepts such as the rotation’s direction,
the movement, or the machine components, requires special 3D
vision that cannot be introduced into traditional notes, but,
however, is easily explained by three-dimensional contents and
animations (Fig. 8). Hence it is difficult for students to find the con-
nection between the theoretical contents described in the books
and the reality of the practice they are carrying out in the lab. With
the introduction of AR elements we can combine theory and prac-
tice, and students will find it easier to relate abstract contents with
real situations.
3.3.2. Virtual information
As just mentioned, we have included virtual files with three dif-
ferent formats in the library: 3D images, video files, and audio con-
tents. The 3D modeling is something completely new for students
accustomed to traditional 2D photo books, and allows the student
to visualize the elements from different angles and perspectives,
broadening their particular vision of the problem. They can provide
more information about objects and processes, and they can also
be manipulated (rotate, zoom, etc.). To make 3D models we have
used two softwares: Autodesk Inventor and Cheetah3D. The
explanatory videos have been performed through collaboration
with the audio-visual service from La Laguna University
(ULLmedia, 2014), composed by professionals from the audio-
visual field. The teacher himself through Camtasia Studio software
performed some simple videos.
We have created and included two kinds of videos for this appli-
cation: recorded videos and 3D virtual animations. The first kind
was recorded with small laboratory experiments and practices that
relate theory to reality, showing the explained processes from a
practical point of view. The material used for these videos is the
same one the students will use later on in the laboratory practical
teaching, so they can become used to it.
Furthermore, we have performed small virtual animations for
viewing invisible abstract concepts in the real world (this is pre-
cisely one of the main strengths of the AR technology). The role
of the magnetic field in the electrical power generation, the inter-
action of ferromagnetic materials with electrical power generation,
the interaction of ferromagnetic materials with electrical currents,
or the electromagnetic forces that explain the operation of electri-
cal engines are some examples of abstract contents. The anima-
tions have been made using Autodesk inventor and Cheetah3D.
Besides being able to visualize virtual contents, it is also possible
to listen to audio explanation files that the teacher added to the text.
This range of actions complemented by augmented reality intends
to increase the interaction between teacher and students, giving a
more direct and dynamic explanation to some complex concepts.
3.3.3. Software, marks, and interface devices
The ElectAR_notes app was aimed to develop versatile and useful
study notes for use in engineering education. The application has
been developed for Android, iOS and Windows OS, aiming to ease
access to augmented contents for students while they study the
theoretical contents.
The pages of study notes are accompanied by a ‘‘double AR
Mark,’’ so it can be used with all kinds of devices, regardless of their
operating system. This way, students can use the didactic material
either on a PC/tablet or with their own mobile device. The Win-
dows app identifies fiducial marks, and it has been developed with
the HUMANAR software library. The mobile app was created with
the Metaio Creator software, and it can identify any symbol or
image. The software and marks in this app are similar to those used
in the previous section.
4. Method
The prototypical solution of the presented system was devel-
oped in the Basic Electrical Engineering course, the first course of
Fig. 6. Recording real objects for obtaining the 3D models using VideoTrace
software.
J. Martín-Gutiérrez et al. / Computers in Human Behavior 51 (2015) 752–761 757
Fig. 7. User studying notes through ElectAR_notes.
Fig. 8. Notes with fiducial marks (for PC use) and marker-less (for smartphone use).
758 J. Martín-Gutiérrez et al. / Computers in Human Behavior 51 (2015) 752–761
the electrical engineering degree. The students were able to study
the provided notes from the teacher, visualizing virtual contents
associated with notes through their mobile devices. The virtual
contents associated with notes are related to work developed in
the practice laboratory. Furthermore, two other applications are
used for reading the electrical plan drawings and training in build-
ing and manipulation of electrical machines. The assessment of the
three applications took place with 50 students of electrical
engineering.
4.1. Technical aspects
The course has six groups of 25 students each. A group was cho-
sen at random for using the three learning support applications
during the semester between February and May. The goal is to
explore the usability of the applications as well as feedback from
students about their use. The System Usability Scale (SUS) ques-
tionnaire was used for measuring usability (see questions in
Table 1), as well as a feedback survey that took place ad-hoc by
the authors (Table 2), applied to 25 students for assessing the
degree of stability and satisfaction achieved by every application.
This group is considered as group 1. Once the course was over
(in May), there was a demonstration of the three applications to
other group of 25 students belonging to the same course (group
2). After the practical demonstration, these students have also
completed the usability and feedback surveys.
The SUS usability questionnaire, originally created by John
Brooke in 1986, allows evaluation of a wide variety of products
and services, including hardware, software, mobile devices, web-
sites, and applications. It enjoyed great feedback as a usability
measure tool. It comprises 10 questions covering the different
aspects of a system’s usability, such as the need of support, train-
ing, and complexity so it is highly valuable as a tool for measuring
the usability of a certain system. The answers cover a Likert scale
ranging from 1 (completely disagree) to 5 (completely agree). The
questionnaire was applied after the survey subjects had the
chance to use the system under assessment. The instant answers
for each item are compiled without allowing time to think
about it, aiming to capture the user’s first impression. This
questionnaire’s particularity is that if the participant does not feel
able to answer a certain item, he may choose the center value on
the scale (3).
The SUS questionnaire is integrated by two subscales: odd and
even questions with different weighting systems. The final result of
the questionnaire is a unique percentage value that denotes a mea-
sure composed by the ease of use from the studied system, given
that scores for individual items are not significant by themselves.
For calculating the SUS score, first we must add each item’s
scores. The contribution from each item to the final score is calcu-
lated as follows:
For items 1, 3, 5, 7 and 9 the score is that which is provided by
the user minus one.
For items 2, 4, 6, 8 and 10, the score is that which is provided by
the user minus five.
The average score obtained by each item is also added.
The final score is obtained multiplying previous addition by 2.5.
The final result will be between 0 and 100.
The students were provided with the download link for install-
ing the applications on their smartphones or laptops. They were
free to use them at the laboratory and for further practice with
electrical plan drawings and installations.
5. Results
The results from the usability survey are compiled in Table 1 for
all three applications performed by both groups of students, the
ones who enjoyed the four-month experience, and the ones who
only received a demonstration.
The usability results offered very high scores according to ease
of use. The survey was performed on both groups (those using it
frequently and those using it just once) for knowing both percep-
tions. The result is analogue in both cases so the score obtained
for the three applications is around 80%. A product’s usability is
considered acceptable for values higher than 55%. Regarding the
feedback survey, it was only completed by students who used
the AR applications during the study (group 1). The survey has
eight answers (items A1–A8). The students evaluate each applica-
tion using the same survey.
Table 1
Usability questionnaire. Weighted Value (WV).
Questions ElectARmanual (WV) ELEC3D (WV) ElectARnotes (WV)
Group 1
n=25
Group 2
n=25
Group 1
n=25
Group 2
n=25
Group 1
n=25
Group 2
n=25
1 I believe I will use this system frequently 4.76 (3.76) 4.68 (3.68) 4.8 (3.8) 4.72 (3.72) 4.68 (3.68) 4.38 (3.48)
2 I regard the system as unnecessarily complex 1.16 (3.84) 1.28 (3.72) 1.129 (3.88) 1.2 (3.8) 1.24 (3.76) 1.44 (3.56)
3 I thought the system was easy to use (2.12) 1.12 (2.2) 1.2 2.08 (1.08) 2.16 (1.16) 2 (1) 2.08 (1.08)
4 I believed I would need technical support for using this system 3.93 (1.08) 3.84 (1.16) 3.88 (1.12) 3.84 (1.16) 3.72 (1.28) 3.64 (1.36)
5 It seemed to me that several features in this system are well
integrated
4.88 (3.88) 4.8 (3.8) 4.8 (3.8) 4.68 (3.68) 4.76 (3.76) 4.6 (3.6)
6 I believe there are many inconsistencies in this system 1.16 (3.84) 1.2 (3.8) 1.16 (3.84) 1.32 (3.68) 1.28 (3.72) 1.4 (3.6)
7 I think most people will learn to use the system quickly (4.92) 3.92 4.96 (3.96) 4.76 (3.76) 4.64 (3.64) 4.72 (3.72) 4.64 (3.64)
8 I regard the use of the system as bothersome 1.2 (3.8) 1.28 (3.72) 1.28 (3.72) 1.36 (3.64) 1.32 (3.68) 1.56 (3.44)
9 I felt quite safe using the system 4.76 (3.76) 4.8 (3.8) 4.96 (3.96) 4.84 (3.84) 4.72 (3.72) 4.6 (3.6)
10 I had to learn many things before being able to use the system 1.08 (3.92) 1.04 (3.96) 1.24 (3.76) 1.36 (3.64) 1.4 (3.6) 1.56 (3.44)
Total 32.92 32.8 32.72 31.96 31.92 30.8
Addition
⁄
2.5 (%) 82.3% 82% 81.8% 79.9% 79.8% 77%
Table 2
Feedback survey.
This survey reflects my feedback about the following application:
hElectARmanual hELECT3D hElectAR_notes
A1 Rate quality from 3D models and animations
A2 Rate quality from 2D models and videos
A3 Application easy to use
A4 Application helps my learning, improving my understanding of contents
A5 I regard this application as useful in the teaching field for this subject as
well as for others
A6 Rate your experience of use about this application
A7 I believe that the application helps mental visualization of abstract
contents
A8 The application can be used anywhere and anytime due to its portability
J. Martín-Gutiérrez et al. / Computers in Human Behavior 51 (2015) 752–761 759
Table 2 shows the formulated questions for finding out the sat-
isfaction of the students using each application. The evaluation
takes place through a 5-point Likert scale.
The results from the three applications are quite similar in
terms of the feedback from students, indicating high scores. The
multimedia content is regarded as a high quality one (A1–A2),
and the application does not show any unforeseen difficulty about
its use, which is quite easy (A3).
The students consider without any doubt that these three appli-
cations are a great help for learning, and they could be used in
other similar subjects (A4–A5). The personal experience while
using it is quite satisfactory as well, showing high values above
average (A6).
The perception of the participants is that tools help to
understand the abstract concepts from electrical engineering
(A7). They consider that both ELECT3D and ElectAR_notes tools
are portable, while the ElectAR_manual is not regarded as such,
due to its availability at the practice laboratory only (A8) (see
Table 3 and Fig. 9).
6. Conclusions
The increasing number of students that opt for studying engi-
neering degrees makes the practice laboratories overcrowded,
worsening the teaching quality and reducing the teacher’s dedica-
tion to every student. Besides, learning and teaching procedures
need to evolve and take into account the high technological profile
that most students show. In some cases, outdated teaching creates
barriers for some students that are used to interacting with
modern technological gadgets and computers.
AR applications allow that in certain teaching/learning contexts,
they can be performed by the student on his own, thus saving tea-
cher’s time spent on repeating explanations. The students gladly
welcome this technology, so a well-planned AR application will
allow them to successfully perform any learning processes.
The tools developed in this work have achieved a dual effect
as they allow the teacher to improve guidance at the training
sessions within the practice laboratory, and to offer attractive
and motivational tools to the student during the learning process
of contents.
AR applied to different learning contexts in the framework of
this work provides proper methods for developing professional
competences from the contents used in this subject, but besides
transversal competences are also developed such as: instrumen-
tal competences (analysis and synthesis skills, planning and orga-
nization skills, solving problems, managing information as well
as taking decisions), personal competences (teamwork, work-
place interpersonal relations skills, critical reasoning), systemic
skills (autonomous learning, leadership, initiative, entrepreneur,
motivation for quality) and others such as the skills to apply the-
oretical knowledge and put it into practice. From the point of
view of effective human resource management for business, the
contribution of training qualified and motivated professionals
for good performance allows employing the right people for
higher profitability, less rotation, higher product quality, lower
costs in manufacturing as well as faster acceptation and imple-
mentation of the organizational strategy (Lytras & Ordóñez de
Pablos, 2008).
The usability study, which the three augmented reality applica-
tions underwent, indicates that they are free of errors in terms of
effectiveness and efficacy. Also, the students left high feedback
scores for the three of them indicating that they feel comfortable
while using them, and consider them to be quite adequate for
learning both practical and theoretical content.
This paper comprises the validation experience of the three
applications that have been integrated in the electrical machines
course in the electrical engineering program. As a consequence of
the good results from the usability and feedback surveys, we rec-
ommend extending the use of these tools to other degrees where
the same subject is being taught (chemical, electronic, mechanical,
and civil engineering degrees.)
The results from the feedback survey and the student’s attitude
observed by the teacher indicate that student motivation is a key
factor, which has improved respecting other academic courses.
We believe that this motivation and attitude toward work has been
reflected over the academic performance. This fact can be conve-
niently studied upon the extension of these tools to every student
of the other subjects mentioned.
Through this empirical study, we have witnessed the great
potential and acceptance of the inquiry-based AR tools. The research
shows that different AR learning scenarios presented in this work
are adequate for promoting collaborative and autonomous learning.
In future actions, we will consider analyzing if students’ academic
performance will be enhanced by the AR learning tool and if learn-
ing is retained any longer on the memory of students using AR learn-
ing tools.
Finally, it should be underlined that augmented reality is a cost-
effective technology for providing students more attractive con-
tents than paper, so we regard as interesting the extension of this
experience to other laboratories such as mechanical and hydraulic
engineering which may help solving any faults on equipment and
physical machines for virtual equipment in order to perform prac-
tical training. The work taking place in this kind of laboratory
should allow that students have their hands free of any item or
device in order to perform them properly interacting with virtual
objects, so we considered using visualization devices at vision level
such as glasses. Augmented reality glasses from META business
start-up will be available on the second semester of 2015, show-
casing great potential (www.spaceglasses.com). Meanwhile, the
Moverio BT200 augmented reality sunglasses manufactured by
the EPSON brand (www.epson.com/moverio) are already available
to us so we will use them in the short term at the mechanical engi-
neering laboratories.
Table 3
Satisfaction survey results.
ElectARmanual ELECT3D ElectAR_notes
A1 4.1 4 4.25
A2 4.2 4.125 4.25
A3 4.6 4.75 4.9
A4 4.8 4 5
A5 5 4.375 4.8
A6 4.5 3.75 4.75
A7 4.7 4.125 4.65
A8 1 5 4.7
Fig. 9. Satisfaction results.
760 J. Martín-Gutiérrez et al. / Computers in Human Behavior 51 (2015) 752–761
References
Azuma, R. T. (1997). A survey of augmented reality. Presence, 6(4), 355–385.
Billinghurst, M., & Kato, H. (2002). Collaborative augmented reality. Proceedings of
Communications of the ACM, 45(7), 64–70.
Brooke, J. (1986). System usability scale (SUS): A quick-and-dirty method of system
evaluation user information. Reading, UK: Digital Equipment Co Ltd..
Bujak, K. R., Radu, I., Catrambone, R., MacIntyre, B., Zheng, R., & Golubski, G. (2013).
A psychological perspective on augmented reality in the mathematics
classroom. Computers and Education, 68, 536–544.
Cai, S., Wang, X., & Chiang, F.-K. (2014). A case study of augmented reality
simulation system application in a chemistry course. Computers in Human
Behavior, 37, 31–40.
Chen, R., & Wang, X. (2008). An empirical study on tangible augmented reality
learning space for design skill transfer. Tsinghua Science and Technology, 13(S1),
13–18.
Cooperstock, J. R. (2001). The classroom of the future: Enhancing education through
augmented reality. In Proceedings of the international conference on human–
computer interaction (pp. 688–692). <http://srl.mcgill.ca/publications/2001-
HCI.pdf> [Retrieved 27.01.14].
Craig, A. B. (2013). Augmented reality applications. In A. B. Craig & M. Kaufmann
(Eds.), Understanding augmented reality (pp. 221–254). USA: Boston.
Cuendet, S., Bonnard, Q., Do-Lenh, S., & Dillenbourg, P. (2013). Designing augmented
reality for the classroom. Computer and Education, 65, 557–569.
European Higher Education Area, EHEA (2010). <http://www.ehea.info> [Retrieved
28.06.14].
Fonseca, D., Martí, N., Redondo, E., Navarro, I., & Sánchez, A. (2014). Relationship
between student profile, tool use, participation, and academic performance
with the use of augmented reality technology for visualized architecture
models. Computers in Human Behavior, 31, 434–445.
Gillet, A., Sanner, M., Stoffler, D., Goodsell, D., & Olson, A. (2004). Augmented reality
with tangible auto-fabricated models for molecular biology applications. In
Proccedings of the 15th IEEE visualization conference (pp. 235–241). http://
dx.doi.org/10.1109/VISUAL.2004.7.
Harrington, M. C. R. (2006). Situational learning in real and virtual space: Lessons
learned and future directions. In Proceeding of SIGGRAPH ’06: ACM SIGGRAPH
2006 Educators program (pp. 48). ACM.
Heejeon, S. (2011). Collaborative learning models and support technologies in the
future classroom. International Journal for Educational Media and Technology,
5(1), 50–61.
Ibáñez, M. B., Serio, A. D., Villarán, D., & Delgado, C. (2014). Experimenting with
electromagnetism using augmented reality: Impact on flow student experience
and educational effectiveness. Computer and Education, 71, 1–13.
Johnson, L., Smith, R., Willis, H., Levine, A., & Haywood, K., (2011). The 2011 horizon
report. Austin, Texas: The New Media Consortium. <http://www.nmc.org/
publications/horizon-report-2011-higher-ed-edition>.
Kaufmann, H. (2003). Collaborative augmented reality in education. Position paper
for keynote speech at imagina 2003 conference, 2003.
Kaufmann, H., & Meyer, B. (2008). Simulating educational physical experiments in
augmented reality. Proceeding of the international conference on computer
graphics and interactive techniques.ACM SIGGRAPH ASIA 2008 educators
programme. New York, NY, USA: ACM. http://dx.doi.org/10.1145/
1507713.1507717.
Kerawalla, L., Luckin, R., Seljeflot, S., & Woolard, A. (2006). ‘‘Making it real’’:
Exploring the potential of augmented reality for teaching primary school
science. Virtual Reality, 10(3–4), 163–174.
Lucke, U., & Zender, R. (2011). 3D interactions between virtual worlds and real life
in an E-learning community. Advances in Human–Computer Interaction.http://
dx.doi.org/10.1155/2011/684202 [Article ID 684202].
Lytras, M. D., & Ordóñez de Pablos, P. (2008). Competencies and human resource
management: Implications for organizational competitive advantage. Journal of
Knowledge Management, 12(6), 48–55.
Martín-Gutiérrez, J., Fabiani, P., Meneses-Fernández, M. D., & Pérez-López, D. C.
(2012). Adaptation of electrical laboratory systems through augmented reality
for optimization of engineering Teaching University. Eurographics 2012-
Education Papers (pp. 3–8).
Martín-Gutiérrez, J., Saorín, J. L., Contero, M., Alcañiz, M., Pérez-López, D. C., &
Ortega, M. (2010). Design and validation of an augmented book for spatial
abilities development in engineering students. Computers and Graphics, 34(1),
77–91.
Pan, Z., Cheok, A. D., Yang, H., Zhu, J., & Shi, J. (2006). Virtual reality and mixed
reality for virtual learning environments. Computers and Graphics, 30(1), 20–28.
Park, S. B., Jung, J. J., & You, E. (2015). Storytelling of collaborative learning system
on augmented reality. In Camacho, D., Kin, S.-W., & Trawinki, B. (Eds.), New
trends in computational collective intelligence, series studies in computational
intelligence (Vol. 572, pp. 139–147).
Quarles, J., Lampotang, S., Fischler, I., & Fishwich, P. (2009). Scaffolded learning with
mixed reality. Computers and Graphics, 33(1), 34–46.
Saleeb, N., & Dafoulas, G. A. (2011). Effects of virtual world environments in student
satisfaction: An examination of the role of architecture in 3D education.
International Journal of Knowledge Society Research, 2(1), 29–48. http://
dx.doi.org/10.4018/jksr.201101010.
Schiller, S. Z., Mennecke, B. E., Nah, F. F.-H., & Luse, A. (2014). Institutional
boundaries and trust of virtual teams in collaborative design: An experimental
study in a virtual world environment. Computers in Human Behavior, 35,
565–577.
Ullmedia – Unidad para la Docencia Virtual, Universidad de La Laguna (2014).
<http://udv.ull.es/portal/ullmedia/> [last access 25.09.14].
Van den Hengel, A., Dick, A., Thormählen, T., Ward, B., & Torr, P. H. S. (2014). Image-
based modelling for all. Software Punchcard: VideoTrace. <http://punchcard.
com.au>.
J. Martín-Gutiérrez et al. / Computers in Human Behavior 51 (2015) 752–761 761
