Using Animated Augmented Reality to Cognitively Guide Assembly

Abstract

Assembly is the process in which two or more objects are joined together. An assembly manual is typically used to guide the tasks required to put together an artifact. As an emerging technology, augmented reality (AR) integrates three-dimensional (3D) images of virtual objects into a real-world workspace. The insertion of digitalized information into the real workspace using AR can provide workers with the means to implement correct assembly procedures with improved accuracy and reduce errors. A prototype animated AR system was configured for assembly tasks that are normally guided by reference to documentation and was tested using a series of experiments. A LEGO model was used as the assembly and experimental tester task. Experimentation was devised and conducted to validate the cognitive gains that can be derived from using AR to assemble a LEGO model. Two formal experiments with 50 participants were conducted to compare an animated AR system and the paper-based manual system. One experiment measured the cognitive workload of using the system for assembly, whereas the other measured the learning curves of novice assemblers. Findings from the experiments revealed that the animated AR system yielded shorter task completion times, less assembly errors, and lower total task load. The results also revealed that the learning curve of novice assemblers was reduced and task performance relevant to working memory was increased when using AR training. Future work will apply the knowledge gained from the controlled assembly experiments to the real-scale construction assembly scenario to measure the productivity improvements. (C) 2013 American Society of Civil Engineers.

Full text

Using Animated Augmented Reality to Cognitively
Guide Assembly
Lei Hou1; Xiangyu Wang2; Leonhard Bernold, M.ASCE3; and Peter E. D. Love4
Abstract: Assembly is the process in which two or more objects are joined together. An assembly manual is typically used to guide the tasks
required to put together an artifact. As an emerging technology, augmented reality (AR) integrates three-dimensional (3D) images of virtual
objects into a real-world workspace. The insertion of digitalized information into the real workspace using AR can provide workers with the
means to implement correct assembly procedures with improved accuracy and reduce errors. A prototype animated AR system was
configured for assembly tasks that are normally guided by reference to documentation and was tested using a series of experiments.
A LEGO model was used as the assembly and experimental tester task. Experimentation was devised and conducted to validate the cognitive
gains that can be derived from using AR to assemble a LEGO model. Two formal experiments with 50 participants were conducted to
compare an animated AR system and the paper-based manual system. One experiment measured the cognitive workload of using the system
for assembly, whereas the other measured the learning curves of novice assemblers. Findings from the experiments revealed that the animated
AR system yielded shorter task completion times, less assembly errors, and lower total task load. The results also revealed that the learning
curve of novice assemblers was reduced and task performance relevant to working memory was increased when using AR training. Future
work will apply the knowledge gained from the controlled assembly experiments to the real-scale construction assembly scenario to measure
the productivity improvements. DOI: 10.1061/(ASCE)CP.1943-5487.0000184.© 2013 American Society of Civil Engineers.
CE Database subject headings: Construction management; Productivity; Digital techniques; Imaging techniques.
Author keywords: Augmented reality; Assembly manual; Cognitive learning curve; Working memory.
Introduction
Assembly is the process in which two or more objects are joined
together. An assembly manual is typically used to guide the tasks
required to put together an artifact. A well-formulated assembly
manual should possess the following assembly information: visual
perspectives of product components/parts, parameters or dimen-
sions, technical requirements in quality, installation and testing
specifications, and other auxiliary information.
The implementation of an assembly task typically consists of work
and nonworkpiece-related activities (Neumann and Majoros 1998).
In each assembly step, the assembler is required to conduct a series
of physical operations (e.g., observing, grasping, installing) and
mentally manual-related processes (comprehending, translating, and
retrieving information context) (Neumann and Majoros 1998).
Neumann and Majoros (1998) also suggested that information-related
activities tend to be cognitive, whereas workpiece-related activities
are kinesthetic and psychomotor. Zaeh and Wiesbeck (2008) sug-
gested that assembly using a manual is a time-consuming process.
Moreover, Zaeh and Wiesbeck (2008) suggested that the process of
assembly based on a planar manual fails to consider the cognitive
issues and the large number of switchovers between physical
(workpiece-related) and mental (manual-related) processes, which
can result in operational suspensions and attentional transitions
occurring in novice assemblers. The time-consuming nature of
activities has also been identified by Towne (1985), who found that
information-related activities (cognitive workload) accounted for
50% of the total task workload. Similarly, Veinott and Kanki
(1995) revealed that 45% of every assembler’s shifts were actually
spent on finding and reading procedural and related information
when assembling hardware that had been repaired. Neumann
and Majoros (1998) identified that individual technicians differed
significantly in how much time they devoted to cognitive/
informational tasks, but demonstrated marginal differences with
respect to operational tasks. The use of an assembly manual for
complex and intricate processes can contribute mental tiredness
and the propensity to commit errors because information retrieval
often increases (Watson et al. 2008). Likewise, Veinott and Kanki
(1995) revealed that 60% of the errors that are committed are pro-
cedural and are attributable to misunderstanding the manual. Such
misunderstanding may arise because of the unilateral retrieval of
information, which may trigger behavioral repetition and therefore
suppresses motivation (Wang and Dunston 2008).
An assembly manual is typically paper-based and contains
a large quantity of information pertaining to product parts/
components, a large amount of which may be redundant and inter-
minable, especially for complex tasks. As a result, this can poten-
tially hinder an assembler’s information orientation and his/her
ability to understand complex assembly relations. It is widely
1Postgraduate, Faculty of Built Environment, Univ. of New South
Wales, Sydney, NSW 2052, Australia. E-mail: houleilei@yeah.net
2Professor, School of Built Environment, Curtin Univ., GPO Box
U1987, Perth, WA 6845, Australia; and International Scholar, Dept. of
Housing and Interior Design, Kyung Hee Univ., Korea (corresponding
author). E-mail: xaingyu.wang@curtin.edu.au
3Associated Professor, School of Civil and Environmental Engineering,
Univ. of New South Wales, Sydney, NSW 2052, Australia. E-mail:
leonhard.bernold@gmail.com
4John Curtin Distinguished Professor, School of Built Environment,
Curtin Univ., GPO Box U1987, Perth, WA 6845, Australia. E-mail:
p.love@curtin.edu.au
Note. This manuscript was submitted on December 15, 2010; approved
on December 5, 2011; published online on August 15, 2013. Discussion
period open until February 1, 2014; separate discussions must be submitted
for individual papers. This paper is part of the Journal of Computing in
Civil Engineering, Vol. 27, No. 5, September 1, 2013. © ASCE, ISSN
0887-3801/2013/5-439-451/$25.00.
JOURNAL OF COMPUTING IN CIVIL ENGINEERING © ASCE / SEPTEMBER/OCTOBER 2013 / 439
J. Comput. Civ. Eng. 2013.27:439-451.
Downloaded from ascelibrary.org by Curtin Univ of Technology 2009 on 08/16/13. Copyright ASCE. For personal use only; all rights reserved.

accepted that the capacity of selective information retrieval and fil-
tering does not occur until assembly experiences and expertise are
acquired, and therefore extra-targeted training activities may some-
times be needed (Agrawala et al. 2003). Using an assembly manual
does not necessarily provide an assembler with the problem-
solving skills that are often required when putting together compo-
nents (Pastora and Ferrera 2009). It often takes months or even
years for a novice to develop expert knowledge of the assembly
processes, particularly those of a complex nature (Hoffman et al.
1998). In some cases, an expert assembler must constantly refer
to an assembly manual for unfamiliar procedures or procedures
that are deemed to be arduous. Aside from movements such as
picking, comparing, grasping, rotating, connecting, and fixing the
to-be-assembled components, assemblers have to undertake several
nonassembly-related kinetic operations to understand the assembly
process by paging up/down, head swivelling, and comparing
various elevations.
In construction, assembly is a process in which workers refer to
technical specifications (information activity) to obtain the right in-
formation (information activity), identify components (workpiece
activity), place the component, compare standards (workpiece
activity), and then make a judgment of its correctness (if necessary,
rework may be required). The entire process is iterative and
repeated, and a learning process is triggered that may lead to im-
proved proficiency as cycles are repeated. An inability to find the
correct materials or an incorrect sequence in a cycle can contribute
to productivity losses for an assembly operation. Construction
crews rely heavily on paper-based documents to access and record
information, which can be cumbersome and labor intensive and
therefore increase the propensity for errors to be made. Therefore,
the way in which assembly information is presented to an assem-
bler influences operational effectiveness. There are four main issues
associated with assembly in construction:
1. Not being able to find the right information contained within
technical drawings;
2. Not being able to find the correct component to be assembled;
3. An incorrect assembly sequence; and
4. Incorrect installation.
An example in which assembly problems may arise occurs dur-
ing the installation of heating, ventilation, and air-conditioning
(HVAC) piping. Workers are required to measure the available in-
stallation and workspace, read from the technical drawings, find
and identify the right pipe component, decide its appropriateness,
install, and then check its correctness. Similarly, the rebar assembly
process usually takes place in a prefabricated shop prior to being
delivered to the site. Workers spend a considerable amount of time
trying to find the right length and diameter of rebar to install. The
assembly sequence is critical because the incorrect placement of
rebar can inhibit access to space inside a welded cage. Workers
usually read rebar plans, find the piece, place and weld it, and then
check its correctness. An efficient way to identify rebar is through
color coding with different flags to differentiate its size and type.
This method, however, does not address the assembly sequence that
is adopted.
The insertion of digitalized information into the real workspace
using augmented reality (AR) can provide workers with the means
to implement correct assembly procedures with improved accuracy
(Wang and Dunston 2006a,b). With this in mind, this paper designs
and develops an animated AR system to guide assembly tasks to
reduce errors and improve operational efficiency. A prototype
animated AR system is configured for assembly tasks that are
normally guided by reference to documentation and is tested using
a series of experiments. The proposed system can facilitate the
transition from paper-based manual systems (information activity)
to workpiece activity by complementing human associative infor-
mation processing and memory. The paper particularly focuses on
the cognitive aspects associated with AR and assembly.
From Virtual to Augmented Reality
Virtual reality (VR) has been used extensively to facilitate the
assembly of products (Ritchie et al. 2007). Product designers
are able to create virtual prototypes for accessories, modules,
and parts in virtual environments (VE). Trial assembly in a virtual
environment enables problematic tasks to be identified and various
assembly methods to be explored. Commercial VE prototyping
software such as computer-aided design (CAD), Pro/Engineer,
and Catia has been widely used to facilitate the product
assembly and design process. Product technicians are capable of
designing and developing various accessories, modules, and parts
with different functions and dimensions and conducting assembly
guidance in a virtual space. Regardless of the accuracy that can be
acquired from using VR for product assembly, errors and defects
can still arise.
Virtual reality attempts to replace a user’s perception of the
surrounding world with a computer-generated artificial three-
dimensional (3D) VE. However, a VE is unable to account for
the diverse interferences such as weather, labor constraints, and
schedule pressure that can arise during the assembly process within
the real world. In addition, computer-generated dimensions, tex-
tures, spatial location, and backgrounds provide a limited level
of realism because of a lack of sensory feedback and are therefore
unable to accommodate for perceptual and cognitive viewpoints
(Wang and Dunston 2006a,b). The lack of interaction between
the virtual and real world hinders the adoption of VR to product
assembly tasks.
Augmented reality has been identified as a solution to address-
ing the problem between virtual and real entities (Azuma et al.
2001). As an emerging technology, AR integrates images of virtual
objects into a real world. By inserting the virtually simulated pro-
totypes into the real world and creating an augmented scene, AR
technology could satisfy the goal of enhancing a person’s percep-
tion of a virtual prototyping with real entities. This gives a virtual
world an ameliorated connection to the real world while maintain-
ing the flexibility of the virtual world. Whereas VR separates the
virtual from the real-world environment, AR maintains a sense of
presence and balances perception in both worlds. Through AR, an
assembler can directly manipulate virtual components while iden-
tifying potential interferences between the to-be-assembled and
existing objects inside the real environment. Therefore, in AR envi-
ronment, an assembler can not only interact with real environments,
but also interact with augmented environments (AE) that are struc-
tured to offset the partial sensory loss that may be acquired within
VR. Furthermore, to improve the feedback of augmentation, addi-
tional nonsituated elements could be added into the assembly pro-
cess such as voice recordings, animation, and video.
Augmented reality has been identified as a key technology that
can be used to improve the product assembly process because it can
take into account human cognition (Salonen et al. 2007). For
example, Salonen et al. (2007) used a multimodality system based
on a head-mounted display (HMD), a marker-based software
toolkit (AR Toolkit), image tracking cameras, web cameras, and
a microphone to examine industrial product assembly. Xu et al.
(2008) developed a markerless-based registration technology to
overcome the inconveniences of applying markers as carriers in
the assembly design process. Augmented reality technology has
also been used extensively in the assembly design of a wide range
440 / JOURNAL OF COMPUTING IN CIVIL ENGINEERING © ASCE / SEPTEMBER/OCTOBER 2013
J. Comput. Civ. Eng. 2013.27:439-451.
Downloaded from ascelibrary.org by Curtin Univ of Technology 2009 on 08/16/13. Copyright ASCE. For personal use only; all rights reserved.

of products, e.g., furniture (Zauner et al. 2003), toys (Tang et al.
2003), and industrial robots (Yamada and Takata 2002). Although
such studies have made a significant contribution to understanding
the product assembly process, several key issues remain unresolved
within the assembly domain. For example, researchers have yet to
acquire an in-depth understanding of an assemblers’cognitive
workload when using AR as an alternative to manual procedures
and VR. The images of the to-be-assembled objects in VR systems
only reflect their bilateral or multilateral positioning, and therefore
do not take into their account the dynamic context (e.g., displace-
ment path and spatial interference). To acquire the information con-
text such as the assembly path and fixation forms of parts/
components, assemblers are often required to rely on their memory
retrieval after being subjected to static augmented cues.
To address this issue, dynamic animation juxtaposed with an AR
platform can be used to enable the assembly process. As a result, it
is envisaged that a higher degree of integration between the infor-
mation retrieval processes and task operations can be achieved.
This is in stark contrast with the manual system in which assembly
typically needs to be conducted between retrieving and interpreting
information, selecting the component to be assembled, and putting
together components. The use of AR enables the to-be-assembled
components to be placed at designated workspaces by following
the virtual and the animated pathways identified from a HMD
or on a computer screen (Fig. 1). The physical components and
their virtual counterparts are able to be spatially overlapped, and
therefore assemblers are only required to conduct one visual
transition—that is, between the selection of those components to
be assembled (workpiece stocking area) and assembly point. Fur-
thermore, an animated AR system is able to predefine the tasks
required (including noninterfered assembly paths) by an assembler
so they can readily follow the process to be considered.
Animated AR System
An animated AR system for improving the construction assembly
process that utilizes marker registration technology and visualiza-
tion is developed and presented. The proposed system for assembly
provides information about components to be mounted and outputs
to be assembled step-by-step so that an assembler can monitor their
progress and ensure they do not damage components that have
already been installed. The proposed prototype involves the tradi-
tional establishment and implementation of an AR, which includes
a computer monitor, predefined paper-based markers, interactive
computer graphics modeling, animation and rendering software
(3DSMAX), an ARToolkit, and an attached OpenGL. Using the
ARToolkit, virtual images of product components can be registered
onto predefined markers and captured in view of monitors using
HMD or a computer screen using a marker tracking camera.
The virtual counterparts of real entities are acquired from
3DSMAX and then plugged into the ARTookit through a graphical
interface. The locomotion along the virtual assembly path for each
virtual component and the method of assembly are registered to the
real components by using the ARToolkit and paper-based markers.
The significant parameters of the to-be-assembled and assembled
objects are graphically identified in accordance to their part/
component textures, weight, color, and specification.
Hardware Establishment
The hardware setup of the animated AR system is depicted in Fig. 2,
and the details are described subsequently.
Workbench (Assembling Area)
This is where the assembly process is executed and the markers are
positioned. The size of the workbench is large enough to sustain the
product components and the markers. When the assembly starts,
assemblers can lay the markers on the surface of the workbench
so that the AR animation can be shown on the monitor. The work-
bench also enables assemblers to observe from different angles and
facilitate their operations from various positions.
Position of Monitor and Manual
The monitor is aligned with the workbench and assemblers on the
upper edge of the workbench. When an assembly task commences,
assemblers are able to execute the process while watching the mon-
itor. As a result, they can focus on the augmented scene displayed
and live tasks on the monitor. This setup eases mental workload and
Fig. 1. Visual transition between the manual guidance and the animated AR system
JOURNAL OF COMPUTING IN CIVIL ENGINEERING © ASCE / SEPTEMBER/OCTOBER 2013 / 441
J. Comput. Civ. Eng. 2013.27:439-451.
Downloaded from ascelibrary.org by Curtin Univ of Technology 2009 on 08/16/13. Copyright ASCE. For personal use only; all rights reserved.

visual transition when implementing the assembly task. A mouse
and keyboard provide assemblers with easy control of the anima-
tion course because they can play, pause, and replay the animation
and move the virtual images in augmented scenes. By rotating the
markers or keyboard controls, different angles of augmented scenes
can be observed by the assemblers. Planar information for
assembly guidance can be retrieved by the animated AR because
it is based on the manual’s procedures. The manual is positioned on
the right of the workbench, braced by a bracket. When implement-
ing the LEGO model assembly task or training task, assemblers are
coerced to frequently switch their attention between workbench
and manual and page up or page down to retrieve information.
Tracking Webcam
The tracking webcam is a Logitech Webcam Pro 9000 HD, which
can ensure a high-definition (HD) view with an autofocus. It
projects to the rotatable workbench to overlap the webcam view
and participants’field of vision. The images of virtual components
and the real components are captured by the webcam so the assem-
blers are required to only focus on the augmenting scene identified
on the monitor.
By tracking the predefined markers, the customized animated
guidance can be displayed on the monitor. The angle between
the webcam projection and the horizontal workbench is fixed in
this instance, but this is only to ensure that the webcam is able
to capture the black frame of the marker and the assemblers’
manipulation.
Paper-Based Markers and Components
Markers are all trained using the ARToolkit. There is a main marker
that is used to animate the process throughout the entire product
assembly, and other markers can be added to cater for specific
purposes; for example, an ancillary marker with pattern 人is set
to present the virtual layout of to-be-assembled components. All
markers are provisionally placed on the left of the workbench.
Similarly, the to-be-assembled physical components are also placed
on the left zone of the workbench, which is the workpiece stocking
area, as depicted in Fig. 2.
Software Setup
Conventional AR environments are based on the ARToolkit in
which virtual objects are usually drawn using pure drawing
functions of OpenGL (Open Graphics Library), a multiplatform
high-level 3D graphics application programming interface (API).
However, if users want to build their own models, they must
acquire the knowledge of OpenGL. For the purpose of facilitating
layman users without OpenGL knowledge, some AR systems have
realized the direct loading of varieties of model files, such as Buil-
dAR and Layer. The aforementioned systems cannot be customized
to fit the experimental requirements of the research to be under-
taken in this paper, specifically issues relating component dimen-
sional comparisons and assembly clue registration. Thus, it was
decided to redevelop a set of functionalities that can dynamically
load model files into the proposed AR system. Akin to other AR
systems, the proposed animated AR system is a user-centered inter-
face between the ARToolkit and any 3D modeling software that
utilizes. 3DS files such as 3DSMAX, MAYA, and CINEMA4D.
In addition, animations can be directly imported into the AR inter-
face through the attached exporters of 3D modeling software and
recognized by the predefined markers without the more sophisti-
cated exporter such as OSGExp. The standard materials and
rendering effects can be securely conserved after being exported.
A multimarker to enable an AR interface with the synchronous
display of multiple virtual objects for assembly purpose was
adopted.
Contents Creation of Virtual Assembly Animations
The assembly task for the experimental evaluation should be
selected to align with the practical application, and to be very
representative and capable of disclosing various effects of different
assembly guidance. However, the safety and manoeuvrability con-
siderations in the experiments restrict the sizes of the assembly
product. Also, the task selected should be complex enough to give
rise to high demands on human cognition. Therefore, the LEGO
MINDSTORMS NXT 2.0 is selected as the experimental content
for the animated AR system because of each components’dimen-
sional disparity (e.g., shape and color) (Fig. 3).
Consequently, the assembly sequence and component installation/
fixation are conceived to be critical issues rather than being
component-based. The LEGO model used consists of 35 spatially-
functioning pieces (Fig. 4). These components are detached in
advance and positioned in the workpiece stocking area. Ten partic-
ipants were recruited for a pilot study to try to assemble the LEGO
model. They were presented with the assembly manual and then it
was removed prior to initiating the assembly process.
None of the participants were able complete the model assembly
within 20 min without the guidance provided (20 min was defined
Fig. 2. Hardware setup and real layouts of model assembly
442 / JOURNAL OF COMPUTING IN CIVIL ENGINEERING © ASCE / SEPTEMBER/OCTOBER 2013
J. Comput. Civ. Eng. 2013.27:439-451.
Downloaded from ascelibrary.org by Curtin Univ of Technology 2009 on 08/16/13. Copyright ASCE. For personal use only; all rights reserved.

as a threshold of complexity), even though free assembly opera-
tions were allowed. The task difficulty matched the needs and re-
quirements of the experimental design. Some components were
similar in shape but different in dimensions, and therefore task
completion was expected to be based on the recalling of the training
contents. The following three aspects of the animated AR system
present the mapping of facilitations:
1. Real-scaled virtual components are able to spatially coincide
with the physical components: In conventional assembly
manuals, the component images are typically down-scaled
or smaller than the physical components; this is because of
the limited size of assembly manuals. The implementation
of a component/part selection process typically depends on
the dimensional labels marked in the assembly manual, or
the similarity of component images and physical components.
It is sometimes difficult to understand the component shape in
an assembly manual and the interrelations that can exist
between components. It is also a challenge to visualize the
spatial structure of a product when comparing different views.
Fundamentally, the problems associated with informational re-
trieval from conventional assembly manuals can be overcome
by using AR techniques. Virtual counterparts of real objects
can be defined in a real-scaled size and observed (each facet
of virtual objects is visible) by rotating markers, which
improves an assembler’s understanding of operations. In the
LEGO model assembly, for example, 35 components were
the same color or approximately the same size, but assemblers
were able to select components correctly by comparing the real
and virtual images of different parts (Fig. 5).
2. Supplemented augmentations to ease ongoing tasks: Special
hints are applied as supplemented augmentations under speci-
fic circumstances; for example, a red arrow in the pin-hold
assembly helps assemblers to confirm the matching relation-
ships in a spatial position. For instance, the third hold from the
right of the red piece matches the first hold from the right of
black piece. The hints also provide the assemblers with
the recommended assembly method. This recommendation is
provided so as to ensure that to-be-assembled components
do not spatially interfere with the already assembled
components. Function keys such as O on the keyboard are
supplemented to detach the pin-hold assembly in the AR
environment if the assemblers do not determine how they
match together (Fig. 6). The diversified supplemented aug-
mentations in the AR animation prototype are generated to
ease the ongoing task.
3. Stepwise guidance creates a framework of association that aids
assembly recall: As previously described, AR animation
creates a framework of association that aids recall commonly
referred to as spatially augmented stimuli. These stimuli
together may form a framework when subjects use a classic
mnemonic technique, the method of loci, to remember a list
of items (Neumann and Majoros 1998). Each association of
a virtual object with a sequential workpiece feature is a basis
for linking memorial pieces in human memory. In the
animated AR system, when each augmented step of assembly
becomes represented on the next one (Fig. 7), this may
increase performance of sequential recall.
This could be possibly explained by proficiency, memory,
and knowledge differences that exist between novices and ex-
perts. Memory capacity is a capacity that may help an expert
assembler mentally construct the contents without actually
spending too much time on retrieving from physical media.
Because of the difference of individual capacity in strategy
of handling memorial pieces or short-term memorial store,
it makes a difference among different people in terms of
the effectiveness of retrieving the memory that stores previous
information. The stepwise guidance enabled by the AR
animation form may be facilitating the linkage of short-term
memorial pieces, and thus be able to improve ergonomic
performance by impacting recall capacity.
Fig. 3. Snapshot of LEGO MINDSTORMS NXT 2.0 and its
components
Fig. 4. LEGO model in 3DSMAX and the animated AR system
JOURNAL OF COMPUTING IN CIVIL ENGINEERING © ASCE / SEPTEMBER/OCTOBER 2013 / 443
J. Comput. Civ. Eng. 2013.27:439-451.
Downloaded from ascelibrary.org by Curtin Univ of Technology 2009 on 08/16/13. Copyright ASCE. For personal use only; all rights reserved.

Fig. 5. Components matching and mismatching in terms of shape and color
Fig. 6. Supplemented augmentations to ease ongoing tasks
Fig. 7. Model is assembled step by step: completion of middle part, left and right parts, and lateral parts
444 / JOURNAL OF COMPUTING IN CIVIL ENGINEERING © ASCE / SEPTEMBER/OCTOBER 2013
J. Comput. Civ. Eng. 2013.27:439-451.
Downloaded from ascelibrary.org by Curtin Univ of Technology 2009 on 08/16/13. Copyright ASCE. For personal use only; all rights reserved.

Hypothesis
The objective of this research is to examine the cognitive potentials
by revealing what specific facilitations the animated AR system
could lend to the assemblers, and to testify the likelihood of short-
ening the learning curve of novice assemblers when implementing
the actual assembly or training. Based on this, the hypotheses are
formulated in four types as follows:
1. When compared to a conventional paper-based manual, the
animated AR system is able to lowering an assembler’s
cognitive workload during the LEGO model assembly task.
2. When compared to a conventional paper-based manual, the
animated AR system shortens the time spent on the LEGO
model selection and assembly operation.
3. When compared to a conventional paper-based manual, the
animated AR system reduces the amount of assembly errors
that arise.
4. Using the animated AR system as a training tool reduces the
learning curve of trainees in cognition-demanded assembly.
This is based on a subhypothesis that training within an
AR environment facilitates longer working memory (WM)
capacity compared to training with a manual.
Human memory, especially WM, normally includes certain
mechanisms for forming memorial associations (chains) between
representations. The formation of a memorial association (chain)
is a process of linking the representations that been previously
retrieved (Unsworth and Engle 2007). The lowering of cognitive
workload through the enhancement of spatial cognition in the ani-
mated AR system might influence the mechanism of short-term
memorial storage and retrieval. The assembler’s task performance
should reflect a certain level of difference after two means of
assembly training, at least from the performance that is related
to memorization, for instance, human behaviors corresponding
to recollecting component assembly sequence and method.
Experimentation
An experimental design pertaining to the use of the animated AR
system for assembly influencing the cognitive issues is evaluated.
The experimental design investigates whether users, especially
novice assemblers, can be facilitated by the AR technology during
assembly. Moreover, the research examines the factors hindering
this facilitation. The research design assists with the identification
of training effects on the posttask performance using AR and an
assembly manual and the relationship between WM and learning
curves. The experimental design consists of three distinct
phases (Fig. 8):
1. Mental rotations;
2. Two main experiments; and
3. A usability evaluation of the animated AR system.
Mental rotations were first undertaken to examine spatial-
cognitive capacity. Then, two experiments were executed to
compare two scenarios: manual and AR. The objective of the first
experiment is to study a person’s cognitive performance when
merging digital virtual information (e.g., AR animation guidance)
into a real assembly workspace as compared with merging the
physical information (e.g., guidance manual) into the real assembly
workspace. The objective of the second experiment is to compare
the learning curves of AR training with assembly manual training.
Whereas prior discussion focused on the differences between
two-dimensional (2D) planar images in manual and 3D spatial
images in AR, the experiments sought to isolate the animated
AR system’s unique advantage by using 3D forms of components
Fig. 8. Evaluating cognitive issues of using the animated AR system and assembly manual in product assembly tasks
JOURNAL OF COMPUTING IN CIVIL ENGINEERING © ASCE / SEPTEMBER/OCTOBER 2013 / 445
J. Comput. Civ. Eng. 2013.27:439-451.
Downloaded from ascelibrary.org by Curtin Univ of Technology 2009 on 08/16/13. Copyright ASCE. For personal use only; all rights reserved.

as guidance in both cases. Therefore, scenario one was a paper-
based 3D manual in which the participants could see the 3D LEGO
components (Fig. 9). In this experiment, 50 graduate students from
the Department of the Built Environment at University of New
South Wales (UNSW) were recruited. None of them had ever used
AR before. All subjects voluntarily participated in the experiments.
All were informed of their rights as research participants as per the
UNSW for Human Subjects Research protocol. Experiments I and
II comprised of 20 and 30 participants, respectively.
Fore-Task-Prejudgement of Cognitive Capacity
The fore-task of mental rotation was undertaken prior to the main
experiments. Its role was to examine each subject’s levels of inher-
ent spatial-cognitive capacity (Fig. 10). Mental rotation is regarded
as a direct and convenient measurement for human capacity of
spatial object cognition. Task processing refers to the procedural
visuo-spatial input, mental manipulation, and back to reality
(output). It is dependent on spatial capacity and cognitive workload
(Zacks 2008). Therefore, the results of determining mental rotation
exercise for spatial-cognitive capacity may be used to provide a
baseline of each subject’s capacity in this domain.
Experiment I—Cognitive Workload
The objective of experiment I is to study a person’s cognitive per-
formance when merging digital virtual information (e.g., AR ani-
mation guidance) into a real assembly workspace as compared with
merging the physical information (e.g., guidance manual) into the
real assembly workspace. A concurrent task strategy (also known
as secondary task strategy) was applied because it reflected the
level of cognitive load imposed by a primary task (Dunlosky
and Kane 2007). This is based on the tentative study that if the
assembly task performances under the two scenarios do differen-
tiate in participants’associated cognitive load, their mental and
motor performance would be differentially influenced by the intro-
duction of concurrent cognitive tasks (Rose et al. 2000). To those
who suffer less cognitive load, they may free up their cognitive
capacity to deal with interfering tasks. In the experimental design,
each of the scenarios assesses different cognitive needs, and
includes a secondary task to examine cognitive workloads.
The physical performance of cognition-related tasks is depen-
dent on mental process. A specific portion of mental resources
are occupied by certain cognitive needs. When a secondary task
is introduced, mental processes may be subject to high demands.
The measurement for cognitive workload includes subjective ana-
lytical and empirical methods (usually involving a questionnaire
comprising of one or multiple semantic differential scales in which
the subject can indicate the experienced level of cognitive load) and
a rating scale technique (which is based on the assumption that
people are able to introspect on their cognitive processes and report
the amount of mental effort expended) (Xie and Salvendy 2000).
Most subjective measures are multidimensional because they
assess groups of associated variables such as mental effort, fatigue,
and frustration, which are highly correlated. Rating scales may
appear questionable, however; it has been demonstrated that people
are quite capable of providing a numerical indication of their per-
ceived mental burden (Gopher and Braune 1984). Furthermore, the
physiological domain provides useful measurements for the recog-
nition of cognitive load, which is based on the assumption that
changes in cognitive functioning are reflected by physiological
variables (Beatty and Lucero-Wagoner 2000).
Taking into account the complexity of measuring equipment and
technical constraints, the psycho-physiological measures were not
considered as evaluation tools in this research. Instead, the possible
compromise is to combine the subjective analytical methods (ques-
tionnaire and interviews) and objective methods (task performance
observation and videotaping) and adopting the rating scale technol-
ogy based on a questionnaire [National Aeronautics and Space
Administration (NASA) task load index] (Hart 2006) (Fig. 11).
The subjective workload measurement techniques using rating
scales are easy to use, inexpensive, reliable, able to detect small
variations in workload, and provide decent convergent, construct,
and discriminate validity (Gimino 2002). The objective measure-
ment techniques are robust to conduct the susceptibility research
and enable the experimental results of both subjective and objective
analysis (Mulhall et al. 2004).
The two-group crossover design was used to minimize the
effects from the learning curve imposed by the different experiment
sequences for 20 subjects (Fig. 8). After the mental rotation quiz
(18 items contained on a mental rotation test sheet), all subjects
scored between 13 and 18 (evaluated as normal spatial ability).
Prior to the LEGO model assembly task, the participants in the
Fig. 9. Scenario of manual and LEGO experiments
Fig. 10. Mental rotation [adapted from Collins and Kimura (1997)]
446 / JOURNAL OF COMPUTING IN CIVIL ENGINEERING © ASCE / SEPTEMBER/OCTOBER 2013
J. Comput. Civ. Eng. 2013.27:439-451.
Downloaded from ascelibrary.org by Curtin Univ of Technology 2009 on 08/16/13. Copyright ASCE. For personal use only; all rights reserved.

two groups (10 in each) with two separate scenarios were
exposed to several pictures of spatial objects and were required
to remember them. In the first period (group 1 in scenario 2, group
2 in scenario 1), the subjects were simultaneously prompted to
listen for the names of objects interspersed within a string of
prerecorded words presented at 3-s intervals. Subjects were then
asked to say yes if they heard the previously shown images of
spatial objects. When they finished the first period, two groups
resumed the second period, but switched over the scenarios.
Therefore, errors made would be calculated based on their
performance. Postexperiment questionnaires were designed to be
completed based on the subjects’experience and feelings during
the experiment.
Data Analysis
Fig. 12 indicates that participants in scenario two had shorter com-
pletion times (7.37 min) compared with subjects in scenario one
(11.91 min). An ANOVA was conducted on the different effects
of guiding methods for the time of completion. In statistical signifi-
cance testing, the p-value is the probability of obtaining a test sta-
tistic at least as extreme as the one that was actually observed,
assuming that the null hypothesis is true. One often rejects the null
hypothesis when the p-value is less than 0.05 or 0.01. When the
null hypothesis is rejected, the result is statistically significant.
In this experiment, the average time of completion for subjects us-
ing individual guidance is statistically significant, Fð1;18Þ¼23.8,
p<0.001. Thus, AR has an advantage in time of completion when
compared with the assembly manual.
The AR animation provides a dynamic demonstration of
consistent information context through animation segments dis-
played in each assembly step. The subjects were able to detect
the existing dimensions from components-in-place and those reg-
istered attached to the virtually to-be-assembled components from
the monitor.
Simultaneously, the animation dynamically demonstrated the
assembly process by approaching the virtually to-be-assembled ob-
jects to those already assembled. This enabled subjects to mimic
each assembly step and complete the real assembly operation with
greater ease. By demonstrating a series of virtual animation
segments registered in the real assembly space, AR is able to
compensate for the mental and cognitive gaps between individual
differences of information retrieval capacity and the task difficulty
imposed on individuals. Consequently, AR eases information
retrieval by integrating the task of searching information and the
task of the actual assembly.
Offering real-time in situ assembly guidance is another charac-
teristic feature of an animated AR system. In each step of experi-
ment I, it was observed that the AR animation scenario dynamically
and sequentially ushered the position changes of spatial compo-
nents by activating each animation segment that was triggered
by the subjects. When completing each animation segment, the
Fig. 11. NASA task load index based on questionnaire (Hart 2006)
Mean time of completion (Min)
Fig. 12. Average time of completion in model assembly
JOURNAL OF COMPUTING IN CIVIL ENGINEERING © ASCE / SEPTEMBER/OCTOBER 2013 / 447
J. Comput. Civ. Eng. 2013.27:439-451.
Downloaded from ascelibrary.org by Curtin Univ of Technology 2009 on 08/16/13. Copyright ASCE. For personal use only; all rights reserved.

animated AR system turned into a visual tool for presenting the
statically augmented component images. In parallel, the animation
was temporarily suspended for the next trigger by subjects. During
each suspended interval, the subjects were given sufficient time to
pick up the components from the rest of the to-be-assembled com-
ponents and position them in their final positions. The assembling
operations and augmented guidance essentially proceeded together.
Fig. 13 indicates the number of errors made when undertaking
the LEGO assembly task. This chart reveals that in scenario 2, sub-
jects had a lower error rate compared with treatment 1 (3.4 versus
1.3). An ANOVA was conducted on the effect of guiding methods
on error assembly. The average number of errors for AR using indi-
vidual guidance is statistically significant, Fð1;18Þ¼6.6,
p¼0.0193. Therefore, AR appears to have an advantage in reduc-
ing error assembly when compared with the assembly manual.
To reduce the time it takes to complete a task, subjects may sub-
ject themselves to mental stress. Under this circumstance, it is criti-
cal that the coherence of the information context should be
guaranteed to cater for the on-task assembly information retrieval.
The animated AR system provides a dynamic demonstration of
consistent information context through animation segments dis-
played within each assembly step. Subjects could detect the
existing dimensions from in-place and virtually to-be-assembled
components from a computer screen or projector. The animation
dynamically demonstrates the assembly process by approaching
the virtually to-be-assembled objects to those assembled in the cor-
rect positions. This enables participants to mimic each assembly
step and lower the difficulty of the operation and task errors.
It was observed that during the experiment, some subjects using
the manual as guidance often did not understand or correctly inter-
pret the exact assembly path. With AR, perspectives can be
changed easily by rotating markers. Some manual participants
complained the manual was too difficult to understand, and some
even reported high frustration of understanding the manual. In pur-
suit of speed, some subjects using the manual believed that they had
understood the specific assembly steps, but had not because a num-
ber of errors were made.
Fig. 14 indicates the mean rating of the NASA task load index.
The statistics show that subjects in scenario 1 had the higher mental
workload than subjects in scenario 2. Rating results indicate that
the subjects in scenario 2 awarded an average score 9.84, which
was lower than in scenario 1 (13.64). An ANOVA was conducted
on the different effects of guiding methods on cognitive load. The
effect was statistically significant (p-value ¼0.0053), and H1, H2,
and H3 are therefore supported. The manual assembly appears to
have greater mental workload for subjects, whereas AR animation
has an average effect of lowering cognitive workload in the LEGO
model assembly task. The animated AR system shortens the time
spent on the LEGO model selection and assembly operation, and
reduces the amount of assembly errors.
The higher mental demand subcategory rating involved in using
the manual (16.3=20 versus 8.7=20) implies that more perceptual
activities were required to complete the assembly and concurrent
memorizing tasks. Trying to reason the spatial relationship of ob-
jects using the manual may have frustrated or discouraged some of
the subjects, which may have induced temporal stress. These con-
siderations can explain why the average ratings of both frustration
level and temporal demand were higher using the manual (frus-
tration score: 14.3=20 for manual and 9.0=20 for the animated
AR system; temporal score: 14=20 for manual and 12=20 for
the animated AR system).
Higher frustration and temporal demand levels were in accor-
dance with the longer performance time while using the manual
as the guidance tool. The p-value for physical demand is less than
0.001, which indicates there was a significant difference in physical
demand for both scenarios. Physical demand in using the animated
AR system is lower (12=20) because the subjects using the ani-
mated AR system did not consistently conduct visual transitions
or movements such as page up/down. This implies that the ani-
mated AR system provided a considerably natural and comfortable
way of guiding the assembly task. The close effort subcategory
score for the animated AR system (8.4=20) and for the manual
method (12.5=20) indicates a lower overall challenge (mentally
and physically) was experienced by the subjects in accomplishing
their level of performance, which was further confirmed by a
significant correlation (p¼0.52).
Experiment II—Learning Curve
The objective of experiment II was to establish learning curves for
the two scenarios to study if there are significant differences in the
performance between of the two groups of trainees using different
Mean number of errors
Fig. 13. Average number of errors in model assembly
Fig. 14. NASA TLX scores for each item for evaluating cognitive workload in model assembly
448 / JOURNAL OF COMPUTING IN CIVIL ENGINEERING © ASCE / SEPTEMBER/OCTOBER 2013
J. Comput. Civ. Eng. 2013.27:439-451.
Downloaded from ascelibrary.org by Curtin Univ of Technology 2009 on 08/16/13. Copyright ASCE. For personal use only; all rights reserved.

training schemes. According to Richardson et al. (1996), decision-
making capacity reflects the time it takes the WM to glean and pro-
cess the properties of the stimulus. The decision-making process
applies to motor performance in which too much complexity leads
to higher error rates and false moves. In addition, the span of WM
of trainees depends on the characteristics of the information to be
acquired.
Experimental Procedure
Experiment II tests the effect of the animated AR system by meas-
uring the performance of two test groups referred to as AR training
and manual training denoted in Table 1. Like in experiment I, ex-
periment II also isolated the animated AR system’s unique advan-
tage by using 3D modeling for training in both scenarios. Three
metrics were used to evaluate performance:
1. Number of assembly trials until assembly was completed with-
out an error;
2. Time consumed to complete a trial; and
3. Number of errors committed during a trial.
Prior to randomly selecting the 30 test trainees, 30 graduate stu-
dents from the Department of the Built Environment at UNSW
were used to pilot the experimental process. Base training, follow-
ing a manual, was limited to one single LEGO model assembly
cycle without a time limit. The test trainees were encouraged
to remember the assembly sequence and component fixation/
installation. After the base training was completed, the trainees
relaxed for 5 min reading material irrelevant to the experiment
(e.g., newspaper).
During this period, the assembly manual was removed, and the
model pieces were laid out on a table. The two test groups of 15
students, AR training and manual training, were now starting the
first trial, one group without a manual and one group without the
assistance of AR. Three generic types of errors emerged:
1. Component selection error;
2. Assembly sequential error; and
3. Fixation/installation error.
Requesting help from the animated AR system or manual was
also considered an error because trainees might err if no guidance
was provided. The errors during unsuccessful trials were added to-
gether for each trainee and group. Subjects were videotaped during
their task assignment so that potential errors could be identified.
They were told how many errors were made and were allowed
to check the steps in which the error had occurred. Because there
was no guidance or information available, trainees had to mentally
retrieve information and recall the assembly steps from their WM
that had been developed in the training sessions.
Data Analysis
In Table 1, the variations in the average amount of errors during a
trial are presented. For the first trial, an average of 6.07 errors were
made by the manual training group compared to 3.67 of the AR
training group. For the second trial, an average of 3.13 errors made
by the 15 manual trainees is significantly higher than the AR
trainees. The number of errors made relates to the trainees’WM
effect in the formal assembly task.
Fig. 15. Average time elapsed within each trial in formal assembly
Table 2. Statistical Results for Time Eclipsing of Each Formal Trial in
Experiment II
Trial F-value p-value Significance
1st 21.68 0.001 Significant
2nd 14.36 0.001 Significant
3rd 4.29 0.05 Significant
Table 1. Training Methods, Number of Trials, and Mean Number of Errors in Formal Assembly
Trial
AR training Manual training
Number of
trainees Mean errors
Number of trainees
who did not err
Number of
people Mean errors
Number of trainees
who did not err
1 15 3.67 0 14 6.07 0
2 15 1 9 14 3.13 1
3 6 0 6 14 0.86 7
4—— — 70 7
Fig. 16. Performance curve of conducting formal assembly
JOURNAL OF COMPUTING IN CIVIL ENGINEERING © ASCE / SEPTEMBER/OCTOBER 2013 / 449
J. Comput. Civ. Eng. 2013.27:439-451.
Downloaded from ascelibrary.org by Curtin Univ of Technology 2009 on 08/16/13. Copyright ASCE. For personal use only; all rights reserved.

Trainees with AR training could remember or recollect more
assembly clues that were memorized in the former training task
than those trained in the manual. The mean time elapsed within
each trial between two trainings is depicted in Fig. 15.
A mean time of 13.07 min was needed for the trainees after AR
training to complete the first trial, compared with a mean time of
18.6 min for the trainees being trained with the manual. Within the
second and third trials, these numbers are 8.83 min (AR) versus
12.73 min (manual) and 7.67 min (AR) versus 9.29 min (manual),
respectively. An ANOVA was conducted on the different effects of
training on the time consumption of each trial. It is statistically sig-
nificant that the mean time in the first trial (SDAR ¼3.71,
SDManual ¼2.72) is dependent on the individual training means
(p¼0.001). Likewise, it is statistically significant for the second
and third trial as well (for the second trial: p¼0.001,
SDAR ¼2.39,SD
Manual ¼3.15; for the third trial: p¼0.05,
SDAR ¼1.63,SD
Manual ¼1.59), as depicted in Table 2.
More trials are likely to be needed until manual-based trainees
complete the final trial without an error, e.g., seven manual-based
trainees completed formal assembly in their third and fourth trials.
The performance curve of conducting a formal assembly is pre-
sented in Fig. 16.
The data illustrate that trainees under AR training spent less time
completing each formal assembly trial. Nine test participants in the
AR training group were able to successfully complete the assembly
after only two trials. However, only six trainees in the group with-
out AR were successful during trial three and seven in the fourth
trial. To satisfactorily complete the assembly process within the
specified time period (i.e., 6 min) and without error or acquiring
additional information, trainees using AR required fewer trials
(2.52) than those using manual training (3.5).
To achieve a satisfactory training effect in terms of three metrics,
i.e., number of assembly trials, time consumed to complete a trial,
and number of errors, the AR trainees need an average of 2.5 times
of trial (¯
t) and 24.83 min (Total), whereas the manual trainees need
an average of 3.5 times of trial (¯
t) and nearly 42.42 min (Total). The
time (Total) is calculated by
Total ¼¯
tׯ
TðtÞð1Þ
where Total = total time of achieving satisfactory training effect;
¯
t= mean number of trials; and ¯
TðtÞ= mean time consumption
within each trial (t¼1, 2, 3, 4).
The use of an animated AR system as a training tool shortens the
learning curve of trainees in cognition-demanded assembly, and
Table 3. Results and Interpretation of Usability Analysis for Animated AR System
Issues Mean Summarized results
Navigation
Did you often feel disoriented? 2.1 Little disoriented
Users felt a little disoriented with nothing in the augmented scene for
the navigational cues or landmarks.
Did the surrounding real background help your spatial comprehension? 3.9 Slightly apparent
This is one of the advantages of AR over manual.
Input mechanism
Did you feel annoyed or inconvenienced when operating the keyboard
or marker to view different angles of the virtual image?
1.8 Very positive
Although here are still some system drawbacks, the user still expresses a
positive attitude toward system control.
Visual output
Did visual output have adequate stability of the images as you moved
with no perceivable distortions in visual images?
3.4 Neutral
It seems that the system lag is tolerable and does not affect the
perception of visual image of users and therefore does not affect their
performance.
Was the field of view (FOV) appropriate for supporting this activity? 4.1 Very appropriate
The broader the projection, the better sense the user has for the
environment and communication with the AR system.
Did the monitor-based visual display create difficulties for observing? 1.9 Very easy
Users felt it was easy to watch the large projection or television monitor
while performing the LEGO assembly task; not like the HMD, which
might result in a cumbersome and uncomfortable feeling, the monitor is
robust enough to support assembly.
Did you believe the LEGO images could be spatially matched with the
physical counterparts?
3.9 Slightly positive
User felt that the virtual augmented components of the LEGO could be
spatially matched with the physical components. Therefore, this
characteristic facilitates the comparison and selection of assembly
components, as stated in Fig. 5.
Was the AR display effective in conveying convincing scenes of models
appearing as if in the real world?
3.4 Neutral
The virtual model looks like it is floating into the air of the real
environment. Neutral rating implies that the combination of virtual
model and real world reaches a level of seamlessness to some extent.
Immersion
With the AR system, were you isolated from and not distracted by
outside activities?
3.3 Neutral
It seems that the users did not feel much distraction from outside
activities by being isolated from the outside, which implies that the AR
system might be useful in focusing users’minds on the task.
Comfort
Was the AR system comfortable for long-term use? 4.1 Very comfortable
Very high score demonstrates the acceptability of animated AR system.
It is not bulky, not triggering user fatigue, and not limiting user mobility.
450 / JOURNAL OF COMPUTING IN CIVIL ENGINEERING © ASCE / SEPTEMBER/OCTOBER 2013
J. Comput. Civ. Eng. 2013.27:439-451.
Downloaded from ascelibrary.org by Curtin Univ of Technology 2009 on 08/16/13. Copyright ASCE. For personal use only; all rights reserved.

training in AR facilitates longer WM capacity compared to training
in a manual. Thus, the evidence provided supports H4. Sample
results of questions and usability suggestions are identified in
Table 3.
Conclusions
The aim of the research was to assess the effectiveness of AR-based
animation in facilitating effective and efficient learning or training
of people involved in the assembly of complex systems. A set of
initial experiments designed to assess the discrepancies between the
traditional guidance and AR was undertaken. Results from the ex-
periments indicate a positive effect of cognitive facilitation when
using an animated AR system. When trainees relied on their
memory and the manual to complete an assembly, they were prone
to making errors. When AR was used, the learning curve of trainees
significantly improved, and fewer errors were made. It is suggested
that the use of AR technology for guiding the assembly process in
the field of construction assembly will provide similar improve-
ments. Moreover, AR can be used to guide novice assemblers when
performing highly complex assembly tasks in which training time
is limited and the potential for errors are either dangerous or costly.
Future research will focus on testing AR with construction oper-
ations with a larger and more diverse set of trainees.
Acknowledgments
The authors would like to thank the five anonymous reviewers for
their constructive comments that have helped improve the quality
of the research reported in this manuscript.
References
Agrawala, M., et al. (2003). “Designing effective step-by-step assembly
instructions.”ACM Trans. Graph., 22(3), 828–837.
Azuma, R., Baillot, Y., Behringer, R., Feiner, S., Julier, S., and MacIntyre,
B. (2001). “Recent advances in augmented reality.”IEEE Comput.
Graph. Appl. Mag., 21(6), 34–47.
Beatty, J., and Lucero-Wagoner, B. (2000). “The pupillary system.”Hand-
book of psychophysiology, J. T. Cacioppo, L. G. Tassinary, and G. G.
Berntson, eds., Cambridge University Press, Cambridge, MA, 142–162.
Collins, D. W., and Kimura, D. (1997). “A large sex difference on a two-
dimensional mental rotation task.”Behav. Neurosci., 111(4), 845.
Dunlosky, J., and Kane, M. J. (2007). “The contributions of strategy use to
working memory span: A comparison of strategy assessment methods.”
Q. J. Exp. Psychol., 60(9), 1227–1245.
Gimino, A. (2002). “Students’investment of mental effort.”Paper pre-
sented at the annual meeting of the American Educational Research
Association, New Orleans, LA.
Gopher, D., and Braune, R. (1984). “On the psychophysics of workload:
Why bother with subjective measures?”Hum. Factors: J. Hum. Factors
Ergon. Soc., 26(5), 519–532.
Hart, S. G. (2006). “NASA-task load index (NASA-TLX); 20 years later.”
Proc., of the Human Factors and Ergonomics Society 50th Annual
Meeting, Human Factors and Ergonomics Society, Santa Monica,
CA, 904–908.
Hoffman, R. R., Crandall, B., and Shadbolt, N. (1998). “Use of the critical
decision method to elicit expert knowledge: A case study in the meth-
odology of cognitive task analysis.”Hum. Factors, 40(2), 254–277.
Mulhall, J. P., et al. (2004). “Subjective and objective analysis of the preva-
lence of Peyronie’s disease in a population of men presenting for pros-
tate cancer screening.”J. Urol., 171(6), 2350–2353.
Neumann, U., and Majoros, A. (1998). “Cognitive, performance, and
systems issues for augmented reality applications in manufacturing
and maintenance.”Proc., IEEE Virtual Reality Annual Int. Symp., IEEE
Computer Society Technical Committee on Computer Graphics and
IEEE Neural Networks Council Virtual Reality Technical Committee,
Atlanta, GA, 4–11.
Pastora, R., and Ferrera, L. (2009). “An improved mathematical program to
solve the simple assembly line balancing problem.”Int. J. Prod. Res.,
47(11), 2943–2959.
Richardson, J. T. E., Engle, R. W., Hasher, L., Logie, R. H., Stoltzfus, E. R.,
and Zacks, R. T., eds. (1996). Working memory and human cognition,
Oxford University Press, New York.
Ritchie, J. M., Robinson, G., Day, P. N., Dewar, R. G., Sung, R. C. W., and
Simmons, J. E. L. (2007). “Cable harness design, assembly and instal-
lation planning using immersive virtual reality.”Virtual Reality, 11(4),
261–274.
Rose, F. D., Attree, E. A. A., Brooks, B. M., Parslow, D. M., Penn, P. R.,
and Ambihaipahan, N. (2000). “Training in virtual environments:
Transfer to real world tasks and equivalence to real task training.”
J. Ergon., 43(4), 494–511.
Salonen, T., Sääski, J., and Hakkarainen, M. (2007). “Demonstration of
assembly work using augmented reality.”Proc., 6th ACM Int. Conf.
on Image and Video, Association for Computing Machinery, New York,
120–123.
Tang, A., Owen, C., Biocca, F., and Mou, W. (2003). “Comparative
effectiveness of augmented reality in object assembly.”Proc., SIGCHI
Conf. on Human Factors in Computing Systems Table of Contents,
Association for Computing Machinery, New York, 73–80.
Towne, D. M. (1985). “Cognitive workload in fault diagnosis.”Rep. No.
ONR-107, Contract No. N00014-80-C-0493 with Engineering Psychol-
ogy Group, Office of Naval Research, Behavioral Technology Labora-
tories, University of Southern California, Los Angeles, CA.
Unsworth, N., and Engle, R. W. (2007). “The nature of individual differ-
ences in working memory capacity: Active maintenance in primary
memory and controlled search from secondary memory.”J. Psychologi-
cal Rev., 114(1), 104.
Veinott, E. S., and Kanki, B. G. (1995). “Identifying human factors issues
in aircraft maintenance operations.”Proc. Hum. Factors Ergon. Soc.
Ann. Meet., 39(14), 950.
Wang, X., and Dunston, P. S. (2006a). “Compatibility issues in augmented
reality systems for AEC: An experimental prototype study.”Autom.
Constr., 15(3), 314–326.
Wang, X., and Dunston, P. S. (2006b). “Mobile augmented reality for
support of procedural tasks.”Proc., Joint Int. Conf. on Computing and
Decision Making in Civil and Building Engineering, Montreal, Canada,
1807–1813.
Wang, X., and Dunston, P. S. (2008). “User perspectives on mixed reality
tabletop visualization for face to face collaborative design review.”
Autom. Constr., 17(4), 399–412.
Watson, G., Curran, R., Butterfield, J., and Craig, C. (2008). “The effect of
using animated work instructions over text and static graphics when
performing a small scale engineering assembly.”Collaborative product
and service life cycle management for a sustainable world, R. Curran,
S. Y. Chou, and A. Trappey, eds., Springer, London, UK, 541–550.
Xie, B., and Salvendy, G. (2000). “Prediction of mental workload in single
and multiple task environments.”Int. J. Cognit. Ergon., 4(3), 213–242.
Xu, K., Chiaa, K. W., and Cheok, A. D. (2008). “Real-time camera tracking
for marker-less and unprepared augmented reality environments”Image
Vis. Comput., 26(5), 673–689.
Yamada, A., and Takata, S. (2002). “Reliability improvement of industrial
robots by optimizing operation plans based on deterioration evaluation.”
CIRP Annals-Manuf. Technol., 51(1), 319–322.
Zacks, J. M. (2008). “Neuroimaging studies of mental rotation: A
meta-analysis and review.”J. Cognit. Neurosci., 20(1), 1–19.
Zaeh, M. F., and Wiesbeck, M. (2008). “A model for adaptively generating
assembly instructions using state-based graphs.”Manufacturing
systems and technologies for the new frontier, M. Mitsuishi, K. Ueda,
and F. Kimura, eds., Springer, 195–198.
Zauner, J., Haller, M., Brandl, A., and Hartman, W. (2003). “Authoring of a
mixed reality assembly instructor for hierarchical structures.”Proc.,
Second IEEE and ACM Int. Symp. on Mixed and Augmented Reality,
IEEE Computer Society, Washington, DC, 237–246.
JOURNAL OF COMPUTING IN CIVIL ENGINEERING © ASCE / SEPTEMBER/OCTOBER 2013 / 451
J. Comput. Civ. Eng. 2013.27:439-451.
Downloaded from ascelibrary.org by Curtin Univ of Technology 2009 on 08/16/13. Copyright ASCE. For personal use only; all rights reserved.