Applications of virtual reality in maintenance during the industrial product lifecycle: A systematic review

Abstract

As a state-of-the-art computer technology, virtual reality (VR) is considered to play an important role in helping manufacturing companies stay competitive in the international market. However, despite the achievements made in the field of VR, it is still an emerging technology that lacks deeper exploration and development in industrial application scenarios, especially in the coming fourth industrial revolution (Industry 4.0). This paper aims to systematically investigate the applications of VR in industrial maintenance to discover evidence of its values, limitations, and future directions so that VR can be guided to better serve manufacturing enterprises in remaining competitive in the coming Industry 4.0. A systematic literature review (SLR) methodology is adopted to review primary studies on this topic, by which 86 studies are ultimately included. The results show that VR has proved its value in benefiting maintenance issues through the product lifecycle. However, VR is still not an indispensable element for the lifecycle management of products regarding maintenance-related issues. Several key findings are concluded based on the analysis of the 86 studies. This review is valuable for researchers who are interested in the application of VR technology in maintainability design, maintenance training or maintenance task assistance.

Full text

This paper has been approved for publication at Journal of Manufacturing Systems. Please cite as:
Guo, Ziyue, Dong Zhou, Qidi Zhou, Xin Zhang, Jie Geng, Shengkui Zeng, Chuan Lv, and Aimin Hao.
(2020). Applications of Virtual Reality in Maintenance during the Industrial Product Lifecycle: A
Systematic Review. Journal of Manufacturing Systems, in press.
https://doi.org/10.1016/j.jmsy.2020.07.007
Applications of virtual reality in maintenance during the industrial product
lifecycle: a systematic review
Ziyue Guoabc, Dong Zhouabc, Qidi Zhouabc, Xin Zhangbc, Jie Gengabc, Shengkui Zengbc, Chuan Lvabc, Aimin Haoac
a State Key Laboratory of Virtual Reality Technology and Systems, Beijing, China
b School of Reliability and Systems Engineering, Beihang University, Beijing, China
c The School of Computer Science and Engineering, Beihang University, Beijing, China
d State Key Defense Science and Technology Laboratory on Reliability and Environmental Engineering, Beijing, China
Abstract As a state-of-the-art computer technology, virtual reality (VR) is considered to play an important
role in helping manufacturing companies stay competitive in the international market. However, despite the
achievements made in the field of VR, it is still an emerging technology that lacks deeper exploration and
development in industrial application scenarios, especially in the coming fourth industrial revolution (Industry
4.0). This paper aims to systematically investigate the applications of VR in industrial maintenance to discover
evidence of its values, limitations, and future directions so that VR can be guided to better serve manufacturing
enterprises in remaining competitive in the coming Industry 4.0. A systematic literature review (SLR)
methodology is adopted to review primary studies on this topic, by which 86 studies are ultimately included.
The results show that VR has proved its value in benefiting maintenance issues through the product lifecycle.
However, VR is still not an indispensable element for the lifecycle management of products regarding
maintenance-related issues. Several key findings are concluded based on the analysis of the 86 studies. This
review is valuable for researchers who are interested in the application of VR technology in maintainability
design, maintenance training or maintenance task assistance.
Keywords: virtual reality, maintenance, lifecycle, industrial products, Industry 4.0
1. Introduction
Virtual reality (VR) is an advanced computer technology that can give users multiple intuitive sensations while
simulating mechanisms in a physical or imaginary world. There is no unified or strict definition of VR, and it
varies in terms of its purposes and specific setups [1-4]. VR can be regarded as a natural extension of
traditional computer graphics to 3D displays with advanced inputs and outputs [5]. Its research can be traced
back to the 1960s and can be divided into both a hardware aspect [6, 7] and a software aspect [8]. The VR
devices in early years were very cumbersome, expensive, and inefficient. With decades of evolution, the
hardware has decreased in size and become cost-effective, the software has become more efficient, and entire

VR systems can provide users greater spatial immersion. Today, VR applications can provide users not only
immersive sight beyond reality but also hearing, touch and even the ability to interact with virtual objects.
With these considerable strides, VR has experienced spurts in development in many fields in recent years. It
has succeeded in attracting interest from industry and academia.
The fourth industrial revolution (also called Industry 4.0) is remodeling worldwide industries, which has
aroused great interest from practitioners and researchers [9] [10]. Industry 4.0 was originally a concept
proposed by the German government, academic institutions and private enterprises; it aims to use advanced
technologies and ideas to bring industrial manufacturing into a new stage[11], i.e., smart manufacturing [12-
14]. Its purpose is to integrate advanced technologies to improve the quality, productivity, and effectiveness
of the manufacturing industry [15]. The value of the product lifecycle can finally be added [15].
VR is considered one of the advanced front-end technologies in Industry 4.0 for supporting smart working
[12]. Modern industrial products have a complete lifecycle [16], through which maintenance issues persist.
The design and manufacturing phase is an important stage of the entire lifecycle and determines the
performances and properties of a product. Maintainability is a feature of a product given during the design
and manufacturing phase and is the relative ease and economy of time and resources when a product needs to
be maintained or restored [17]. It is defined as “the measure of the ability of an item to be retained in or
restored to specific conditions when maintenance is performed by personnel having specified skill levels,
using prescribed procedures and resources at each prescribed level of maintenance and repair” by MIL-STD-
721 [18]. Maintenance costs account for a significant percentage of the lifecycle costs. According to estimates,
the maintenance cost occupies 10-25% of the direct operating cost for an aircraft [19]. However, well-designed
maintainability can prominently minimize the downtime and reduce the maintenance cost of a product.
Therefore, it is increasingly important for companies to enhance the maintainability of their products to remain
competitive in international markets.
How to consider maintenance in the design stage has always been a concern. Early maintainability design
works rely mainly on physical prototypes or full-scale solid models of products. However, in the design stage
of product development, physical prototypes are often difficult to make. Some necessary analysis and
verification work cannot be carried out, which hinders the implementation of design for manufacturing and
assembly (DFMA) [20, 21], resulting in poor design effects on maintainability. At the end of the design stage,
the product design is frozen. Even if design problems are found, improving maintainability incurs huge design
costs due to schedule and funding reasons.
Currently, maintainability design work is usually done through computer-aided design (CAD) tools, with
which designers can produce animations of maintenance processes and conduct design analyses and
evaluations based on simulations. However, this work is time-consuming, laborious, and subject to expert
experience [22]. If a designer can intuitively feel the virtual prototype of a product in the early design stage
and even interact with it, rather than employing his or her imagination or an animation, the efficiency of the
maintainability design work will greatly improve. The prosperity of VR offers that possibility[23]. From the
perspective of design, VR can be regarded as a natural extension of traditional CAD tools. With advanced 3D
digital technologies, digital mockups (DMUs) of products, maintenance tools, and equipment can be
visualized in a virtual environment (VE). Designers can interact with objects in a VE through interaction
devices to simulate a maintenance process and verify the design status. Thus, in the early stages, VR can be
used to expose problems in maintenance design and help a designer make changes in time. The maintainability
design approach using VR does not require physical prototypes and can help designers better understand a
product, which can overcome the shortcomings of traditional design methods. In addition, VR can be applied
to maintenance training and on-the-spot operation and maintenance (O&M).
VR has now been researched and applied in many academic institutions and industrial sectors. However, it is
still a newly emerging technology with many issues that require further study. This paper aims to address three
questions. How is VR applied in the lifecycles of industrial products? What are the current concerns of VR in
terms of industrial purposes? Where should VR go in the coming Industry 4.0? To provide clear answers to
these three questions, a reasonable entry must be found. In this paper, the maintenance issues of industrial

products are chosen as a starting point to explore the above questions. The maintenance issues are complicated
and include many aspects that should be considered during different lifecycle stages. Many studies in recent
years have researched how VR is leveraged in maintainability and assembly design, on-the-spot O&M, and
maintenance training [24-26]. The rest of this paper is organized as follows. Section 2 conducts a general
analysis of recent studies. Section 3 reports the up-to-date VR applications for maintenance issues of the
product lifecycle and the noteworthy technologies based on the application of VR to maintenance issues.
Based on the above contents, Section 4 points out the future potential and challenges.
2. Study selection and analysis
A systematic literature review (SLR) is leveraged as the methodology of this paper. An SLR is a mature means
for selecting, evaluating and interpreting all studies relevant to a particular research question or topic area, as
proposed by Kitchenham [27]. It has been successfully applied in many fields to summarize existing research,
investigate defects, and identify new opportunities and challenges based on current studies [28-31]. The
process of an SLR can be generally divided into three stages, namely, planning, conducting and reporting
reviews [27], and each phase has concrete steps to follow, which are shown in Fig. 1. The goals of this paper
were stated in Section 1: investigate the current applications of VR for industrial purposes and reveal its future
challenges and potential. A protocol defines the methods that will be used in a systematic review. These
methods usually include setting questions that are intended to be answered, strategies to search for primary
studies, criteria to exclude and include studies, rules for assessing the quality of studies, and data extraction
and synthesis. The questions intended to be answered were also proposed in Section 1. The rest of the methods
are explored in this section.
1.Planning review 1.1 Goal determination
1.2 Protocol development
3. Reporting review 3.1 Structure and contents
3.2 Writing paper
2. Conducting review
2.1 Search strategy
2.2 Study selection criteria
2.3 Quality assessment
2.4 Data extraction
2.5 Data synthesis
Concrete steps
General stages
Fig. 1. An overview of the SLR process.

2.1 Search Strategy
A search strategy should be predefined for a systematic review to ensure the completeness of the search results
and avoid researchers supporting only their preferred literature [27]. This review focuses on primary studies
concerning how VR is actively applied in the lifecycles of industrial products from the perspective of
maintenance, which are retrieved from specific databases that include the Web of Science (WoS), IEEE Xplore
and Scopus. The Boolean operator “AND” is applied in these databases. These three databases provide a
sufficient number of peer-reviewed articles for this SLR, which cover most relevant journal articles and
conference proceedings. The topics “virtual reality AND maintainability” and “virtual reality AND
maintenance” are chosen to retrieve the initial search results. Considering that there is no uniform or strict
definition of VE, these two sets of general strings may yield as many valuable studies as possible.
To ensure that the research content of the literature retrieved is up to date, the timespan is set from 2010 to
2020, during which the field of VR has made impressive strides.
The study search was primarily conducted on January 24, 2019. Then, a subsequent search work was
conducted on March 2, 2020. A total of 2524 studies were retrieved based on the search strategy. It should be
pointed out that because VR is a field of rapid development, there is a possibility that some subsequent studies
were omitted. The EndNote reference management software was used to manage the retrieved studies.
However, 762 studies were found to be duplicated across the three databases, and they were removed from
the search result. Finally, 1762 studies were reserved after the first round of screening.
2.2 Selection criteria
For an SLR, establishing selection criteria is an essential task after studies are retrieved from a database. The
selection criteria commonly consist of inclusion criteria (IC) and exclusion criteria (EC). With these criteria,
authors can downsize these studies and then select the studies that are directly related to the topic of the review.
The establishment of IC and EC is usually based on the experience and knowledge of the authors, as well as
the preference for the journal, institution, and publication year [27]. Some medical researchers have suggested
that this information should be removed before the review process, but the evidence shows that doing so yields
no improvement in the review results [32].
In this review, the establishment of IC and EC is based on the authors’ knowledge and experience regarding
VR and product design. Considering that the topic of this review has a strong industrial connotation, the
authors also queried several experts who have abundant experience in product development and maintenance
service in enterprises for advice on establishing the IC and EC.
Table 1. IC and EC for filtering papers.
No.
Inclusion criteria
1
The paper should be written in English and well structured. The topic of the study should be clear.
2
The study primarily reports on advanced VR-based methods or systems that can be used in practical
industrial cases concerning maintenance.
3
The paper reports on technical research that can improve the effects of VR.
4
The paper should be either a conference proceeding or journal article.
No.
Exclusion criteria
1
The method in the paper is obsolete.
2
The paper proposes only a method or system with no practical application to design or maintenance.
3
The paper does not have a valid unique digital object identifier (DOI).
For the first exclusion criterion shown in Table 1, the bases for judging whether the method is obsolete are as
follows:

⚫ Whether the proposed method can provide the user with an immersive experience;
⚫ If not, whether the method can simulate a physical process, rather than simply producing an “animation.”
The main reason for setting these two criteria is that, in the early days of design processes, many design and
evaluation works mainly relied on the designer to make a virtual simulation in CAD software, which was
presented in an animated manner. Broadly defined, this type of method belongs to VR. However, it does not
let the designer become immersed in the simulation, and the simulation process is based on the experience of
the designer, which does not represent the state of VR.
For the remaining 1762 papers after deduplication, the second-round screening work was divided into two
stages. In the first stage, the authors reviewed the title and abstract based on the IC and EC to determine
whether the paper conforms to the subject. After the first stage, a total of 180 studies were included based on
the IC and EC by three of the authors (i.e., ZG, DZ and QZ). For any discrepancies, majority voting was used
to determine whether the study should be included or excluded. The main reason for the exclusion of the
remaining 1582 articles was that a considerable part of the articles’ themes were inconsistent with this review’s
purpose. For example, a considerable number of articles [33-35] focused on surgical training. In the second
stage, the authors read the full text of the 180 studies filtered from the first stage. Using the IC and EC again,
85 studies were eventually retained. The main reasons for filtering out another 95 studies are threefold. 1)
Some studies claimed that their topic was an integration of VR, but the main content was almost all AR-related,
not VR-related. 2) Although some articles addressed the VR issue, the methods proposed in those articles
were too outdated. For example, some papers proposed a method to conduct virtual simulation based on
DELMIA (Digital Enterprise Lean Manufacturing Interactive Application), but they did not conduct more in-
depth research and simply followed the basic simulation process of DELMIA. In the authors’ opinion, it did
not represent the state of the art of the application of VR in industry. 3) Finally, some papers were not written
in English, and we could not conduct further review work. In addition, study [36] was also selected after a
reference scan.
2.3 Study quality assessment
Study quality assessment is an effective way to control the quality of a study. According to the opinions of
[27], conducting a study quality assessment can make the IC and EC more specialized, help in weighing the
importance of individual studies and highlight future research. There is no consensus regarding how to define
the quality of a study; it depends on the purpose of the article’s use. However, the authors agree with the
opinion suggested by [37, 38], which is that the quality of a study relates to the extent to which research can
eliminate bias and maximize internal and external validity, where validity refers to the extent to which the
method of the study can prevent systematic errors [39]. To conduct a study quality assessment, quality
assessment criteria (QAC) should be established, which are the standards used to determine whether a selected
study is suitable under the review’s scope. QAC are usually in the form of a checklist. Reviewers can assess
the quality of the selected studies by answering questions in the checklist. In many cases, these questions are
qualitative but carry weight. In this review, the authors established a QAC checklist with 5 questions. Each
question carries a weight of “1,” “0.5” or “0” when the answer is “approval,” “partial approval” or
“disapproval,” respectively. Table A.1 shows the QAC checklist for this review.
Table 2. The screening results after different SLR stages.
Database
First
Retrieval
After
Deduplication
After
Abstract
Screening
After Full-
text
Screening
After
Reference
Scanning
After
Quality
Assessment
WoS
609
1762
180
85
86
86
IEEE Xplore
400
Scopus
1515

It should be noted that the study quality assessment provides only a feasible way to assess the quality of the
studies, and thus, the result of the assessment cannot be regarded as the “pure truth” of a study’s quality. There
are multiple contributing reasons. First, many primary studies are often not reported well, so it is difficult to
evaluate the quality of the QAC checklist itself. Although something may not have been reported, it does not
mean that it was not done. Study [27] suggests that researchers obtain more information from the authors of
the study. Second, there is no unified procedure to establish the QAC, which usually vary with the discipline.
This will lead to uncertainty in the QAC to some extent. Last, study quality assessment is a two-sided coin.
On the one hand, it provides researchers with a concrete tool to assess a study. On the other hand, the
subjectivity of researchers during this process will make the assessment results controversial. In study [29],
the authors indicated the subjectivity of the assessment, and they did not exclude any study based on the results.
The results can be regarded only as a reference.
Based on the QAC, two groups of authors assessed the quality of the 86 studies (Group 1: ZG and XZ, Group
2: DZ and QZ). Each group gave a final score for every study. When each group finished the assessment work,
Cohen’s Kappa score was reported to check the consistency. We set the threshold as 3.5 to determine whether
the study is included. All the discrepancies arising during the assessment were processed by the Delphi method
[40] until a final agreement for inclusion was reached. Cohen’s Kappa score of the assessment result of the
two groups was 0.689, which indicates that the assessments from the two groups were substantially consistent.
Table A.2 shows the final assessment results based on the QAC. Table 2 shows the screening results for the
different stages of the SLR in this paper.
2.4 Data extraction
The goal of data extraction is to collect the valuable information from the primary study, which is required by
the authors to conduct a review. In this review, the following data are collected: (1) application fields, (2) the
purpose of using VR, (3) hardware, (4) software and (5) the novel method or technique, if any. To ensure the
integrity of the extracted data, two authors (ZG and QZ) work together to complete the data extraction process
to prevent omissions. Based on these data, we can perform further analysis and synthesis, which are shown in
the following sections.
3. Data synthesis and result analysis
3.1 Application fields of VR in industry
Based on the 86 selected studies, we can conduct an analysis of the industrial application fields of VR. Table
3 shows the field distribution, where IMA represents industrial maintenance and assembly. Seventeen studies
focus on the development of the methods used in VR technology and do not clearly indicate the application
fields. Therefore, these 17 studies are excluded when developing the statistics. In addition, study [25] has dual
application fields (military industry and transportation) and is therefore counted twice.
The nuclear industry is the largest field that applies VR, contributing 30% of the studies. VR has been widely
applied in the nuclear industry and exerts tremendous effects throughout the lifecycles of nuclear components.
The nuclear industry usually involves complex systems, such as the tokamak fusion component, which has
extremely harsh requirements for safety [41-43]. Any oversights during the early design or subsequent
maintenance will lead to a catastrophe. Study [44] notes the several challenges that the W environment in the
steady-state tokamak (WEST) project faced during O&M: (1) intangible actual environment, (2) narrow space
to introduce new components and conduct assembly and maintenance operations, (3) poor fault tolerance and

extreme accuracy, and (4) tight schedule and cost constraints in the production of physical prototypes. To
meet these challenges, VR has been applied because of its ability to enable visualizing a lifelike environment
and to further offer solutions to engineers [45-47]. The benefits of VR to the nuclear industry are obtained
throughout the lifecycle, from the design stage to the operation stage.
IMA is another field that widely utilizes VR, of which prejob training and maintenance task support are two
main application directions. For prejob training, an immersive VR system improves the efficiency of
knowledge transfer with didactic reduction [48-52]. With high-quality stereopsis and immersion, the VR
system trains workers to finish their tasks in a shorter time and with higher accuracy [53]. For on-site task
support, researchers have also developed many VR applications to assist in task execution. Study [54]
proposed an integrated VR system to help an on-site maintenance worker cooperate with a remote expert to
solve undocumented failures. In that case, the VR system provides the remote expert with a synchronous
visual workspace to improve his understanding of the on-site situation. In addition, the VR system enables the
remote expert to create 3D instructions for the on-site task and sends them to a handheld tablet. In another
case, the researcher developed a VR-based teleoperation system for the manipulation of an unmanned aerial
vehicle [55]. The VR scene results in a stable and robust teleoperation, and the quality of the manipulation is
improved.
Transportation, especially the automotive field, is also a domain where VR is widely available as a powerful
tool for designers and engineers. The survey result for different domains in study [56] is also in agreement
with the conclusion that automotive manufacturers benefit greatly from the application of VR. The industrial
application of VR in transportation mainly manifests in the following aspects: (1) car service training and
effectiveness research [57], (2) design verification and evaluation for assembly, maintenance and ergonomics
[58, 59], (3) vehicle health monitoring during the operation stage [60], and (4) pure technology development
and research for improving the effectiveness of VR [25, 61-63]. Due to the time span (2010-2020) set for the
selected studies in this review, a conclusion can be safely drawn that more primary studies and applications
are discoverable if the timeline is enlarged. In addition, many primary applications by car manufacturers may
not be found in the published papers. A feasible way to access them is to conduct interviews or conference
calls. Study [56] reports several applications in practical R&D processes in automotive manufacturers or
institutes, such as the General Motors Design Lab, the Ford Motor Co. and Caterpillar.
For the power industry, the main usage scenario of VR is maintenance training. Compared to traditional
training programs, VR-based training can improve safety, accessibility and effectiveness[64, 65]. Study [66]
pointed out that many important components in power stations have a high demand for maintenance, but the
field environment is inaccessible to trainees because of safety issues. Traditional maintenance programs are
based on 2D materials, such as pictures and manuals, which are ineffective. Alternately, from the perspective
of safety, study [24] noted that the maintenance of high-voltage overhead power lines is high-risk, and any
accidents can be deadly. Therefore, the maintenance workers must be well trained and skilled. Before field
access, they must receive comprehensive training in a safe and lifelike environment.
Table 3. The distribution of the fields of the 86 studies.
Nuclear industry
21
IMA
16
Transportation
10
Electric power
5
Aerospace industry
9
Military industry
3
Oil industry
3
Refrigeration industry
2
Ironmaking industry
1

The aerospace and military industries normally have tight schedules and cost requirements. In addition, the
products usually involve complex systems, which have high demands on safety and availability[67, 68]. By
integrating VR technology and other design methods, time and costs can be dramatically reduced [69], design
flaws can be discovered [70], and the effects on training can be improved [71-76]. For IMA, studies [77-79]
research the general VR-based methods and evaluate their effectiveness. The studies from the oil [80],
refrigeration [81] and ironmaking [82] industries also have familiar VR applications in their respective fields,
such as the aforementioned.
3.2 The effects of VR through the lifecycle
The complete lifecycles of industrial products have several stages. These stages may vary for different types
of products but generally can be divided into the design, manufacturing and operation stages. How to reduce
lifecycle costs in different stages has received more attention from enterprises due to fierce international
competition. Relevant research shows that 70% of product lifecycle costs are determined in the design stage
[17]. Therefore, a good design will bring considerable economic benefits to an enterprise. In actual situations,
the costs in the operation stage are still considerable due to many uncertainties. However, it can be confirmed
that when greater attention is paid to the design stage, opportunities to reduce costs and enhance performance
will increase.
Time
LCC
Concept and Design Prototype and
manufacturing Operation and
maintenance
Actual costs
Opportunity to
reduce costs
Stage
What can
VR do?
IMA verification,
including accessibility,
ergonomics, collision
detection, and safety
•Maintenance training
•Health monitoring
•Maintenance support
•Maintenance scheme
verification
• Design visualization
• Model check
• Assist in decision-
making
• Critical design review
Fig. 2. Lifecycle cost of industrial products and the effects of VR from the perspective of maintenance.
VR can play a role in the entire lifecycles of industrial products in helping to achieve the aforementioned goals
from the perspective of maintenance, which are important in improving product quality and optimizing design
[83-86]. Fig. 2 shows a relationship between lifecycle and cost and the effects of VR in each stage based on
the selected papers. Designers and researchers have found numerous entry points for VR in product lifecycles.
In the authors’ opinion, the most basic function of VR is to realize the visualization of information, which is
the foundation of the other outspread functions. The visualization of VR has two meanings. First, VR
visualizes inaccessible industrial scenarios. There are many reasons for the inaccessibility of real scenes. For
example, study [36] points out that tokamak maintenance involves some crucial activities. However, the
locations are inaccessible to humans because of heat flux and radiation. Visualization of the tokamak could
aid in evaluating and optimizing the remote handling (RH) plan in the design and operation stages. Second,

VR fulfills the need for data visualization. In this application scenario, VR visualizes abstract contents that
could not be previously realized, providing engineers with an interface to better understand products, even in
the early design stages. This process usually requires integrating VR with a numerical model. Because a VR
system can realize visualization and physical simulation, engineers can verify the assembly and maintenance
process of products in the VE at the early stage of design, predict design defects and avoid the rework caused
by these defects in the assembly and maintenance stage. In [44], with the help of HTC VIVE and Unity 3D,
the authors not only visualized the assembly and maintenance VR scene of WEST (W (for tungsten)
environment in steady-state tokamak) but also developed a tool based on a Monte Carlo system to display the
radiation dose rate via a colored fog and dots in a VR scene. A virtual dosimeter was also developed to display
the real-time radiation dose received by an operator and the accumulated dose since the beginning of the
operation (see Fig. 3(a)). In [66], integrating VR with computational fluid dynamics (CFD), the authors
developed a virtual training package for boiler systems in a power station. The package can visualize the
temperature and velocity in the combustion zone of a boiler, with colored flames, contours and streamlines.
Trainees can even hover inside and outside a boiler with the help of a wand to better understand how the boiler
works. Experts can also evaluate the work efficiencies of a boiler and make optimizations (see Fig. 3(b)).
Compared with a traditional display method, VR provides a delicate way to visualize data, by which users can
obtain more details.
(a) (b)
Fig. 3. Data visualization of the radiation dose in WEST [44] (a), and the temperature and velocity in the
combination zone of a boiler [66] (b).
In addition to the advantage of visualization, more extended functions of VR have been developed by
engineers to service the product lifecycle. Study [45] reports the three uses of VR in the international
thermonuclear experimental reactor (ITER) to optimize the lifecycle: design review, RH simulation and man-
in-loop simulation. Fusion components have a strict requirement for high design accuracy. With VR, designers
can detect inconsistencies of the CAD model in the design stage. Equipped with a haptic device and motion
capture (mocap) system, engineers can even conduct advanced assembly and maintenance verification to find
design flaws in accessibility ahead of actual operation tasks. The nuclear industry is a typical field where VR
technologies are fully utilized over the lifecycles of critical components. Several studies [45, 87-92] have
claimed that VR can help nuclear engineers and enforce the optimization principles ALARA (as low as
reasonably achievable).
The automotive industry has also popularized VR as a means to service the entire lifecycle of a vehicle. A
vehicle is a typical system that involves a human-machine interaction. Hence, human factors must be
considered in the design stage. Study [58] indicated the deficiencies of a traditional design method; although
designers can undertake the ergonomic design or evaluation of passenger spaces with a physical human model
or template derived from a database, more detailed and complex problems still cannot be solved due to some
limitations (see Fig. 4). With the assistance of VR, designers can accurately conduct many ergonomic design
works using digital virtual humans with different percentiles in an immersive environment to optimize the
design of the passenger space. Study [59] reports a virtual method to perform ergonomic and safety evaluations

during the assembly of the wheels of a new electric vehicle. This VR-based method can bring many benefits
when compared to the traditional methods. Fig. 5 shows the VR-based design work for a vehicle. In addition
to its application in design review, VR is also successfully used in the operation stage of products. In study
[60], the authors developed an innovative ecosystem, “gAR-age,” to help maintenance workers carry out
vehicle maintenance work. It is a feedback system that improves the interaction between the maintenance
worker and environment to support health monitoring and data-driven decision-making and shorten the time
taken.
Fig. 4. Designers performing ergonomic design with physical human templates. Photograph courtesy of
OPEL.
(a) (b)
Fig. 5. Designers checking a car in an immersive environment (a) [58] and evaluating a tire assembly (b)
[59].
Among these fields, maintenance training is another main application direction. Study [69] reports an
advanced haptic device combined with a unique VR simulation at the German Aerospace Center (DLR) to
help with training for elaborate and expensive on-orbit service (OOS) tasks, such as cleaning up space debris
by teleoperation. VR provides a safe and very realistic environment for training. The integration of a physical
engine and novel haptic algorithm dramatically enhances the immersion of the operations (see Fig. 6). Studies
[71, 93-95] evaluate the effectiveness of VR-based training compared to traditional training by experiments.
Several positive conclusions are drawn: (1) the training effect in a VE is familiar for practical training, that is,
the skill learned based on the VE can be successfully transferred into a real situation [71, 96], (2) the
participants trained through the VE perform better than those with traditional training (learning-by-observing
and explanation), (3) people undergoing VR-based learning-by-doing training have a quicker and better ability
to recover skills after a period than those undergoing traditional training [93], and (4) in terms of the training
effect, there is no significant difference between portable VR hardware and a CAVE (cave automatic virtual
environment) [97]; hence, automotive manufacturers can choose to invest in cheap and advanced portable VR
hardware to launch car service training programs [95].

Fig. 6. DLR’s advanced VR interaction system (HUG) for training on OOS tasks [69].
3.3 Hardware
An integral VR system usually consists of more than one hardware device for full operation. In general, the
hardware of a VR system can be classified into three types based on their effects, namely, display devices,
motion capture devices and interactive devices. A display device is the essential element of a VR system,
which outputs stereoscopic images to users. The ultimate output effects will make the images users see the
same as those of reality. Table 4 shows the display devices considered in the 86 selected studies. Some studies
did not clearly mention the display device they used; thus, they are not included in the statistics.
Table 4. The display devices in the 86 selected studies.
Type
Number
Display wall
13
PC
16
CAVE
5
Portable
device
SENSICS
2
HTC VIVE
9
Oculus Rift
4
NVIS ST50
1
VR1280
1
Vuzix STAR
1
Unstated
1
Display walls, PCs and portable devices are three commonly used types of display equipment. Compared to
the CAVE, these three types have relatively lower costs and space requirements and are therefore more
feasible for most enterprises and labs. Despite the disadvantages of high cost and bulkiness, the CAVE has
high resolution and offers a strong immersive experience. Civilian applications, such as VR games and
teaching, have high requirements on the display effect to achieve user immersion. However, for industrial
applications of VR, the display effect may not always be the top priority. Study [44] notes that the maintenance
and assembly tasks of WEST are delicate, where the gaps of some diverters are only 0.1 mm, which requires
very refined operations. To simulate these operations with VR, plausible reflections on physical interactions,
such as haptic interactions in precise welding tasks, rather than visual fidelity, are required. On the other hand,
with the fast innovation of hardware, portable VR platforms (such as HTC VIVE and Oculus Rift) are closing
the gap in terms of an immersive experience. Study [95] mentioned that when evaluating the effect of car
service training with the system usage scale (SUS) [98] and trust in technology measures (TTM) indexes [99],
there is no obvious difference between the CAVE and a portable device in terms of the effect of car service
training (see Fig. 7).

Fig. 7. The TTM and SUS perceived by participants in two training groups through a CAVE and the
portable device zSpace to learn a car service procedure [95].
Fig. 8. Four-channel CAVE system [100].
Mocap systems are responsible for capturing and processing the motions of users to provide tracking for the
user’s view and positioning for interaction. For the CAVE and display walls, an additional mocap is required
to provide a motion capture function. Common types of mocap technology are optical, inertial, mechanical
and magnetic. Portable VR devices usually embed sensors and algorithms to realize motion tracking. In
addition, somatosensory devices, such as Kinect [101] and Leap Motion [102], can position and capture
motion in specific spaces [103], and they have relatively low costs [104].
In addition to lifelike visual effects, interactive devices are important media for enhancing the reality of VR
environments, through which a user can interact with objects in a VE and receive sensory feedback, such as
haptic and auditory feelings. The use of an interactive device usually requires cooperation with a mocap device
to obtain accurate positions for command execution. Joysticks or wands are the most commonly used
interaction devices. They enable the user to hover in the VE, move objects and execute many other interactions.
Data gloves and haptic devices are other frequently used interaction devices, with which a user can perform
more elaborate operations, such as grasping and picking. They have been widely used in industrial applications,
especially in some cases that have high requirements for elaborate operations. Fig. 8 shows a complete VR
environment that includes a CAVE, mocap and interactive device.

3.4 Development platform
To develop a VR application, one or more platforms are used to create content, generate scenarios and assign
interaction functions. Generally, these platforms can be classified as a programming language, game engine
or 3D application. Not all authors of the 86 studies mentioned the specific platform they used in their research.
Table 5 shows the development platforms used in the 86 studies.
Table 5. The development platforms used in the 86 studies. The number of uses is shown in parentheses.
Platform type
Mentioned in the studies
Programming
language
C++ (5), C# (4), JAVA (2), KML (1), XML
(1), XAML (1), HTML (2), JSON (1)
Game engine
Unity 3D (21), OGRE (2), OSG (5), Bullet (2),
UNREAL (1)
Industrial 3D
application
SolidWorks Composer (1), CATIA (10),
JACK (7), 3ds Max (2), Blender (1), Comos
Walkinside (1), Visionary Render (2),
ParaView (1), VR4Robots (5)
In these studies, the programming languages are used to create VR content and develop user interfaces. Among
these platforms, game engines are the most used. Different from games, the applications of VR in industry not
only pursue a good sense of immersion but also require environmentally correct physical behaviors. These
game engines, such as BULLT, offer dynamic simulation, are open source, and provide an active community
for developers [105]. In addition, game engines give developers greater autonomy to create their VR
applications. Industrial 3D applications are also used, such as SolidWorks Composer, CATIA, and JACK.
With these apps, CAD models can be directly produced. They can also create interactive VR scenarios but
have certain limitations compared to game engines. However, compared to game engines, industrial apps are
user-friendly for designers and engineers with limited knowledge of programming and computer graphics.
3.5 Current research points in industrial applications
The first accepted concept of “virtual reality” came from Ivan Sutherland’s essay “The Ultimate Display [106]”
in 1965 [56], which was published more than 50 years ago. However, the prosperity of VR in various fields
is a more recent occurrence, with significant strides in hardware and computer graphics. With the continuous
expansion of the application scenarios of VR in industrial fields, there are some challenges that need to be
solved.
3.5.1 Content authoring
An important VR application scenario is the O&M of industrial products [107]. Users need to author operating
instructions and information in a VR system to support on-the-spot O&M or training. However, current
authoring solutions have several drawbacks. They do not split the virtual instructions from the object, so the
user has to author the same operations twice for different objects. In addition, users have to author content
from scratch [108, 109].
To solve these problems, study [107] proposed an authoring method consisting of a workflow and an ontology
based on the collaborator’s skills, combined with a digital twin. This method has several advantages: (1) the
time needed to author operations is reduced, from 96 minutes to 51 minutes in a validation case, (2) an
operation can be authored once for different purposes, and (3) there is no requirement on a user’s programming

skill. In study [108], the authors proposed a conceptual industrial AR application based on HTML and
JavaScript Object Notation (JSON). A skilled person can author the maintenance instructions from a
traditional manual without the need to create them from scratch. In addition, the application can switch
between different context presentation modes, namely, AR mode, VR mode and 2D mode (see Fig. 9).
(a) (b) (c)
Fig. 9. Three modes for displaying the content of the airplane landing gear maintenance task: AR mode (a),
VR mode (b), and 2D mode (c). The user can switch between modes through a natural gesture [108].
3.5.2 Situational awareness
Human beings are interactive creatures that enjoy interacting with the outside world [110]. The ultimate VR
system will make users feel that they are in a realistic and interactive environment. To fulfill that objective, in
addition to improving visual fidelity, the perceptions of touch, smell and hearing are important. Through
experiments, several studies [111] [112] [113-115] found that haptic feedback in VR systems has a great
influence on the performance of personnel who participate in VR training or tasks. Thus far, research on
haptics in VR has aroused much attention, which has practical significance. For the nuclear industry, many
fusion components have very sophisticated structures. Operators or robots carry out their work only in narrow
spaces, which requires high professional skill [116] [36]. OOS missions, such as space debris clean-up, are
costly and delicate. VR can provide a safe and effective environment for the simulation and training of OOS
missions. Photorealistic graphics and accurate haptic feedback can greatly enhance the truthfulness of these
simulations [69].
Through an experimental research study, the authors in [116] drew the conclusion that haptic feedback can
enhance the awareness of users in a VR environment for a given task (see Fig. 10). Study [69] proposed a
haptic rendering algorithm and developed a bimanual haptic device called HUG to solve the physical
interaction in an immersive VE. Study [117] introduced a physical component into virtual maintenance
training to produce haptic feedback for the maintenance tool and components, which can better transfer
maintenance skills to a real work environment.
Fig. 10. The experimental environment. The participant sits on a real chair in the CAVE facing a virtual
table and is equipped with a vibrator on his hand [116].

3.5.3 Motion capture
Motion tracking is a very crucial function for a VR system because the correctness of motion tracking and
user orientation will enable an effective interaction between the user and immersive VR environment [87]
[103]. As mentioned in Section 3.3, there are many types of commercial mocap solutions. However, these
solutions require attaching markers or sensors to the joints of the human body to obtain accurate and reliable
motion-tracking results, which is intrusive and costly [103]. In addition, if multiple users in a scene overlap
and create a shielding effect, the effect of motion capture will be greatly reduced. In response to these problems,
study [103] proposed a novel mocap method that fuses multiple Kinects to expand the tracking area and
tolerate the shielding effect between multiple users. A further fusion strategy can integrate Kinect and head-
mounted Leap Motion device to track full-body motion as well as full-hand details (see Fig. 11).
Fig. 11. Fusion strategy that integrates a Kinect device and Leap Motion device to achieve multiperson
tracking and full-hand details in a collaborative VR environment [103].
Although the current mocap solutions have satisfactory accuracy, their technical structure is very sophisticated.
A poor electromagnetic environment or illumination intensity often makes them unable to work properly. The
robustness in complex industrial environments is insufficient.
3.5.4 System running efficiency
An integrated VR system usually consists of several modules or subsystems for full functionality. The efficient
collaboration between these modules has an important impact on the performance of the VR system.
Alternately, in addition to a strong sense of immersion, industrial practitioners have higher requirements for
the performance of a VR system to ensure its good running in a variety of complex industrial scenarios. In
some cases, VR systems are even integrated into the existing platforms to synergistically meet specific usage
requirements. On this level, the VR system needs not only efficient collaboration among internal modules but
also coordination and compatibility with the external environment to improve the overall system running

efficiency. The research in this field is of great significance to improve the popularity of virtual reality in
industry.
For example, study [118] points out that a VR system usually applies real-time interactive systems (RISs),
which consist of many modules, such as image recognition, computation, and interactions. For consistency,
these modules usually have similar semantics and are coupled to each other. However, at the same time, the
reusability requirements of software demand decoupling between these modules. This contradiction is called
the “coupling dilemma.” To address the problem, study [119] proposed five semantic-based techniques for an
RIS to improve software maintainability, which extend the entity-component-system (ECS) pattern and
promote decoupling and modularity. In another case [80], a VR system is integrated into a bilateral tele-
operation system to help engineers perform daily maintenance or verification tasks under the extreme
environmental conditions of oilfields. To solve the problem of long-distance communication between a field
slave (KUKA youBOT) and VR master (Leap Motion), the message queue telemetry transport (MQTT)
communication protocol is applied, which can provide stable and clear field data to the VR master to visualize
an oilfield scene.
4. Discussion and key findings
Based on the foregoing analysis of the 86 studies, several key findings can be concluded and recognized as
benefits or future challenges that need to be further developed. They are summarized as follows:
⚫ VR proves its value to maintenance in the product life cycle
In Section 3.2, the positive effect of VR on maintenance through the product lifecycle was preliminarily
analyzed. Regarding maintenance issues, the 86 studies investigated in this review showed that VR plays
a role not only in a certain stage of the product but also in the whole lifecycle. VR promotes the
effectiveness of maintainability design, which enables multidisciplinary experts to enhance their
comprehension of products and reduces the cost of reworks [120, 121]. In the operation and maintenance
stage, researchers have developed more successful VR paradigms to assist in the maintenance task.
Nuclear and IMA engineers widely use VR to assist them in the teleoperation of robots in extreme
environments to complete maintenance tasks, by which the quality of the manipulation of a robot is
improved [122-125]. Aviation enterprises use virtual reality systems to train new employees. An
immersive training environment offers better knowledge transfer efficiency [126, 127]. The potential of
virtual reality technology for maintenance in the whole life cycle is being explored. The greatest value of
VR is its ability to reduce the time and cost invested by stakeholders in maintenance-related issues at all
stages of production [120, 121, 128].
⚫ VR makes maintenance personnel safer
Although automation technology plays an important role in modern industrial activities, the completion
of maintenance tasks still depends on manual operation or intervention due to the complexity [129].
Occupational safety has always been the focus of maintenance because of possible health problems and
accidents [130]. VR plays a positive role in improving the safety of personnel in the maintenance process,
which is mainly reflected in two aspects. First, VR helps stakeholders vividly simulate the maintenance
of a product, by which a design review can be conducted and the design flaws can be predicted effectively.
VR has been successfully used to help evaluate accessibility [90, 128], radiation dose [122, 131], fatigue
[129] and other human factors [130, 132]. On the other hand, VR facilitates the process of the on-site
maintenance task. Researchers have developed a variety of VR-based interaction paradigms to help in
maintenance training [126, 127], remote operations [55, 133], safety-related alerts [91] and real-time
maintenance guidance [54].
⚫ It is difficult to develop a VR-for-maintenance system from scratch

Although several mature VR devices with good response in civilian fields are on the market, it is another
challenge to build a VR system that can be successfully used to solve maintenance issues in industry, for
which there are multiple reasons. Unlike civil scenarios, industrial maintenance has complicated
environmental restrictions, such as illumination, radiation and electromagnetic compatibility. These poor
environmental conditions challenge the robustness of the VR system, which is usually manifested in the
reliability of motion capture [103] and synchronism of simulation [124]. Second, building a VR
application for maintenance requires many preparatory works. Multidisciplinary knowledge and skills
such as kinematics simulation, digital modeling, process planning and programming, need to be mastered
so that developers can achieve a high-quality system [90, 134], which can be a major challenge for a team
that is not well prepared.
⚫ Haptic feedback is more important for maintenance
One of the research focuses of VR studies is the pursuit of realistically stereoscopic visual effects. After
all, 80% of the external information processed by the brain comes from vision [135]. However, for
maintenance applications, several studies have indicated that haptic feedback from VR systems can better
improve user situational awareness and performance compared with stereoscopic vision input.
Maintenance is a typical process of human-machine interactions, in which most of the interactions are
implemented by the human hand. Therefore, whether VR can accurately simulate the haptic feedback in
the real maintenance process is very important. The experiments of study [111] showed that haptic
feedback is more efficient than visual feedback in improving the performance of maintenance workers,
which can help correct the operation trajectory. Another study [112] also concluded that haptic feedback
is very helpful for the enhancement of situational awareness in teleoperation tasks, reducing the
completion time and improving the quality of manipulation. In the nuclear industry, haptic feedback is
very important, as it improves the accuracy and safety of VR-based remote maintenance tasks [124, 128].
⚫ The integration of VR and maintainability design is still sporadic
Although VR has proved its prominent value in resolving maintenance issues, the problem of integrating
VR into maintenance-related activities in the life cycle has still not been solved. The dilemmas mainly lie
in two aspects. On the one hand, with the rapid development of IT systems, computer-aided technologies
(CAx) have been widely used in the current process of product lifecycle management [136]. Currently,
the application paradigms of these CAx tools have been standardized. As an emerging technology, VR
technology currently cannot seamlessly integrate into the work process of these CAx tools. For example,
studies [44] [45] point out that the limitations of current VR are the difficulties in loading complex CAD
models, and no method can effectively convert CAD models into VR scenes. On the other hand, the
integration of VR into current product maintenance activities lacks a predefined methodology, which
hinders the further application of VR [130]. If stakeholders want virtual reality in their daily maintenance
activities, it involves not only technological innovation but also changes in workflow and manner of
working, especially considering that the implementation of maintenance issues often belongs to more than
one department of an enterprise.
⚫ Evaluating the return on investment is necessary
It is not easy to develop a satisfactory VR-for-maintenance system. The construction of a fully functional
VR system often means that much capital cost is required [48, 92, 137], which usually includes not only the
purchase cost but also the daily maintenance cost after the system is put into use. In addition, the
complexity of VR systems puts forward high requirements for users' professional knowledge and skills
[134]. Therefore, it is very worthwhile for stakeholders to assess the return on investment before they want
to invest in a VR system. As an emerging technology, the effectiveness of VR in industrial application
scenarios should be further confirmed. It has been noted that in some application scenarios, the difference
between virtual models and real objects may lead to maintenance safety problems [125] and evaluation
mistakes [138, 139]. The correct use of VR systems requires experience accumulation. Before investing in
a VR system, stakeholders should make a full assessment.

In addition to the abovementioned key points, within a larger picture, the convergence of VR and other
advanced technologies is also worth exploring against the background of Industry 4.0.
VR is considered one of the key technologies for putting Industry 4.0 into practice [12, 140-142]. Regarding
maintenance issues, the application of VR can be extended to the entire product lifecycle when connected with
other advanced technologies. The IoT unites things and integrates their data so that we can better perceive and
control the digital world. VR can act as a realization approach in the process. Study [88] reports a mobile
safety monitoring system for maintenance work on the Large Hadron Collider that is based on IoT and VR.
The system can gather the radiation dose, environmental temperature, body temperature, air composition and
other data, which are then transmitted and analyzed by on-site users in a real-time fashion. On-site users can
finish this process with a mobile VR-based interface. In study [80], the authors proposed a bilateral
teleoperation system that can perform long-distance inspections of oil and gas equipment under extreme
environmental conditions. The system meets the needs of industrial IoT (IIoT). Through the VR environment
developed by Unity 3D, a skilled operator in a safe environment can perceive the on-site status with the
perceptions of vision, auditory and haptics and then control the KUKA youBOT robot to perform
corresponding operations. In study [107], the authors proposed a method-based ontology and digital twin for
augmented and virtual reality (AVR) operations. The ontology-based digital twin of the real entity was
embedded in an AVR system. According to the workflow proposed by the authors, the authoring work for
AVR operations can be edited once but for different purposes. New AVR operations can be extended without
a new version of AR or VR players. In addition, the user does not need any development experience to perform
the authoring jobs. In another study [66], the authors developed a digital twin based on CFD for the boiler
systems of coal-fired power stations. This digital twin can accurately simulate the changes in temperature and
velocity in a boiler under different parameters; these changes can then be visualized through VR technology.
By systemically investigating 86 studies on the practical application of VR to maintenance for various industry
fields, this review provides evidence that VR is gradually integrating with other advanced technologies to
maximize the benefits to stakeholders, helping them make decisions and increasing work efficiency in
maintenance issues.
5. Conclusions
VR has been a persistently trending topic in recent years among academia and in industry. In this paper, an
SLR is applied to explore the three questions raised in Section 1. To answer these questions, a total of 2524
primary studies were retrieved from 3 databases using given strings. After deduplication and filtering based
on IC and EC, 86 papers were retained. QAC were established to assess the quality of the 86 selected papers.
After the quality assessment, the 86 studies were included, and valuable information was extracted and
synthesized from these studies.
An SLR turns out to be an effective tool for systematically reviewing the primary studies to summarize the
evidence of the benefits and limitations of VR in industrial maintenance. This method can undoubtedly be
extended to other systematic reviews of related topics. By tracking down empirical evidence, this review not
only discovers the value of VR but also finds new challenges and future directions in the coming Industry 4.0:
VR has proved to be a very useful tool for serving the entire lifecycle of industrial products, optimizing
development processes, improving maintenance efficiency and safety, and perhaps best of all, reducing
lifecycle costs. As an emerging technology, VR still faces several challenges. Friendliness needs to be
improved to make it easier for designers or engineers to use or author content according to their purposes.
Portability and robustness need to be improved to better adapt to complex industrial environments. More
importantly, the application paradigm needs to be developed to seamlessly integrate VR into product lifecycle
activities.

To ensure the objectivity and validity of the SLR, a widely accepted literature review method is adopted to
investigate the subject. The establishment of IC and EC is determined after several independent discussions.
When conducting the quality assessment for the included studies, the Cohen Kappa statistic is applied to
measure the disagreement between authors. The Delphi method is adopted to eliminate these disagreements
between authors. This SLR method is completely repeatable and can be used to review relevant topics.
Therefore, it can be concluded that this SLR contributes to the topic of VR in maintenance, and it provides
value for relevant research.
This SLR has two limitations. The second search work was carried out in early 2020; thus, there is a possibility
that a few studies were not retrieved from the database due to publication delays. Second, there exists a certain
subjectivity in the QAC and the process of study quality evaluation, but we have adopted effective methods
to minimize this subjectivity.
Future review works will focus on investigating successful application paradigms of VR through the product
lifecycle to summarize the general implementation procedure of VR in Industry 4.0. In addition, explanatory
interviews of industrial companies will be conducted to investigate how VR affects actual product
development processes in enterprises rather than in laboratories.
Acknowledgements
The authors acknowledge the financial contribution provided by the National Key R&D Program of China
(2017YFF0106407) and Foundation of State Key Defense Science and Technology Laboratory on Reliability
and Environmental Engineering (No. 614200405011217).
Appendix
Table A.1
QAC checklist.
No.
Content
QAC1
Is the purpose of the article well defined?
QAC2
Is the structure of the article complete?
QAC3
Is the method proposed in the article clearly stated?
QAC4
Is the validity of the method proposed in the article well proven
in an actual case?
QAC5
Has the result been sufficiently discussed?
Table A.2 The final assessment results of the studies.
Study
QAC1
QAC2
QAC3
QAC4
QAC5
Total
[88]
1
1
1
1
1
5
[77]
1
1
1
1
1
5
[36]
1
1
1
1
1
5
[46]
1
1
1
0.5
0.5
4
[89]
1
1
1
1
1
5
[41]
1
1
1
1
1
5
[45]
1
1
1
1
1
5
[64]
1
1
1
1
1
5

[24]
1
1
1
1
1
5
[25]
1
1
1
0.5
1
4.5
[60]
1
1
1
1
1
5
[59]
1
1
1
1
1
5
[93]
1
1
1
1
1
5
[95]
1
1
1
1
1
5
[67]
1
1
1
1
1
5
[100]
1
1
1
1
1
5
[68]
1
1
1
1
1
5
[69]
1
1
1
1
1
5
[71]
1
1
1
1
1
5
[78]
1
1
1
1
1
5
[79]
1
1
1
1
1
5
[81]
1
1
1
1
1
5
[109]
1
1
1
1
1
5
[117]
1
1
1
1
1
5
[103]
1
1
1
1
1
5
[119]
1
1
1
1
1
5
[114]
1
1
1
1
1
5
[65]
1
1
1
1
1
5
[63]
1
1
1
0.5
1
4.5
[87]
1
1
1
1
0.5
4.5
[42]
1
1
1
1
0.5
4.5
[43]
1
1
1
1
0.5
4.5
[44]
1
1
0.5
1
1
4.5
[66]
1
1
1
1
0.5
4.5
[137]
1
1
1
0.5
1
4.5
[62]
1
1
1
0.5
1
4.5
[108]
1
1
1
1
0.5
4.5
[61]
1
1
0.5
0.5
1
4
[94]
1
1
1
0.5
0.5
4
[80]
1
1
1
0.5
0.5
4
[107]
1
1
1
1
0
4
[58]
1
1
0.5
0.5
0.5
3.5
[116]
1
1
0.5
0.5
0.5
3.5
[82]
1
1
0.5
0.5
0.5
3.5
[72]
1
0.5
1
0.5
0.5
3.5
[55]
1
1
1
1
1
5
[115]
1
1
1
0.5
0.5
4
[73]
1
0.5
0.5
1
0.5
3.5
[134]
1
1
1
1
1
5
[126]
1
1
1
1
1
5
[49]
1
1
1
1
1
5
[124]
1
1
1
1
0.5
4.5
[50]
1
1
1
1
0.5
4.5
[112]
1
1
0.5
0.5
1
4
[74]
1
1
0.5
1
0.5
4

[127]
1
1
1
0.5
0.5
4
[53]
1
1
1
1
1
5
[132]
1
1
1
1
0.5
4.5
[129]
1
1
1
0.5
0.5
4
[84]
1
1
1
0.5
1
4.5
[92]
1
1
1
0.5
0.5
4
[75]
1
1
1
0.5
0.5
4
[133]
1
1
0.5
0.5
0.5
3.5
[121]
1
1
1
0.5
0.5
4
[47]
1
0.5
1
1
0
3.5
[54]
1
1
1
0.5
0.5
4
[128]
1
1
1
1
0.5
4.5
[131]
1
1
0.5
0.5
0.5
3.5
[90]
1
0.5
1
1
0.5
4
[51]
1
0.5
0.5
1
0.5
3.5
[52]
1
1
1
1
1
5
[96]
1
1
1
0.5
0.5
4
[120]
1
0.5
1
1
0.5
4
[85]
1
1
1
0.5
0.5
4
[113]
1
1
1
0.5
0.5
4
[123]
1
0.5
0.5
1
0.5
3.5
[122]
1
0.5
0.5
1
0.5
3.5
[91]
1
1
1
1
0.5
4.5
[111]
1
1
1
0.5
0.5
4
[125]
1
1
0.5
1
0.5
4
[86]
1
0.5
0.5
1
0.5
3.5
[76]
1
1
1
0.5
1
4.5
[48]
1
1
1
1
1
5
[138]
1
1
1
1
1
5
[139]
1
1
1
1
0.5
4.5
[130]
1
1
1
1
1
5
References
[1] J. Steuer, Defining virtual reality: Dimensions determining telepresence, Journal of communication, 42 (1992) 73-93.
[2] S. Valin, A. Francu, H. Trefftz, I. Marsic, Sharing viewpoints in collaborative virtual environments, in: Hawaii International
Conference on System Sciences, 2001.
[3] R. Packer, K. Jordan, W. Gibson, Multimedia :from Wagner to virtual reality, W.w.norton, (2002).
[4] J.Q. Coburn, I. Freeman, J.L. Salmon, A Review of the Capabilities of Current Low-Cost Virtual Reality Technology and Its
Potential to Enhance the Design Process, Journal of Computing & Information Science in Engineering, (2017).
[5] S. Jayaram, H.I. Connacher, K.W. Lyons, Virtual assembly using virtual reality techniques, Computer-Aided Design, 29 (1997)
575-584.
[6] I.E. Sutherland, A head-mounted three dimensional display, in: Afips, 1968.
[7] F.C. Huang, K. Chen, G. Wetzstein, The light field stereoscope: immersive computer graphics via factored near-eye light field
displays with focus cues, ACM, 2015.

[8] G. Reitmayr, D. Schmalstieg, An open software architecture for virtual reality interaction, in: Acm Symposium on Virtual
Reality Software & Technology, 2001.
[9] F. Chiarello, L. Trivelli, A. Bonaccorsi, G. Fantoni, Extracting and mapping industry 4.0 technologies using wikipedia,
Computers in Industry, 100 (2018) 244-257.
[10] Y.X. Liao, F. Deschamps, E.D.R. Loures, L.F.P. Ramos, Past, present and future of Industry 4.0-a systematic literature review
and research agenda proposal, International Journal of Production Research, 55 (2017) 3609-3629.
[11] M.L. Hoffmann Souza, C.A. da Costa, G. de Oliveira Ramos, R. da Rosa Righi, A survey on decision-making based on system
reliability in the context of Industry 4.0, Journal of Manufacturing Systems, 56 (2020) 133-156.
[12] A.G. Frank, L.S. Dalenogare, N.F. Ayala, Industry 4.0 technologies: Implementation patterns in manufacturing companies,
International Journal of Production Economics, 210 (2019) 15-26.
[13] H. Kagermann, J. Helbig, A. Hellinger, W. Wahlster, Recommendations for implementing the strategic initiative INDUSTRIE
4.0: Securing the future of German manufacturing industry; final report of the Industrie 4.0 Working Group, Forschungsunion, 2013.
[14] F. Tao, Q. Qi, A. Liu, A. Kusiak, Data-driven smart manufacturing, Journal of Manufacturing Systems, 48 (2018) 157-169.
[15] L.S. Dalenogare, G.B. Benitez, N.F. Ayala, A.G. Frank, The expected contribution of Industry 4.0 technologies for industrial
performance, International Journal of Production Economics, 204 (2018) 383-394.
[16] S. Fukushige, K. Yamamoto, Y. Umeda, Lifecycle scenario design for product end-of-life strategy, Journal of Remanufacturing,
2 (2012) 1.
[17] R.F. Stapelberg, Handbook of Reliability, Availability, Maintainability and Safety in Engineering Design, United Kingdom
Springer, (2009).
[18] A.M. ARMY-AM-HQ US, DOD-HDBK-791, (1988).
[19] C.H. Friend, Aircraft maintenance management, (1992).
[20] K.L. Edwards, Towards more strategic product design for manufacture and assembly: priorities for concurrent engineering,
(2002).
[21] O. Battaia, A. Dolgui, S.S. Heragu, S.M. Meerkov, M.K. Tiwari, Design for manufacturing and assembly/disassembly: joint
design of products and production systems, International Journal of Production Research, 56 (2018) 7181-7189.
[22] H.L. Lockett, K. Arvanitopoulos-Darginis, An Automated Maintainability Prediction Tool Integrated with Computer Aided
Design, Procedia CIRP, 60 (2017) 440-445.
[23] X. Liu, G. Peng, X. Liu, Y. Hou, Development of a collaborative virtual maintenance environment with agent technology,
Journal of Manufacturing Systems, 29 (2010) 173-181.
[24] A.A. Garcia, I.G. Bobadilla, G.A. Figueroa, M.P. Ramirez, J.M. Roman, Virtual reality training system for maintenance and
operation of high-voltage overhead power lines, Virtual Reality, 20 (2016) 27-40.
[25] A. Marzano, I. Friel, J.A. Erkoyuncu, S. Court, Design of a Virtual Reality framework for maintainability and assemblability
test of complex systems, in: J. Erkoyuncu (Ed.) Cirpe 2015 - Understanding the Life Cycle Implications of Manufacturing, 2015,
pp. 242-247.
[26] G. Peng, H. Yu, X. Liu, J. Yang, X. He, A desktop virtual reality-based integrated system for complex product maintainability
design and verification, Assembly Automation, 30 p.333-344.
[27] B. Kitchenham, Procedures for performing systematic reviews, Keele, UK, Keele University, 33 (2004) 1-26.
[28] G. Mujtaba, L. Shuib, N. Idris, W.L. Hoo, R.G. Raj, K. Khowaja, K. Shaikh, H.F. Nweke, Clinical text classification research
trends: Systematic literature review and open issues, Expert Systems with Application, 116 494-520.
[29] R. Palmarini, J.A. Erkoyuncu, R. Roy, H. Torabmostaedi, A systematic review of augmented reality applications in maintenance,
Robotics And Computer-Integrated Manufacturing, 49 (2018) 215-228.
[30] A. Booth, A. Sutton, D. Papaioannou, Systematic approaches to a successful literature review, Sage, 2016.
[31] E. Ruschel, E.A.P. Santos, E.d.F.R. Loures, Industrial maintenance decision-making: A systematic literature review, Journal of
Manufacturing Systems, 45 (2017) 180-194.
[32] J.A. Berlin, Does blinding of readers affect the results of meta-analyses?, The Lancet, 350 (1997) 185-186.
[33] L.M. Knab, A.H. Zureikat, H.J. Zeh, III, M.E. Hogg, Towards standardized robotic surgery in gastrointestinal oncology,
Langenbeck's Archives of Surgery, 402 (2017) 1003-1014.
[34] L.M. Guseila, A. Saranathan, E.L. Jenison, K.M. Gil, J.J. Elias, Using virtual reality to maintain surgical skills during periods
of robotic surgery inactivity, Journal of Robotic Surgery, 8 (2014) 261-268.

[35] A. Winkler-Schwartz, K. Bajunaid, M.A.S. Mullah, I. Marwa, F.E. Alotaibi, J. Fares, M. Baggiani, H. Azarnoush, G. Al Zharni,
S. Christie, A.J. Sabbagh, P. Werthner, R.F. Del Maestro, Bimanual Psychomotor Performance in Neurosurgical Resident Applicants
Assessed Using NeuroTouch, a Virtual Reality Simulator, Journal of Surgical Education, 73 (2016) 942-953.
[36] P. Dutta, N. Rastogi, K.K. Gotewal, Virtual reality applications in remote handling development for tokamaks in India, Fusion
Engineering and Design, 118 (2017) 73-80.
[37] P. Alderson, S.R. Green, J. Higgins, Cochrane Reviewers' Handbook 4.2.2, (2004).
[38] K.S. Khan, G. Ter Riet, J. Glanville, A.J. Sowden, J. Kleijnen, Undertaking systematic reviews of research on effectiveness:
CRD's guidance for carrying out or commissioning reviews, NHS Centre for Reviews and Dissemination, 2001.
[39] D. Moher, A.R. Jadad, G. Nichol, M. Penman, P. Tugwell, S. Walsh, Assessing the quality of randomized controlled trials: an
annotated bibliography of scales and checklists, Controlled clinical trials, 16 (1995) 62-73.
[40] N. Dalkey, O. Helmer, An experimental application of the Delphi method to the use of experts, Management science, 9 (1963)
458-467.
[41] D. Keller, L. Doceul, F. Ferlay, G. Jiolat, J.J. Cordier, I. Kuehn, B. Manfreo, J. Reich, ITER design, integration and assembly
studies assisted by virtual reality, Fusion Engineering and Design, 88 (2013) 1951-1954.
[42] C. Louison, F. Ferlay, D. Keller, D.R. Mestre, Operators’ accessibility studies for assembly and maintenance scenarios using
virtual reality, Fusion Engineering and Design, 124 (2017) 610-614.
[43] G. Micciche, L. Lorenzelli, F. Frascati, G. Di Gironimo, R. Mozzillo, Remote Handling Refurbishment Process for the European
IFMIF Target Assembly: Concept Design, Simulation and Validation in Virtual Environment, IEEE Transactions on Plasma Science,
45 (2017) 1824-1830.
[44] L. Meunier, D. Keller, P. Guédon, Virtual Reality: Lessons learned from WEST design and perspectives for nuclear environment,
Fusion Engineering and Design, 136 (2018) 1337-1341.
[45] D. Keller, L. Doceul, F. Ferlay, C. Louison, A. Pilia, K. Pavy, L. Chodorge, C. Andriot, Use of virtual reality for optimizing the
life cycle of a fusion component, Fusion Engineering and Design, 101 (2015) 186-191.
[46] A. Pilia, C. Brun, L. Doceul, L. Gargiulo, J.C. Hatchressian, D. Keller, R. Le, S. Poli, B. Zago, Application of virtual reality
tools for assembly of WEST components: Comparison between simulations and physical mockups, Fusion Engineering and Design,
98-99 (2015) 1589-1592.
[47] J.F. Koning, M.R. De Baar, B.S.Q. Elzendoorn, C.J.M. Heemskerk, D.M.S. Ronden, W.J. Schuth, Analysis of ITER upper port
plug remote handling maintenance scenarios, Fusion Engineering and Design, 87 (2012) 515-519.
[48] E. Blumel, T. Haase, Virtual Reality Platforms for Education and Training in Industry, in: J. Grundspenkis, M. Kirikova, Y.
Manolopoulos, L. Novickis (Eds.) Advances in Databases and Information Systems, 2010, pp. 1-7.
[49] J.J. Roldan, E. Pena-Tapia, P. Garcia-Aunon, J. Del Cerro, A. Barrientos, Bringing Adaptive and Immersive Interfaces to Real-
World Multi-Robot Scenarios: Application to Surveillance and Intervention in Infrastructures, Ieee Access, 7 (2019) 86319-86335.
[50] N. Randeniya, S. Ranjha, A. Kulkarni, G. Lu, Virtual Reality Based Maintenance Training Effectiveness Measures - A Novel
Approach for Rail Industry, in, Institute of Electrical and Electronics Engineers Inc., 2019, pp. 1605-1610.
[51] A.G. Hequet, Maximize efficiency and safety of smelters through advanced multipurpose simulator solutions, in, 2012, pp.
909-911.
[52] C.J.M. Heemskerk, M.R. De Baar, H. Boessenkool, B. Graafland, M.J. Haye, J.F. Koning, M. Vahedi, M. Visser, Extending
Virtual Reality simulation of ITER maintenance operations with dynamic effects, Fusion Engineering and Design, 86 (2011) 2082-
2086.
[53] D.Y.d. Moura, A. Sadagic, The Effects of Stereopsis and Immersion on Bimanual Assembly Tasks in a Virtual Reality System,
in: 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), 2019, pp. 286-294.
[54] M. Kleiber, T. Alexander, C. Winkelholz, C.M. Schlick, Ieee, User-centered Design and Evaluation of an Integrated AR-VR
System for Tele-Maintenance, in: Proceedings 2012 Ieee International Conference on Systems, Man, and Cybernetics, 2012, pp.
1443-1448.
[55] G.A. Yashin, D. Trinitatova, R.T. Agishev, R. Ibrahimov, D. Tsetserukou, AeroVr: Virtual Reality-based Teleoperation with
Tactile Feedback for Aerial Manipulation, in: 2019 19th International Conference on Advanced Robotics (ICAR), 2019, pp. 767-
772.
[56] L.P. Berg, J.M. Vance, Industry use of virtual reality in product design and manufacturing: a survey, Virtual Reality, 21 (2017)
1-17.

[57] (!!! INVALID CITATION !!! [53-55]).
[58] M.H. Abidi, A.M. El-Tamimi, A.M. Al-Ahmari, S.M. Darwish, M.S. Rasheed, Virtual Ergonomic Assessment of First Saudi
Arabian Designed Car in a Semi-Immersive Environment, in: M. Sreekumar, M. Zoppi, P. Nithiarasu (Eds.) International Conference
on Design and Manufacturing, 2013, pp. 622-631.
[59] A. Kulkarni, A. Kapoor, Asme, VIRTUAL TOOLS FOR SAFETY AND ERGONOMIC EVALUATIONS OF ELECTRIC
VEHICLE ARCHITECTURE, 2014.
[60] S. Shah, S. Petluru, R. Singh, S. Srivastava, M. Assoc Comp, gAR-age: A Feedback-Enabled Blended Ecosystem for Vehicle
Health Monitoring, 2018.
[61] G. Di Gironimo, R. Mozzillo, A. Tarallo, From virtual reality to web-based multimedia maintenance manuals, International
Journal on Interactive Design and Manufacturing, 7 (2013) 183-190.
[62] S. Qiu, Q. He, X. Fan, D. Wu, Virtual human hybrid control in virtual assembly and maintenance simulation, International
Journal of Production Research, 52 (2014) 867-887.
[63] S. Qiu, X. Fan, D. Wu, Q. He, D. Zhou, Virtual human modeling for interactive assembly and disassembly operation in virtual
reality environment, International Journal Of Advanced Manufacturing Technology, 69 (2013) 2355-2372.
[64] P.N.A. Barata, M.R. Filho, M.V.A. Nunes, Consolidating Learning in Power Systems: Virtual Reality Applied to the Study of
the Operation of Electric Power Transformers, IEEE Transactions on Education, 58 (2015) 255-261.
[65] X.Y. Yin, Y. Gu, S.G. Qiu, X.M. Fan, VR&AR Combined Manual Operation Instruction System on Industry Products: A Case
Study, in: X. Shen, X. Zhang, Z. Zhou, G. Zhang, X. Luo (Eds.) 2014 International Conference on Virtual Reality and Visualization,
2014, pp. 65-72.
[66] F. Li, L.T. Shen, G.W. Tang, B. Wu, C.Q. Zhou, J.E. Heim, Asme, DEVELOPMENT OF A COMPREHENSIVE VIRTUAL
TRAINING PACKAGE FOR POWER PLANT BOILER, 2014.
[67] C. Dopart, K. Jackson, M.D. Carpenter, B.E. Cohanim, J.A. Hoffman, A mission guided investigation of operational functions
and user interface for Mobility Augmenting Jetpack with Integrated CMGs (MAJIC), in: 2014 IEEE Aerospace Conference, 2014,
pp. 1-9.
[68] J. Geng, Y. Li, R.R. Wang, Z.L. Wang, C. Lv, D. Zhou, A virtual maintenance-based approach for satellite assembling and
troubleshooting assessment, Acta Astronautica, 138 (2017) 434-453.
[69] M. Sagardia, K. Hertkorn, T. Hulin, S. Schätzle, R. Wolff, J. Hummel, J. Dodiya, A. Gerndt, VR-OOS: The DLR's virtual reality
simulator for telerobotic on-orbit servicing with haptic feedback, in: 2015 IEEE Aerospace Conference, 2015, pp. 1-17.
[70] Z. Guo, D. Zhou, J. Chen, J. Geng, C. Lv, S. Zeng, Using virtual reality to support the product’s maintainability design:
Immersive maintainability verification and evaluation system, Computers in Industry, 101 (2018) 41-50.
[71] F. Ganier, C. Hoareau, J. Tisseau, Evaluation of procedural learning transfer from a virtual environment to a real situation: A
case study on tank maintenance training, Ergonomics, 57 (2014) 828-843.
[72] P. Fernandes, A. Pinheiro, G. Cruz, A.M. Maia, L. Morgado, P. Martins, H. Paredes, B. Fonseca, J.B. Lopes, J. Cravino, Demo:
Multi-user virtual world simulator of F-16 aircraft engine mechanical maintenance, in: 2013 2nd Experiment@ International
Conference (exp.at'13), 2013, pp. 166-167.
[73] X. Shao, X. Wei, S. Liu, Research On Aircraft Virtual Assembly Technology Based On Gesture Recognition, in: 2019 IEEE
1st International Conference on Civil Aviation Safety and Information Technology (ICCASIT), 2019, pp. 147-150.
[74] M. Numfu, A. Riel, F. Noel, Virtual reality based digital chain for maintenance training, in: G.D. Putnik (Ed.), Elsevier B.V.,
2019, pp. 1069-1074.
[75] X. Li, Q. Gao, Z. Zhang, X. Huang, Collaborative virtual maintenance training system of complex equipment based on
immersive virtual reality environment, Assembly Automation, 32 (2012) 72-85.
[76] S.R. Bowling, Evaluating the effectiveness of a priori information on process measures in a virtual reality inspection task,
Journal of Industrial Engineering and Management, 3 (2010) 221-248.
[77] N. Gavish, T. Gutierrez, S. Webel, J. Rodriguez, M. Peveri, U. Bockholt, F. Tecchia, Evaluating virtual reality and augmented
reality training for industrial maintenance and assembly tasks, Interactive Learning Environments, 23 (2015) 778-798.
[78] F. Ferrise, G. Caruso, M. Bordegoni, Multimodal training and tele-assistance systems for the maintenance of industrial products,
Virtual and Physical Prototyping, 8 (2013) 113-126.
[79] N. Yuviler-Gavish, S. Krupenia, D. Gopher, Task analysis for developing maintenance and assembly VR training simulators,
Ergonomics in Design, 21 (2013) 12-19.

[80] J.E. Naranjo, E.C. Lozada, H.I. Espin, C. Beltran, C.A. Garcia, M.V. Garcia, Flexible Architecture for Transparency of a
Bilateral Tele-Operation System implemented in Mobile Anthropomorphic Robots for the Oil and Gas Industry, Ifac Papersonline,
51 (2018) 239-244.
[81] D. Regazzoni, C. Rizzi, Digital Human Models and Virtual Ergonomics to Improve Maintainability, Computer-Aided Design
and Applications, 11 (2013) 10-19.
[82] C.B. Vieira, V. Seshadri, R.A.R. Oliveira, P. Reinhardt, P.M.P. Calazans, J.B. Vieira Filho, Applying virtual reality model to
green ironmaking industry and education: ‘a case study of charcoal mini-blast furnace plant’, Transactions of the Institutions of
Mining and Metallurgy, Section C: Mineral Processing and Extractive Metallurgy, 126 (2017) 116-123.
[83] X. Jian, S. Cai, Q. Chen, A study on the evaluation of product maintainability based on the life cycle theory, Journal of Cleaner
Production, 141 (2016) 481-491.
[84] P.M. Liston, A. Kay, S. Cromie, C. Leva, M. D'Cruz, H. Patel, A. Langley, S. Sharples, S. Aromaa, Evaluating the iterative
development of VR/AR human factors tools for manual work, Work, 41 (2012) 2208-2215.
[85] D. Faas, J.M. Vance, BREP IDENTIFICATION DURING VOXEL-BASED COLLISION DETECTION FOR
HAPTIC MANUAL ASSEMBLY, 2012.
[86] S. Burrafato, A. Maliardi, P. Ferrara, T. Grasso, E. De Marchi, R. Campaci, Virtual reality in D&C: New approaches towards
well digital twins, in, Offshore Mediterranean Conference (OMC), 2019.
[87] Z. Yang, T. He, L. Shang, P. Long, L. Hu, Development of high-immersive simulation system for designing maintenance
strategy and its application to CLEAR-I, Annals of Nuclear Energy, 83 (2015) 309-315.
[88] M.F. Alam, S. Katsikas, O. Beltramello, S. Hadjiefthymiades, Augmented and virtual reality based monitoring and safety system:
A prototype IoT platform, Journal of Network and Computer Applications, 89 (2017) 109-119.
[89] K. Chizhov, M.K. Sneve, I. Szoke, I. Mazur, N.K. Mark, I. Kudrin, N. Shandala, A. Simakov, G.M. Smith, A. Krasnoschekov,
A. Kosnikov, I. Kemsky, V. Kryuchkov, 3D simulation as a tool for improving the safety culture during remediation work at
Andreeva Bay, Journal of Radiological Protection, 34 (2014) 755-773.
[90] D. Keller, C. Dechelle, L. Doceul, S. Madeleine, J.P. Martins, Y. Measson, J.C. Patterlini, J. Wagrez, ITER Equatorial Port plug
engineering: Design and remote handling activities supported by Virtual Reality tools, Fusion Engineering and Design, 86 (2011)
2105-2108.
[91] A. Cryer, G. Kapellmann-Zafra, S. Abrego-Hernández, H. Marin-Reyes, R. French, Advantages of Virtual Reality in the
Teaching and Training of Radiation Protection during Interventions in Harsh Environments, in: 2019 24th IEEE International
Conference on Emerging Technologies and Factory Automation (ETFA), 2019, pp. 784-789.
[92] Z.R. Li, J. Wang, Z.Y. Yan, X.Y. Wang, M.S. Anwar, An Interactive Virtual Training System for Assembly and Disassembly
Based on Precedence Constraints, in: M. Gavrilova, J. Chang, N.M. Thalmann, E. Hitzer, H. Ishikawa (Eds.) Advances in Computer
Graphics, Cgi 2019, 2019, pp. 81-93.
[93] S. Borsci, G. Lawson, D. Salanitri, B. Jha, When simulated environments make the difference: the effectiveness of different
types of training of car service procedures, Virtual Reality, 20 (2016) 83-99.
[94] W.X. Quevedo, J.S. Sanchez, O. Arteaga, M. Alvarez, V.D. Zambrano, C.R. Sanchez, V.H. Andaluz, Virtual Reality System for
Training in Automotive Mechanics, in: L.T. DePaolis, P. Bourdot, A. Mongelli (Eds.) Augmented Reality, Virtual Reality, and
Computer Graphics, Avr 2017, Pt I, 2017, pp. 185-198.
[95] S. Borsci, G. Lawson, B. Jha, M. Burges, D. Salanitri, Effectiveness of a multidevice 3D virtual environment application to
train car service maintenance procedures, Virtual Reality, 20 (2016) 41-55.
[96] T. Gutiérrez, J. Rodríguez, Y. Vélaz, S. Casado, A. Suescun, E.J. Sánchez, IMA-VR: A multimodal virtual training system for
skills transfer in Industrial Maintenance and Assembly tasks, in: 19th International Symposium in Robot and Human Interactive
Communication, 2010, pp. 428-433.
[97] C. Cruz-Neira, D.J. Sandin, T.A. Defanti, R.V. Kenyon, J.C. Hart, The Cave - Audio Visual Experience Automatic Virtual
Environment, Communications of the Acm, 35 (1992) 64-72.
[98] J. Brooke, SUS: A quick and dirty usability scale, 1995.
[99] D. McKnight, M. Carter, J. Thatcher, P. Clay, Trust in a specific technology: An Investigation of its Components and Measures,
2011.
[100] Z.Y. Guo, D. Zhou, J.Y. Chen, J. Geng, C. Lv, S.K. Zeng, Using virtual reality to support the product's maintainability design:
Immersive maintainability verification and evaluation system, Computers in Industry, 101 (2018) 41-50.

[101] F. Menna, F. Remondino, R. Battisti, E. Nocerino, Geometric investigation of a gaming active device, in: F. Remondino, M.R.
Shortis (Eds.) Videometrics, Range Imaging, and Applications Xi, 2011.
[102] T. Guzsvinecz, V. Szucs, C. Sik-Lanyi, Suitability of the Kinect Sensor and Leap Motion ControllerA Literature Review,
Sensors, 19 (2019).
[103] P. Leoncini, B. Sikorski, V. Baraniello, F. Martone, C. Luongo, M. Guida, Multiple NUI Device Approach to Full Body
Tracking for Collaborative Virtual Environments, in: L.T. DePaolis, P. Bourdot, A. Mongelli (Eds.) Augmented Reality, Virtual
Reality, and Computer Graphics, Avr 2017, Pt I, 2017, pp. 131-147.
[104] Y. Ma, K. Mithraratne, N.C. Wilson, X. Wang, Y. Ma, Y. Zhang, The Validity and Reliability of a Kinect v2-Based Gait
Analysis System for Children with Cerebral Palsy, Sensors (Basel, Switzerland), 19 (2019).
[105] J. Hummel, R. Wolff, T. Stein, A. Gerndt, T. Kuhlen, An Evaluation of Open Source Physics Engines for Use in Virtual Reality
Assembly Simulations, in: G. Bebis, R. Boyle, B. Parvin, D. Koracin, C. Fowlkes, S. Wang, M.H. Choi, S. Mantler, J. Schulze, D.
Acevedo, K. Mueller, M. Papka (Eds.) Advances in Visual Computing, Isvc 2012, Pt Ii, 2012, pp. 346-357.
[106] I. E. Sutherland, The Ultimate Display, 2001.
[107] H. Vincent, J. Benoit, S. Xavier, B. David, Inoovas — Industrial ontology for operation in virtual and augmented scene: The
architecture, in: 2017 4th International Conference on Control, Decision and Information Technologies (CoDIT), 2017, pp. 0300-
0305.
[108] T. Engelke, J. Keil, P. Rojtberg, F. Wientapper, S. Webel, U. Bockholt, Ieee, Content First - A concept for Industrial Augmented
Reality Maintenance Applications using Mobile Devices, in: 2013 Ieee International Symposium on Mixed and Augmented Reality,
2013, pp. 251-252.
[109] F. Tian, P. Wang, C. Dong, J. Gao, H. Tang, L. Qian, H. Xu, 3D design tools for equipment manufacturing and exhibition
based on internet, in: 2016 International Conference on Audio, Language and Image Processing (ICALIP), 2016, pp. 40-46.
[110] S. Dan, Designing Gestural Interfaces, Oreilly Vlg Gmbh & Co, (2008).
[111] D. Chamaret, S. Ullah, P. Richard, M. Naud, Integration and evaluation of haptic feedbacks: From CAD models to virtual
prototyping, International Journal on Interactive Design and Manufacturing, 4 (2010) 87-94.
[112] J.v. Oosterhout, C.J.M. Heemskerk, H. Boessenkool, M.R.d. Baar, F.C.T.v.d. Helm, D.A. Abbink, Haptic Assistance Improves
Tele-Manipulation With Two Asymmetric Slaves, IEEE Transactions on Haptics, 12 (2019) 141-153.
[113] D. Faas, A hybrid method for haptic feedback to support manual virtual product assembly, in, 2011, pp. 1491-1498.
[114] L. Pelliccia, F. Klimant, A. De Santis, G. Di Gironimo, A. Lanzotti, A. Tarallo, M. Putz, P. Klimant, Task-based motion control
of digital humans for industrial applications, in: R. Teti (Ed.) 10th Cirp Conference on Intelligent Computation in Manufacturing
Engineering - Cirp Icme '16, 2017, pp. 529-534.
[115] Y.F. Yang, P. Yang, J. Li, F. Zeng, M. Yang, R. Wang, Q. Bai, Research on virtual haptic disassembly platform considering
disassembly process, Neurocomputing, 348 (2019) 74-81.
[116] C. Louison, F. Ferlay, D. Keller, D. Mestre, Acm, Vibrotactile feedback for collision awareness, 2015.
[117] F. Loch, U. Ziegler, B. Vogel-Heuser, Integrating Haptic Interaction into a Virtual Training System for Manual Procedures in
Industrial Environments, IFAC-PapersOnLine, 51 (2018) 60-65.
[118] M.E. Latoschik, H. Tramberend, Short Paper: Engineering Realtime Interactive Systems: Coupling and Cohesion of
Architecture Mechanisms, in: Eurographics Conference on Virtual Environments & Second Joint Virtual Reality, 2010.
[119] M. Fischbach, D. Wiebusch, M.E. Latoschik, Semantic Entity-Component State Management Techniques to Enhance Software
Quality for Multimodal VR-Systems, IEEE Transactions on Visualization and Computer Graphics, 23 (2017) 1342-1351.
[120] W. Ghadban, V. Kozina, B. Kelly, Improving project delivery using virtual reality in design reviews -a case study, in, Society
of Petroleum Engineers, 2019.
[121] S.P. Leino, S. Lind, J. Heikkilä, O. Uuttu, Integration of manual work related information to PLM, in, 2010, pp. 1353-1360.
[122] L. Dan, Z. Jiawan, Z. Yi, The virtual simulation system of nuclear radiation dose field based on virtual reality technology, in:
2019 Chinese Automation Congress (CAC), 2019, pp. 4381-4385.
[123] B.S.Q. Elzendoorn, M. De Baar, D. Hamilton, C.J.M. Heemskerk, J.F. Koning, D.M.S. Ronden, Analysis and validation center
for ITER RH maintenance scenarios in a virtual environment, Fusion Engineering and Design, 87 (2012) 263-266.
[124] N. Rastogi, A. Kumar Srivastava, Control system design for tokamak remote maintenance operations using assisted virtual
reality and haptic feedback, Fusion Engineering and Design, 139 (2019) 47-54.
[125] C. Chabal, A. Proietti, J.F. Mante, J.M. Idasiak, Virtual Reality Technologies: a Way to Verify Dismantling Operations First

application case in a highly radioactive cell, 2011.
[126] T.D. Rupasinghe, M.E. Kurz, C. Washburn, A.K. Gramopadhye, Virtual reality training integrated curriculum: An aircraft
maintenance technology (AMT) education perspective, International Journal of Engineering Education, 27 (2011) 778-788.
[127] M. Muszyńska, D. Szybicki, P. Gierlak, K. Kurc, A. Burghardt, M. Uliasz, Application of Virtual Reality in the Training of
Operators and Servicing of Robotic Stations, in: L.M. Camarinha-Matos, H. Afsarmanesh, D. Antonelli (Eds.), Springer New York
LLC, 2019, pp. 594-603.
[128] K. Kim, B.S. Park, H.D. Kim, S. Hassan, J. Yoon, Robotic virtual manipulations of a nuclear hot-cell digital mock-up system,
Assembly Automation, 31 (2011) 17-28.
[129] L. Ma, D. Chablat, F. Bennis, W. Zhang, B. Hu, F. Guillaume, Fatigue evaluation in maintenance and assembly operations by
digital human simulation in virtual environment, Virtual Reality, 15 (2011) 55-68.
[130] F. Bernard, M. Zare, J.C. Sagot, R. Paquin, Integration of human factors into the design process of helicopter maintainability,
Human Factors and Ergonomics in Manufacturing & Service Industries, 29 (2019) 350-360.
[131] R. Kelly, R. Skilton, J. Naish, Real-time volumetric rendering of radiation fields using 3D textures, Fusion Engineering and
Design, 146 (2019) 551-554.
[132] M. Mahdjoub, D. Monticolo, S. Gomes, J.C. Sagot, A collaborative Design for Usability approach supported by Virtual Reality
and a Multi-Agent System embedded in a PLM environment, CAD Comput Aided Des, 42 (2010) 402-413.
[133] I. Li, Y. Liu, X. Zang, Human-Machine Interaction Control System of a Remote Maintenance Manipulator for Tokamak, in,
Institute of Electrical and Electronics Engineers Inc., 2019, pp. 336-340.
[134] A. Shamsuzzoha, R. Toshev, V. Vu Tuan, T. Kankaanpaa, P. Helo, Digital factory–virtual reality environments for industrial
training and maintenance, Interactive Learning Environments, (2019).
[135] C. Haupt, A.B. Huber, How axons see their way - axonal guidance in the visual system, 13 (2008) 3136-3149.
[136] C. Werner Dankwort, R. Weidlich, B. Guenther, J.E. Blaurock, Engineers' CAx education—it's not only CAD, Computer-
Aided Design, 36 (2004) 1439-1450.
[137] B.L. Li, Y.X. Bi, Q. He, J. Ren, Z.H. Li, A low-complexity method for authoring an interactive virtual maintenance training
system of hydroelectric generating equipment, Computers in Industry, 100 (2018) 159-172.
[138] F. Bernard, M. Zare, J.C. Sagot, R. Paquin, Using Digital and Physical Simulation to Focus on Human Factors and Ergonomics
in Aviation Maintainability, Human Factors, 62 (2020) 37-54.
[139] F. Bernard, M. Zare, J.C. Sagot, R. Paquin, Virtual Reality Simulation and Ergonomics Assessment in Aviation Maintainability,
in: S. Bagnara, R. Tartaglia, S. Albolino, T. Alexander, Y. Fujita (Eds.) Proceedings of the 20th Congress of the International
Ergonomics Association, 2019, pp. 141-154.
[140] C.J. Turner, W. Hutabarat, J. Oyekan, A. Tiwari, Discrete event simulation and virtual reality use in industry: new opportunities
and future trends, IEEE Transactions on Human-Machine Systems, 46 (2016) 882-894.
[141] M. Kerin, D.T. Pham, A Review of Emerging Industry 4.0 Technologies in Remanufacturing, Journal of Cleaner Production,
(2019) 117805.
[142] S. Mittal, M.A. Khan, D. Romero, T. Wuest, A critical review of smart manufacturing & Industry 4.0 maturity models:
Implications for small and medium-sized enterprises (SMEs), Journal of Manufacturing Systems, 49 (2018) 194-214.