Available via license: CC BY 4.0
Content may be subject to copyright.
a Corresponding author: xutianwei@ynnu.edu.cn
The Design and Implementation of Computer Hardware Assembling
Virtual Laboratory in the VR Environment
Shaoming Sun1, Tianwei Xu2,a and Juxiang Zhou1
1Key Laboratory of Educational Information for Nationalities, Ministry of Education, Yunnan Normal University, Kunming, 650500, Yunnan,
China
2Graduate Student Department, Yunnan Normal University, Kunming 650500, China
Abstract. In solving the problems of slowly updated laboratory equipment, heavy hardware wastage and potential
danger in the course of traditional computer hardware assembling experiment, this article proposes a design and
concrete implementation of virtual laboratory for computer hardware assembling based on immersive VR environment.
Adopt 3Ds Max and Unity3d software to create 3D models and build the scene, and Mojing SDK as the VR display
effect and the tool of interactive interface, the article achieved the design and implementation of virtual laboratory
application in mobile devices. Current virtual laboratories based on Virtools can only run in Windows environment,
and this design overcomes such limitation. In addition, with rich scenes and tutorial, this design combines the head-
mounted display and somatosensory controller to greatly promote the immersion and interactivity, thus enhancing the
interests of students. This is a new trail to improve the traditional experimental teaching effect.
1 Introduction
Virtual reality technology, also known as soul technology,
is based on computer technology and uses relevant science
and technology to form a virtual environment similar to the
surroundings. In this virtual environment, learners can do
whatever they want to do in the real world[1]. Virtual
reality has three characters: interaction, immersion and
imagination. According to the degree of immersion and
interaction, virtual reality can be generally divided into
semi-immersive desktop virtual reality system and
immersive virtual reality system[2]. The semi-immersive
desktop virtual reality system mainly relies on tools like
3D modelling provided by computer to create a screen-
based virtual environment. Users can realize human-
computer interaction through input tools such as mouse
and keyboard. The advantages of this system lie in its low
cost and high penetration rate[3]. The disadvantage is the
poor immersion effect caused by the influence of the
operating environment. The immersive virtual reality
system refers to completely closing the visual, auditory
and tactile sense by using a head-mounted display, data
gloves or other handheld devices and thereby achieving a
good immersive experience.
With the development of virtual reality technology,
especially in the first year of VR, well-known companies
such as Google, Facebook, HTC, Xiaomi, and Storm have
directly or indirectly launched their VR products and plans.
Virtual reality has evolved from computer simulation to
immersive virtual reality, undoubtedly providing more
technical support for the application of virtual reality in
education. Its application in education is realized by
students learning in a three-dimensional virtual
environment presented by virtual reality related hardware
and software, whose application value is mainly reflected
in four aspects：firstly, creating a more realistic learning
scenario for learners and increases their learning
experience through multisensory interaction[4]; secondly,
enhancing learners’ motivation and participation[5];
thirdly, allowing learners to learn independently; lastly,
bridging the gap between theory and practice. In recent
years, virtual reality technology has been widely used in
various teaching fields such as physics, chemistry, biology,
history, medicine, agriculture, dance and aerospace,
playing a significant role in assisting and promoting
students’ learning as well as providing a new way to
improve the traditional teaching models.
At present, the application of virtual reality technology
in education has achieved remarkable results. However,
compared with foreign technology and application level,
domestic practice research is still on its initial stage[6]. In
Chinese education domain, most theoretical and practical
studies are based on desktop virtual reality system
(Desktop-VR) with its interactivity and immersion to be
improved. Taking computer hardware assembly course as
an example, many scholars have conducted related
researches, such as Yan Lina of Sichuan Normal
University, Ge Qiaoyan of Zhejiang University of
Technology and Li Qiang of Bohai University, proposing
computer hardware assembly virtual labs based on 3Ds
Max and Virtools technology. On the whole, however,
due to the limitations of development technology and
© The Authors, published by EDP Sciences. This is an open access article distributed under the terms of the Creative Commons Attribution License 4.0
(http://creativecommons.org/licenses/by/4.0/).
MATEC Web of Conferences 232, 01051 (2018) https://doi.org/10.1051/matecconf/201823201051
EITCE 2018
hardware, the virtual labs only support running in the
Windows environment and require learners to interact with
their objects through keyboard and mouse during the
learning process, lacking immersion and interactivity and
leading to insufficient interest in learning, even though
they meet learners’ basic learning needs. This study adopts
Unity3d technology as the development platform, Mojing
SDK as the development tool for VR effects, smart phone
as the operating environment for virtual labs and storm
mirror and daydream sensor handle as virtual reality
display and interactive devices, building a computer
hardware assembly virtual lab in a immersive VR
environment to solve the previous problems that computer
hardware assembly virtual laboratory is not highly
immersive and cannot create real teaching situations which
lead to low learning efficiency.
2 Design of virtual laboratory in the VR
environment
2.1 Teaching content design
Computer Hardware Assembly is one of the compulsory
courses for students majoring in computer application in
secondary vocational and technical schools, playing a vital
role in the whole process of discipline development.
Through the study of this course, learners can have a better
understanding of the basic knowledge of computer
hardware and improve their practical ability to assemble
computers. Therefore, the computer hardware assembly
virtual laboratory is based on the teaching objectives of the
computer hardware assembly of the secondary vocational
school. Different presentation methods are utilized
according to the difficulty level of knowledge points[7].
When designing the teaching content of hardware’s names,
functions and parameters, the three-dimensional model
and text are used to increase the perceptual intuition of
teaching given that the teaching content of this part is
mainly to enable learners to correctly understand the
names, functions and parameters of the computer hardware.
In terms of simulative assembly, the video demonstration
is used at first to help students make clear of the points for
attention in the process of computer hardware assembly
and master how to properly assemble a computer,
considering that this is the core part of the entire virtual
laboratory. After watching the demonstration video,
learners are asked to start the installation situation using
somatosensory handle with an aim to improve their
practical ability. When it comes to designing the teaching
content of common hardware faults, students are supposed
to be familiar with them and corresponding solutions to
enhance their problem-solving ability. Therefore, the
teaching content is presented in the form of text.
2.2 Functional module design
Based on the above analysis of the teaching contents, the
functional module of the virtual laboratory is divided into
three ones: basic learning module, virtual experiment
module and experimental feedback module. The specific
function introduction is shown as follows.
2.2.1 Basic learning module
The basic learning module can be divided into two sub-
modules: hardware model display and hardware function
introduction. The main function of the hardware model
display module is to enable learners to roam in the virtual
world from the first-person perspective through the
somatosensory handle interaction device and observe the
basic structure of each hardware from any angle, greatly
enhancing learning’s intuitiveness and making up for the
teaching problem that the students can't observe the
computer hardware up close and for a long time due to the
lack of hardware equipment in the traditional experimental
teaching. The hardware function introduction module
introduces the basic functions of hardware and the current
mainstream parameters. Students can not only know the
basic functions and parameters of hardware, but also able
to apply the knowledge in daily lives after learning, so that
they can understand the computer performance based on
the configuration list when they purchase a computer in the
future[8]. When certain hardware is updated over time,
administrator can adjust the teaching content in the system
as needed to ensure its cutting edge.
2.2.2 Virtual experiment module
This module is the core part of the entire virtual lab,
consisting of two sub-modules: assembly video demo and
simulative assembly. The teaching content of the assembly
video demonstration is to produce a teaching video of
computer hardware assembly based on learning theory and
teaching theory after analysing the textbook Computer
Hardware and Assembly Maintenance and learner
characteristics majoring in computer application of
secondary vocational schools, helping them understand the
correct installation steps and precautions so as to improve
the scientificity and standardization of the simulative
assembly process. Learners can also watch the video after
simulative assembly as needed and find out the wrong
operations during the assembly process. This part revoles
around learners’ simulating to assemble a computer and
creating a virtual learning environment through virtual
reality technology to get a sense of presence. Learners
operate the somatosensory handle during the learning
process and receive corresponding feedbacks. If correct,
the selected computer hardware will move to the
corresponding position of the motherboard and the system
will give feedback on the correct installation; if wrong, the
system will present corresponding prompt message to help
students complete the experimental task smoothly. The
experimental results are significantly better than some
existing computer hardware assembly virtual labs.
2.2.3 Experimental feedback module
This module is mainly composed of two sub-modules:
common fault cases and experimental tests. Common fault
case module is designed to introduce high-frequency faults
2
MATEC Web of Conferences 232, 01051 (2018) https://doi.org/10.1051/matecconf/201823201051
EITCE 2018
of the computer hardware, such as loss internal storage,
and the corresponding solutions, assisting students to
development a solution in their minds and deal with it
alone when they encounter a hardware failure as well as
improve their practical ability to solve problems. In terms
of the experimental test module, a test is required after the
studying the basic learning module and the virtual
experiment module. The content of the exercises is based
on the teaching objectives and content, going from the easy
to the difficult and complicated, largely increasing
students’ self-confidence and enhance their learning effect.
They can evaluate their own learning results after the test
and fill in the gaps of knowledge.
3 Implementation of Virtual Laboratory
Based in the VR Environment
3.1 The construction of development
environment
This virtual lab uses Unity3d as the main development
environment in software, Mojing SDK as a development
tool for virtual reality and 3Ds MAX as a development tool
for 3D models.
First, create a new project file in Unity3d software and
a new Assets folder in the project directory used to store
Mojing SDK, 3D models, image, audios and videos. In this
way, we can dynamically watch scenes’ finish effect by
calling the memory loading mechanism configured by
Unity3d at any time during the development process with
an aim to improve the loading speed and running
performance of scenes.
Then, import the Mojing SDK into the project file
through Import Package under the Assets menu bar, delete
the original Main Camera in the scene, create a new
MojingMain as the main camera in the virtual lab, and then
utilize the Import New Assets under the Assets menu to
import all the resources needed into the project.
Finally, for each function page, we create multiple
scenes, skyboxes, and ground systems to load its teaching
content. This is the core content of the entire virtual lab.
Command SceneManager. Load Scene to realize the
switches between scenes.
3.2 The realization of the main functional modules
in the virtual laboratory
According to the design of above functional modules, this
essay divides the virtual lab into various learning modules
so that learners can learn the teaching content in the
corresponding modules according to their actual needs. In
the development process of the virtual lab, each
experimental function module is packaged in a separate
scenario and linked by the hierarchical script so that
learners can selectively study as required. The specific
implementation of each scenario is as follows:
3.2.1 Main menu module
As soon as learners enter the virtual lab, they first come
across the menu interface (as shown in Figure 1).
Considering that the use objects of virtual lab are students
majoring in computer application from secondary
vocational school, the design of the interface layout
adheres to the principle of integrity and navigation.
Integrity refers to whether the UI button is consistent with
the background colour in the scene and whether learners
feel the entire scene interface is a whole during use, so as
to avoid unnecessary interference distracting learners’
attention. Navigation refers to whether the set UI button
can navigate accurately and ensure learners clearly choice
the corresponding learning module according to their
actual needs without confusion. A total of four UI
interactive buttons are set in the main menu interface,
correspondingly Basic Learning, Experimental Assembly,
Experimental Feedback, and Experimental Help.
Experimental Help provides a guide for learners who use
the virtual lab for the first time and make them clear of the
precautions during use to help learner can better them
better finish the learning tasks. Learners can choose the
corresponding learning module through gaze interaction.
In terms of realizing gaze interaction, first we should
import the UI resources made in Photoshop CS6 into the
Unity project file, create a canvas in the scene where UI is
placed, add a Box Collider to UI for collision detection and
script code that triggers events to the UI and HeadCtrl in
the Canvas for gaze interaction. The core script code is as
follows:
private void SetSelectedNode()
{
var forward = mainCamera.transform.forward;
if (Physics.Raycast(transform.position, forward, out hit,
000))
{
currentNode=hit.collider.gameObject.GetComponent<
Basiccontrol>();
} }
Figure 1. Main menu interface
3.2.2 Basic learning module
The main function of basic learning module is to create a
highly immersive and interactive virtual learning
environment for learners. During the learning process,
3
MATEC Web of Conferences 232, 01051 (2018) https://doi.org/10.1051/matecconf/201823201051
EITCE 2018
learners can use the somatosensory handle to wander in the
scene from a first-person perspective and observe the
hardware model from any angle according to their own
needs (as shown in Figure 2), which can greatly make up
for the teaching problem in the traditional experimental
teaching process that students cannot observe a certain
hardware at a close distance and for a long time due to the
lack of hardware equipment. Learners can interact with
each hardware model through the head-controlled gaze
interaction during the learning process and form a
preliminary concept in their minds about name, function,
basic parameters and other related knowledge of the
hardware (as shown in Figure 3) to carry out computer
simulation assembly according to different hardware
functions. If students want to learn certain hardware again,
they can push the trigger button on the somatosensory
handle to go back and study it repeatedly.
Figure 2. Basic learning scene interface
Figure 3. Hardware introduction interface
Virtual tour is about making use of the displacement of the
virtual camera to realize the effect of scene exploration. To
be honest, to achieve virtual exploration is to form a
mapping relation between the data of the smartphone
accelerometer sensor and four directions of the virtual
camera displacement, namely, front, back, left and right.
First, create a Character Controller in the scene and add
script code to it. Apply Controller.Move in the Update
function to dynamically update the controller's location in
the script and Input.acceleration method to obtain data of
the phone transmitter. This method is mainly used to get
the sub-scales of X axis, Y axis and Z axis of the mobile
phone, which are represented by Input.acceleration.x，
Input.acceleration.y and Input.acceleration.z. During the
roaming process, the character controller cannot advance
if it collides with other objects in the scene and only turn
left, right or back to avoid penetrating other objects and to
enhance the immersive experience. This function is
realized by object engine provided by Unity3d to add a
Collider to the virtual camera and 3D model in the scene.
The core script code of scene roaming function is as
follows:
private void UpdateFunction()
{
Vector3 moveDistance = Vector3.zero;
if (MoveWithPlatform())
{
Vector3newGlobalPoint=movingPlatform.activePlatfor
m.TransformPoint(movingPlatform.activeLocalPoint);
moveDistance=(newGlobalPointmovingPlatform.activ
e-GlobalPoint);
if (moveDistance != Vector3.zero)
controller.Move(moveDistance);
} }
3.2.3 Virtual experiment module
The virtual experiment module consists of two parts:
assembly video demonstration and simulative assembly.
As for assembly video demonstration, some precautions
and specific operating steps in the process of assembling
computer hardware in the real environment are presented
in the form of video. During use, learners can first interact
with the 3D model of the blackboard in the lab through the
ray emitted by the somatosensory handle. When the
collision event is triggered, the teaching video of computer
assembly will be played, forming a preliminary concept on
learners’ minds about how to install the machine correctly
before the simulative assembly and laying a theoretical
foundation for the subsequent simulative assembly. After
watching the video, learners can simulate the installation
by themselves. Given that the simulation assembly is a
core part of the entire virtual laboratory, the design of the
whole scene follows the principle of education, science
and art. The quality of education and science are mainly
reflected in the design of the content based on the teaching
objectives in line with students' cognitive rules. The
artistic nature is reflected in the layout, colour and light of
the scene interface to arouse students' interest. Learning
interest. In this part, a highly immersive virtual experiment
environment is created to guide students to complete the
simulative assembly experiment based on the theoretical
knowledge previously learned. Taking the simulative
installation of CPU as an example, all the hardware models
on the console can be detected by using a somatosensory
handle to control the direction of rays (as shown in Figure
4). Detecting the 3D model of Hardward means
determining the desired hardware. By this time, the system
records the current location of the detected hardware and
the name of the data and takes the hardware as the launch
point to re-emit the ray to the correct position of the
motherboard. The ray will detect the collider set on the
main board and feed back the corresponding Tag data to
4
MATEC Web of Conferences 232, 01051 (2018) https://doi.org/10.1051/matecconf/201823201051
EITCE 2018
the API in the script. When the value obtained on the
hardware is judged by the value obtained at a certain
position on the main board and matches the condition, the
hardware will move with the ray to the correct position on
the motherboard and give correct feedback. If wrong, the
corresponding prompt is given to help users successfully
complete the experimental task (as shown in Figure 5). The
core script code is as follows:
void FindTarget()
{
if (hit)
{
if (info.transform.tag == "Hardward")
{
if(info.transform.name==HardwardNumber.ToString())
{
targetMoveObject = info.transform.gameObject;
lastPosition = targetMoveObject.transform.position;
isMove = true;
isFind=false;
targetMoveObject.GetComponent<BoxCollider>().ena
bled=false;
HardwardNumber++;
} } } }
Figure 4. Gamma ray detection interface
Figure 5. Simulated assembly interface
3.2.4 Experimental feedback module
Experimental feedback module is a supplement and check
after studying the first two modules, mainly consisting of
common fault cases and experimental tests. The common
hardware failure module is mainly presented in the form
of classification, so that learners can learn more common
faults of hardware in the learning process without
confusion. This module is realized by controlling the ray
direction by a somatosensory handle to collide with each
UI in the scene and enter the corresponding learning unit.
The effect is shown in Figure 6. The design of the
experimental test module abides by the principle of
openness, not only telling right or wrong but also
providing more space for reflection and improvement,
because effective feedback should be oriented and
illuminating and indirectly solve some specific problems
encountered by students during the experiment. Given that
the presenting form is exercise and learners may take some
notes in the learning process, this part is mainly presented
in the form of single screen and interacts in the form of
touch screen. The effect is shown in Figure 7.
Figure 6. Common fault case interface
Figure 7. Experimental test interface
4 Conclusion
In this paper, a virtual laboratory of computer hardware
assembly is built in the VR environment based on the
training needs of computer hardware assembly experiment
teaching in traditional environment and by combining 3Ds
Max and Unity3d technologies. The virtual laboratory uses
VR glasses and somatosensory handles as the presentation
platform and interaction mode so that learners can
immerse themselves in the process. At the same time, the
scene creation and teaching design of the three functional
modules---basic learning module, virtual experiment
module and experimental feedback enhance students'
5
MATEC Web of Conferences 232, 01051 (2018) https://doi.org/10.1051/matecconf/201823201051
EITCE 2018
interest in learning, which not only can effectively make
up for the shortcomings in the traditional experimental
teaching process, but also solve the limitation that the
existing virtual lab based on virtools technology can only
run in Windows environment. In the subsequent researches,
the interaction mode in virtual experimental process and
the degree of refinement of the interaction operation will
be further improved. In addition, AI speech recognition
system will be taken into consideration so that learners can
complete virtual experiments through multiple interaction
modes and the authenticity and operability of the
experiment will be improved. In this way learners can
better complete the experimental tasks and improve their
learning motivation and experience.
References
1. QinPing. Zhao, Scientia Sinica(Informationis) 39, 2-
46 (2009)
2. Gengzhong. Zheng, Computer Systems &
Applications 2, 62-65 (2008)
3. Pei. Pei, The Chinese Journal of ICT in Education 8,
9-4-96 (2018)
4. Nan. Ding, Modern Educational Technology 27, 19-
25 (2017)
5. Yuan. Gao, Dejian. Liu, Zhenzhen. Huang, Huairong.
Huang, e-Education Researchn 37, 77-87+103 (2016)
6. L. Benetazzo, M. Bertocco, F. Ferraris, et al., IEEE
Transactions on Instrumenta-tion & Measurement,
49, 349-356 (2000)
7. B. Spanlang, J.M. Normand, D. Borland, et al.,
Frontiers in Robotics & Ai, 1,9 (2018)
8. E. Roy, M.M. Bakr, R. George,Saudi Dental Journal,
29, 41-47 (2017)
6
MATEC Web of Conferences 232, 01051 (2018) https://doi.org/10.1051/matecconf/201823201051
EITCE 2018
