Content uploaded by Vladeta Stojanovic
Author content
All content in this area was uploaded by Vladeta Stojanovic on Apr 30, 2021
Content may be subject to copyright.
A Conceptual Digital Twin for 5G Indoor
Navigation
Vladeta Stojanovic1, Hossein Shoushtari2, Cigdem Askar3, Annette Scheider4,
Caroline Schuldt5, Nils Hellweg6, and Harald Sternberg7
1, 2, 3, 4, 5, 6, 7Department of Hydrography and Geodesy, HafenCity University Hamburg, Germany
Email: {name.surname}@hcu-hamburg.de
Abstract—With the introduction of practices from Industry 4.0
in various Architecture, Engineering, Construction, Owner and
Occupant (AECOO) sectors, especially in Facilities Management
(FM), new requirements concerning integration of digital data
sources arise with prominence. Particularly with the increased
use of the Digital Twin (DT) paradigm, the need to capture, store,
process and present various data concerning the current and
predicted states of the built environment becomes paramount.
A conceptual and prototypical system design, focusing on in-
tegration, processing, analysis and visualization of data related
to indoor navigation within modern buildings, is presented and
discussed. Approaches for processing and presentation of key
data sources, particularly indoor point clouds and as-is Building
Information Modeling (BIM) data, combined with simulated
5G signal as localization approximation, are presented and
discussed. A particular focus is placed on leveraging the Service-
Oriented Architectures and Systems (SOA/SOS) paradigm for
implementation of complex systems for meeting integration
requirements of such diverse data sources. Finally, we discuss
how such approaches can benefit from current and predicted use
of 5G technologies, and provide experimental results from a case
study conducted within a modern university campus building.
The presented case study results demonstrate the feasibility of
our approach, and provides a framework for future expansion,
integration and evaluation with planned 5G infrastructure.
Index Terms—Point Clouds; BIM; Service Oriented Comput-
ing; Localization; 5G Simulation
I. INTRODUCTION
Current advancements in adaptation of Industry 4.0 prac-
tices for Architecture, Engineering, Construction, Owner and
Occupant (AECOO) stakeholders, particularly in the realm
of Facility Management (FM), have created a paucity for
the research and development of integration strategies for
digital data sources [1]. Such data sources may include
as-designed/built/as-is Computer-Aided Design (CAD) and
Building Information Modeling (BIM) data, digital and dig-
itized FM documents as well as historical or current sensor
data - all pertaining to the past, present and future operational
state of a building [2]. With the current standardization of
5G technology, additional data sources captured via 5G in-
frastructure should be available for assessing the current state
of the built environment, for, e.g., indoor navigation [3] or
FM within a Smart Building context [4]. The fusion of all
these data sources into a dynamic representation is based on
the currently promoted Digital Twin (DT) paradigm [5]. A
DT is able to encapsulate and represent the physical state
of an object as its cyberphysical counterpart [6]. The use
of DTs has potential benefit for engagement of stakeholders
through interactive visualization [7] and enables an up-to-date
virtual representation of their facilities and related states and
processes [8].
A. Problem Statement
The focus of this research is on indoor navigation based
on the DT paradigm. The deployment and use of a DT
requires access to various data sources related to the specific
lifecycle stage of the building. For FM applications, historical
and current data sources are needed in order to perform key
analytics and generate results for furthering FM stakeholder
engagement. Additionally, future states also need to be pre-
dicted. This poses three key problems:
1) The selection and analysis of required data used to
represent the historical and current states of a building
(e.g., existing BIM data).
2) The selection and analysis of required data used to
generate predictions for future states of a building (e.g.,
signal and sensor data obtained from 5G infrastructure).
3) The combined analysis and representation of these data
sources.
B. Research Contributions
To address these problems, we present an approach for
integration, processing, analysis and visualization of key data
sources for indoor environments. Our focus is on indoor
localization and navigation tasks with a FM context (Fig. 1).
An SOA for integration of such data for DT representations
is presented and discussed. A prototypical implementation of
key system components is tested using data from a real-life
university building with a simulation for 5G-based localiza-
tion. We also use synthetically generated point clouds, based
on as-is BIM data, in order to test our point cloud processing
and reconstruction methods. The fusion of historical, current
and predicted data sources is then used to generate indoor
navigation results, which are interactively visualized alongside
existing BIM and FM data (e.g., 2D floor plans, 3D models and
point clouds). The final output is presented as a web-based,
interactive visualization aimed at engaging FM stakeholders
for indoor routing and navigation tasks.
Fig. 1. The proposed approach of using 5G data and BIM data within a DT paradigm for indoor navigation and localization tasks related to FM.
II. RE LATE D WOR K
A. Requirements for Indoor Navigation Modeling
A key requirement of indoor navigation is the determination
of navigable areas within the chosen representation (e.g.,
2D floor plan, BIM, etc.). A navigable area is defined as
a region in 2D or 3D space where an agent representing
a user or an autonomous entity is able to move around -
in order to get from one location to another. A common
name for data structures used to represent such navigable
areas are ”routing graphs” [9] or ”navigation meshes” [10].
The use of routing graphs is therefore a requirement for
pedestrian indoor navigation, especially for approximation
of optimal paths (e.g., using Dijkstra’s algorithm and its
variants [11]). Existing BIM data and semantics (e.g., Industry
Foundation Classes (IFC) schematics [12]) can be used in
order to determine the geometric boundary constraints of
areas that are navigable [13]. Methods for discretization of
3D space and approximation of topological derivatives using
geometric computation principals also play an important role
for generation of routing graphs [14] [15].
B. Capture and Representation of Indoor Environments
In AECOO applications, the use of point clouds enables the
area-wise capture of intricate physical details of indoor and
outdoor built environments, real-world objects and locations,
with varying levels of visual fidelity [16]. Point clouds can be
generated by using photogrammetric techniques and by laser
scanning [17] [18]. In laser scanning, laser signals emitted by
3D scanners are used to capture the reality based on time of
flight or phase shift methods [19]. For AECOO applications,
in particular Operations & Maintenance (O&M) procedures
within FM, the current state of the physical environment can be
captured using point clouds and analyzed for further decision
making [20].
The simulation of the laser scanning process can also be
performed, known as ”synthetic data” generation, using as-
designed/as-built or as-is 3D models of indoor environments
as the main processing input [21]. The output of such a process
is a point cloud with similar qualities as that of a physically
captured point cloud, and can be used for various testing key
processes (e.g., training machine and deep-learning models for
semantic segmentation [22]).
Apart from manual generation of as-is and as-built BIMs,
the application of the “Scan2BIM” process enables automated
reconstruction of segmented point clouds to semantically-rich
and higher-level geometric representations (e.g., as-is BIM
data in the form of IFC models [23]). This can also include
both 2D and 3D floor plan generation [24]. Since point
clouds themselves are ambiguous, they need to be processed
and enriched semantically in order to make them useful for
visualization and analysis applications [25].
C. Localization Methods within the 5G Paradigm
The increased use of smartphones with integrated cost
and energy efficient sensors for navigation and localization
correspond to the ubiquitous positioning vision proposed by
Park in [26]. Determining the optimal route to get between
two points is often a bottleneck for FM tasks, and solving
such a problem requires the approximation of the user’s
current location within a building as well as the approximate
location of the destination point. One key approach to solving
this problem is based using autonomous localization methods
- independent from any other infrastructure solution, e.g.,
WLAN or UWB networks, which need an app registration
or a predefined label dataset [27]. However, such autonomous
localization performance would be limited both in physics-
based solutions [28] [29], or machine learning-based ap-
proaches [30] [31], due to the well known sensor drift and
the absence of a realistic dataset respectively [27].
The current introduction and adaptation of 5G networks
is of great interest for application of indoor localization
within a BIM-based FM usage context [32]. The use of
5G infrastructure and technology could increase the accuracy
of indoor navigation applications, and would therefore be
useful to consider for FM-related tasks by providing benefits
within a Smart Building context (e.g., stable and low-latency
communication, multi-device support and use for data-heavy
streaming applications such as Augmented and Virtual Reality
(AR/VR) [4]). Since the 5G technologies would be available
to the vast majority of the population, it can be expected to be
a staple technology to benefit indoor navigation, and capable
of solving current open localization challenges in the built
environment [27]. Variants of such 5G infrastructure (e.g.,
using a specialized 5G network that uses slightly different
frequencies than a ’normal’ network), can also provide similar
benefits.
D. Service-Oriented Architectures and Systems
The use of the DT paradigm further places demand on
availability of historical and current digital information for
a given building [33]. The use of service-oriented computing
and system architecture enables the design and deployment
of complex software components and services, which are
capable of meeting the requirements of accessing, storing
and processing versatile data sources - along with the benefit
of hardware decoupling between the server and the client
systems [34].
A particular advantage of using SOA when implementing
SOS-based FM software systems (e.g., CAFM, BAS and
IWMSs [35]), is the processing and streaming of specific
results related to FM tasks for a given building (i.e., visu-
alizations of recent spatial changes of office furniture and/or
equipment, occupancy levels, environmental sensor data, ap-
proximations for indoor localization,etc. [36]). Once received,
these results can be presented on any connected client devices
of stakeholders running, e.g., a compatible web-browser on a
commodity mobile device [37].
III. APP ROAC H
The main paradigm for presentation, interaction and deci-
sion making is envisioned with the use of a DT. The type
of data utilized by the DT would depend on its intended
use case, but in most cases the data needs to represent the
physical characteristics of the entity that is presented digitally.
For FM-specific applications this would include as-is point
clouds and their generated semantics, as-is BIM data, as well
as any existing data pertaining to the current representation and
operational status of a building (e.g., Mechanical, Electrical
and Plumbing (MEP) plans and status reports, floor plans, as-
built/as-designed BIM, current or historical sensor data, etc.).
The fusion of these data sources when generating the required
result of the user’s computation tasks is the essence of the
DT [38].
The Data Processing and Data Analytics components of
such a DT system would be implemented as services that are
run when a specific task is requested, and would process and
stream the result of the task back to the user. Such an approach
fits well with the use of service-oriented computation, and is
the recommended approach for implementation of DTs for
AEC applications [39]. All of the processes can be automated
and/or simulated, and included as software components and
services within a SOS implementation (Fig. 2).
a) Point Cloud Processing: The capture of the as-is
physical state of indoor environments can be accomplished
with the use of point clouds. The use of indoor laser scanning
can produce point clouds representing intricate details and can
provide useful base-data for further processing and semantic
enrichment (Fig. 3) However, if the capture of point clouds
is not possible or prohibited due to various factors, their
capture can be simulated using existing 3D models of indoor
environments as the main inputs. Using methods such as
Monte Carlo point sampling [40], or complete LiDAR device
simulation [41], synthetically generated point clouds can be
used as data sources for experimental assessment of key
processing, semantic enrichment and visualization methods.
Further processing of captured or generated point clouds
commonly involves registration, sub-sampling, noise filtering
and per-point normal vector computation. Such tasks can be
automated within a point cloud processing pipeline, and imple-
mented with an SOS [42]. Most point cloud data management
is currently done at the file level and there are different
file formats specifically used for storing point cloud data
(e.g., LAS [43]). As point clouds can be very large, they
cannot always be loaded completely into system memory. To
enhance performance for accessing, analyzing and viewing
point clouds, different spatial data structures such as octrees
or k-d trees can be used [44].
b) Database Integration: For FM use where point clouds
need to be generated frequently, storing and distributing thou-
sands of separate point cloud files is not sufficient for basic
point cloud data manipulation. Here, a Database Management
System (DBMS) represents an alternative that is easier to oper-
ate and scalable [45]. A relational or non-relational DBMS can
be used, with additional special spatial indexing operations, in
order to quickly access point cloud representations of, e.g.,
entire floors or specific rooms of a building, for processing
and analysis. This also allows for the integration of point
clouds with additional static (e.g., IFC models [46]), and
spatio-temporal data sources (e.g., collected or real-time sensor
data [36]), using non-relational DBMS that can accommodate
the changing representation specifications for smart building
related data.
c) Semantic Enrichment of Point Clouds: While point
clouds can be used to represent intricate details of indoor en-
vironments that can be interpreted by viewers with domain ex-
pertise, point clouds themselves do not contain any semantics
by default. Therefore, the use of supervised and unsupervised
machine and deep-learning algorithms is needed to add useful
semantics to point clouds (e.g., prior to reconstruction to as-
is BIM representations). Unsupervised methods include seg-
mentation and clustering methods such as RANSAC, Region
Growing, k-means and DBSCAN clustering [47] [48]. These
methods attempt to group together points that share similar
features (e.g., spatial position, color, normal vector direction,
etc.) or fit to a geometric primitive (e.g., a hyperplane using
variants of the Least Squares method).
Supervised methods include the use of learning models
where examples of what the model should output is based on
observations from existing data (i.e., a posteriori knowledge).
This commonly includes the use of Convolutional Neural
Networks (CNNs) that can classify 2D and 3D data (e.g., point
clouds, images of point clouds and higher-level geometric
representations, e.g., voxels) into user defined categories using
a classification model trained on previous examples [49].
d) As-Is BIM and Routing Graph Data Generation:
The generation of as-is BIM data is based on the detection
of semantically segmented point clusters, from which the
geometry is used to infer the dimensions of the element to
be reconstructed, and the label is used to infer the semantic
associated with the element, usually with association to an IFC
specification at a given Level of Detail (LOD). Semantically
Fig. 2. High-level representation of key processes, system integration, data sources and hierarchy representation for the conceptual use of a DT for indoor
navigation tasks.
Fig. 3. An example of a point cloud of one of the areas in the university
building presented in the case study, captured using indoor laser scanning and
sub-sampled to 556 410 points.
segmented point clouds can also be used to generate 2D/3D
floor plans. Such floor plans can either be 2D vector image
data or 3D triangulated meshes.
For indoor navigation applications, besides geometric rep-
resentations, an additional geometric data structure is needed
to represent navigable areas. With a routing graph represen-
tation, the edges represent navigable paths, while the nodes
are represented by vertices. Routing graphs and navigation
meshes can be obtained both from 2D and 3D floor plan
representations. For 2D floor plan representations, they are
commonly generated using medial-axis approximation meth-
ods [50], while for 3D floor plans they can be derived either
from the triangulated mesh topology representing the floor
area, or from the semantics derived from the IFC elements
representing navigable and room boundary elements [13].
e) 5G Simulation, Processing and Localization Methods:
For navigation applications, the use of localization methods
is required for approximating the user’s position in relation
to their surroundings. In order to approximate the user’s
initial location, different absolute positioning algorithms (e.g.,
triangulation, trilateration and multilateration), can be used to
determine the approximate location of the user [28]. Addi-
tionally, the user’s current location can be estimated based
on the fusion of the maps information, routing graphs, the
absolute positions and the received sensor data readings by
using different state estimation algorithms (e.g., Monte Carlo
Particle Filtering or Extended Kalman Filtering) [29].
With the availability of precise localization offered by the
coming releases of 5G networks, user’s mobile devices can
capture and process localization data, as well as provide
additional sensor data output such as barometer readings (i.e.,
used to measure height changes). The proposed approach relies
on such input data in order to determine the user’s approximate
location, and can further make use of IoT and Smart Building
paradigms by utilizing environmental sensor sources (e.g.,
RFID and NFC tags in building rooms [51]).
f) Representation and Interactive Visualization: Engage-
ment with stakeholders through visualization plays an im-
portant role in furthering understanding amongst all levels
of domain expertise. The use of interactive visualization,
particularly for FM tasks, is crucial for enabling the visual
representation and interpretation of key built environment ele-
ments and processes (e.g., planned renovations, item inventory,
emergency route planning, etc.) [20]. The use of interactive
visualization can greatly benefit indoor navigation tasks, where
there is a need to combine as-is representations of buildings
(either as 2D floor plans and/or with combined/exclusive 3D
representations [52]). In such cases, stakeholders can use the
as-is representation to interactively view their current location
within the building, and use generated optimal paths for
navigation between set markers.
Visualization of key data sources is enabled using real-
time 3D rendering, mainly using existing game engines (e.g.,
Unity [53]) or Web3D frameworks (e.g., Three.js [54]). If
dealing with complex data sources (e.g., visually complex
3D models or point clouds), out-of-core and SOS rendering
methods can be utilized [44]. The representation of the as-is
built environment, either as a raw or semantically enriched
point cloud, or as an reconstructed higher-level geometric
representation with associated semantics (e.g., 2D/3D floor
plans, IFC models, etc.), enables stakeholders to visualize the
environment as they may see it in real life, but with additional
visualization metaphors and idioms [55].
IV. CAS E STU DY
A. Overview
(a)
(b)
Fig. 4. (a) The planned 5G antenna locations around the perimeter of the
building used in the case study. (b) An example of the architectural floor plan
of one of the main floors of the university building.
A 5G Non-Standalone (5G NSA) campus network and
an additional experimental system will be realized at the
presented university building location between Q3 2021 and
Q1 2022. Applications from the field of indoor navigation
will be tested there and will serve research purposes. The
planned campus network will consist of four outdoor antennas
arranged around the building (Fig. 4(a)). Currently this process
is in the planning stages, and more specific implementation
details will be determined in the near future. For the planned
outdoor and indoor network, the use of private frequencies
in band B43/N78 (3.7–3.8 GHz) with a Long-Term Evolution
(LTE) anchor band is planned. The aim is to set up, operate
and use the networks as realistically as possible, as they can
subsequently be set up and used as productive systems at other
locations. The project will also set up a second experimental
network. This network will work with frequencies in the range
of 26 GHz–78 GHz to achieve even higher accuracy for the
estimation of the position. In addition to this and for further
research purposes, indoor units will also be provided on two
floors of the building. In this way, the project will create
different scenarios for testing indoor navigation.
Prior to the implementation of this infrastructure, it was
decided to test key components in terms of their feasibility
to generate and process data required for representation for
indoor localization and navigation. As the 5G infrastructure
is still absent, it was decided to simulate partially the key
missing aspects, in order to validate the feasibility claim of
the DT paradigm, presented approach and prototypical SOA
and SOS. We therefore simulate the indoor 5G network for
the presented case study, using the basic cellular positioning
measurements and algorithms. The reference points are used
as the basis of such a simulation.
We particularly focus on the aspect of localization and
navigation within a given floor space of the main university
building (Fig. 4(b)). Using an as-is BIM data of the fourth floor
area of the university building, we simulate the laser scanning,
positions and signals of 5G antennas. We use this simulated
5G data to approximate the coordinates of the user as they are
virtually navigating the indoor area. For the implemented case
study, we present and discuss key SOS components for pro-
cessing data sources for generating indoor navigation results
that are presented to the user through interactive visualization.
The virtual navigation and user interaction is enabled through
a web-based application, while processing and analysis of data
sources is implemented server-side as SOS components.
B. Prototypical System Design and Implementation
We present a conceptual SOA, with key processes imple-
mented as SOS software components and a Web3D-based
client application (Fig. 5). The Web3D-based client is able
to display either a point cloud or as-is BIM representation
of an indoor environment, the user’s simulated and approxi-
mated position. The generated optimal path between the user’s
current location and a defined point anywhere on the as-is
BIM/point cloud can also be visualized. This optimal path,
the as-is BIM/point cloud and the user’s approximate location
are displayed to the user interactively - enabling the user to
inspect the each of the 3D scene elements from different views.
In terms of the back-end implementation, the prototype
application is able to process point cloud data and create
an initial version of the 2D floor plan. We also make use
of a manually generated IFC model at LOD 300 of the
same floor plan area, in order to provide additional data
source for analysis and visualization of indoor navigation
tasks. Additionally, the simulated 5G signal and localization
estimates are implemented, along with a software component
for optimal path generation using routing graphs generated
from either the 2D floor plan or the 3D as-is BIM. The
access to metadata related to the 2D floor plan, BIM, point
cloud and FM data is provided via a DBMS also running
on the server, while the actual files are stored on the same
server within a specified directory structure. For the back-
end systems, PostgreSQL DMBS is used as object-relational
DBMS. With its spatial database extension PostGIS [56], it
provides support for georeferenced objects that enable location
queries (e.g., for processing of 2D floor plan representations).
The simulated 5G capture works based on the reference points
and can be combined with physics-based sensor fusion (i.e.,
pedestrian dead reckoning). The accuracy of such simulation
can be defined by considering the current 3GPP release
specification [57]. In other words, having the truth trajectory
leads to a 5G simulation data by giving them some noise
Fig. 5. A high-level system design for the presented prototype application, with key SOS components implemented server-side.
and timestamp. The time stamps are estimated using manual
video analysis. The simulated points can be the basis for
measurement estimation considering the pre-defined random
noises. For instance, using the distance between one position
and the antennas, measurement like time of arrival can be
simulated.
The server-side software components are primarily imple-
mented in JavaScript in Node.js [58], with additional com-
ponents implemented in Python 3.6 and called as server-side
scripts by a Node.js extension. Psycopg [59] can be used to
connect to the database server and communicate with Python
scripts. This DB API 2.0 compliant PostgreSQL [60] database
adapter is designed for multi-threaded applications and main-
tains its own connection pool. It is mostly implemented in the
C language, thus being efficient. Communication between the
client and server is enabled via the Socket.IO [61] library. The
client-side application is implemented in HTML5 [62], and
uses the WebGL-based Three.js [54] rendering framework to
facilitate real-time 3D viewing of the generated results.
V. IMPLEMENTATION AND RESULTS
Fig. 6. The as-is BIM of the university building area used in the case study.
We present experimental results obtained from each of the
key SOS component implementations, using mostly simulated
data related to a functional office floor space within a modern
university building (Fig. 6). We present and discuss processing
of point cloud representations of this environment (generated
synthetically from the as-is BIM), and the resulting floor
plan that is generated automatically from the point cloud
representation using the initial version of the Floor Plan
Generation software component. Furthermore, we discuss the
use of simulated 5G signals for localization estimation, and
provide examples of generating a routing graph from the
approximated 2D floor plan as well as from the as-is IFC LOD
300 model of the entire floor area. We use these routing graphs
for the computation of an optimal path for aiding in indoor
navigation tasks. Finally, we present the initial visualization
results running on a Web3D-enabled browser on a commodity
client computer.
A. Point Cloud Processing
For the point cloud processing results, the point cloud of
the fourth floor area of the university building is generated
using simulated laser scanning. We make use of the as-is LOD
300 IFC model, from which we extract a triangulated mesh,
and sample points using uniformly distributed point sampling
for each of the mesh triangles [63]. Once generated, further
processing is performed via the Point Cloud Processing soft-
ware component, using the Open3D Python framework [64]
for key processing tasks (Fig. 7). These key tasks include: sub-
sampling, outlier point removal and geometric segmentation.
The result of these processes are point clusters representing
key structural elements which can be used as floor plan layers.
Registration was not performed for this case study, though
the implementation is capable of accomplishing this using
automated registration methods, e.g., Iterative-Closest Point
(ICP) matching algorithm [65].
After sub-sampling, using a voxelized sampling to ensure
uniform removal of points, the original point cloud is reduced
to a fraction of its original points (approx. 75% of points are
removed). The overall visual fidelity of the point cloud is pre-
served, while increasing the coarsity and removing redundant
overlapping and closely spaced points. For the outlier point
removal, a statistical outlier removal method was used [66],
where the points forming clusters that are considered too far
away from the main point cluster groups were removed. This
enabled the removal of outlier points especially noticeable at
the edges of the sub-sampled point cloud. Finally, the point
clusters representing the planes of key structural components
(namely walls, floors and ceilings) are detected and segmented
using the iterative RANSAC method, which attempts to fit
matching points to a set number of hyperplanes [67]. The
horizontal point clusters forming the floor representation are
then used for the 2D floor plan approximation by the Floor
Plan Generation component.
Fig. 7. Initially segmented point cloud, with the main floor plan boundary
and secondary wall structure boundaries displayed.
B. Floor Plan Generation
A 2D floor plan can be generated either from the existing
as-is BIM (i.e., IFC model), or from the as-is point cloud
representation. Generating the 2D floor plan from the IFC
requires selecting each building storey element and projecting
it onto a 2D plane, where each of the IFC components are
rendered as vectorized paths and associated element semantics,
and exported together as a SVG file [68] . This SVG file is then
further parsed and converted to a GeoJSON [69] file (Fig. 8),
which can then be used as 2D floor plan data for required
indoor navigation tasks.
Fig. 8. An example of a semantically-rich GeoJSON file, containing vector-
ized paths along with IFC element shapes and semantics (e.g., a wall element).
A 2D floor plan can also be approximated from a hori-
zontally sliced layer of the post-processed point cloud. The
initial 2D concave or convex hulls of this point cloud layer
(including both primary and secondary shape boundaries, i.e.,
shapes within shapes), are approximated based on the approach
described in Stojanovic et al. (2019) [24]. This allows for
the capture of boundaries of the floor plan, using adjustable
parameter values (Fig. 9). Once the initial boundaries are
approximated, they are simplified as their resulting boundary
shape is usually noisy. The resulting 2D floor plan approxi-
mation is then exported as a GeoJSON file.
Fig. 9. A generated floor plan based on a point cloud, with boundary
evaluation errors that need to be corrected in further post-processing steps.
C. 5G Simulation and Localization
Using the Simulated 5G Capture components, the antenna
placement and three 5G positioning signals are simulated
(Fig. 10). This is calculated based on some reference points
and given noise. In this way, one may simulate 5G-based
coordinates with different precision. This is needed for fu-
sion algorithms when such a simulated infrastructure-based
positioning has a high range of noise. One can enter the
number of desired antennas and positions. Then, they can
be entered to the simulated environment and by calling a
calculation function, the predefined noises, frequencies and
some measurement results can be computed.
D. Optimal Path Approximation
The optimal path generation is performed using the gen-
erated routing graphs either from the 2D floor plan or the
3D floor plan derived from the as-is IFC representation. For
routing graph generation of 2D floor plans, we make use of
a generalized Voronoi-based medial axis transform [50], in
order to generate 2D line graphs representing navigable areas
(Fig. 11(a)), while for routing graph generation from the 3D
floor plan we use the triangulated mesh of the floor element
boundary representation in order to generate a “navigation
mesh” [10] (Fig. 11(b)).
For computation of the shortest paths using routing graphs,
we make use of the A* shortest path algorithm [70]. The A*
Fig. 10. An example of localization of three points (red circles) in the simu-
lated environment, with six different 5G antenna placements. This simulation
component is tested via a simple user interface that allows the placement of
5G antennas anywhere on the vectorized GeoJSON version of the 2D floor
plan (generated from the as-is BIM representation).
(a)
(b)
Fig. 11. (a) An example routing graph of a navigable floor plan area variant,
based on the approximated primary boundary of the 2D floor plan. (b) An
example routing graph/navigation mesh (purple) of the main navigable areas
of the floor plan, based on the triangulated 3D model of the as-is BIM
representation.
algorithm uses the vertex and edge connections that form the
routing graph, in order to evaluate and construct the shortest
navigation path. This approach was tested using the 3D model
of an as-is BIM (Fig. 6). For testing the 3D floor plan and
routing graph, the generated shortest path computation allows
the user to select and set starting and ending points, between
which the shortest path will be computed (taking into account
any obstacles, e.g., walls that may be in between the starting
and ending points (Fig. 12)).
VI. DISCUSSION
We have presented a conceptual SOA and prototypical
implementation of key SOS components for integrating 5G,
Fig. 12. An example of shortest path (blue) computation result between the
starting and current point (green) and ending point (red) – based on the result
of the A* algorithm and making use of the underlying routing graph (i.e.,
navigation mesh).
BIM and point cloud data for DT representations of indoor
built environments. For our case study, we have focused on
the topic of indoor navigation, using an approach of combining
point clouds, 2D floor plans, BIM and simulated 5G signal
data. We can create 2D floor plan approximations from input
point clouds, and derive and generate routing graphs from both
2D floor plans and 3D models (obtained from BIM data).
Using the 5G-based simulated coordinates, we are able to
simulate the expected localization approximation correspond-
ing to the indoor areas within the university building - where
the expected 5G infrastructure will be operational in the near
future. In this way, one can approximate the user’s indoor
position. We further combine this with optimal path generation
between the user’s current location and target destination. The
generated routing graphs are used with search algorithms to
calculate the shortest path and visualize it within a 2D/3D
floor plan context. The fusion of the outputs of all of these
results from the SOS components is presented interactively via
a Web3D interface, where the user can inspect the as-is point
cloud, 2D floor plan, 3D BIM model, current approximated
location and optimal path results. Our presented approach is
based on the DT paradigm, which has evolved from previous
research mainly focusing on BIM-based and IoT integration
approaches [71] - though such approaches do not take into ac-
count, e.g., dynamically updated systems, frequently changing
built environment states and indoor space configurations, and
are not designed to predict future states. Apart from a Web3D-
based approach for interactive visualization of the results
obtained using service-oriented computation, another approach
can be the use of existing game engines for computation
and presentation for FM scenarios [72]. The use of the SOS
paradigm is also not the only way to implement DT solutions,
and approaches based on, e.g., Supervisory Control and Data
Acquisition (SCADA) [73] and Enterprise Resource Planning
(ERP) [74] have also been investigated as potential architecture
solutions in previous research.
VII. CONCLUSION
We have proposed our approach based on a concept of a
DT, which enables the fusion of historical, current and pre-
dicted (or simulated) data sources for processing, analysis and
representation – within a service-oriented paradigm. We have
implemented and tested all of the key software components
responsible for processing, analysis and visualization of key
data sources needed to generate valuable information about
localization and indoor navigation within a university campus
building. We have identified the use of point clouds as a
key source for as-is representation of indoor environments,
alongside existing as-is/as-built BIM data (e.g., IFC models,
point clouds, etc.). We have presented and discussed methods
for generation of approximated 2D floor plans from point
clouds, and the generation of routing graphs from such floor
plans as well as 3D mesh representations derived from existing
IFC models. Finally, we have implemented and described
state-of-the art approaches for routing graph and optimal path
computation. We have designed a simulation for 5G-based
coordinate estimation to overcome with the localization task.
The presented approach lays a solid foundation for future work
focusing on developing a versatile indoor navigation software
solution based on the DT and service-oriented computing
paradigms.
The use of simulation processes for generation of point
cloud and 5G sensor and signal data has enabled the rapid
development and testing of the key software components –
without needing to source such data from complex or non-
existent sources. For future work, we plan to evaluate our
approach using real-world point clouds captured using laser
scanning as well as make use of the planned 5G infrastructure
for capturing real signal and sensor data. The positioning
approaches can achieve the fusion of sensor data, infrastructure
input and consider the map matching features. While we make
use of a DBMS for storage for data, we did not explicitly
evaluate its performance for data retrieval and queries, and
this is also planned for future work. We also plan to further
optimize and improve the 2D floor plan generation methods, in
order to have better approximations of secondary boundaries.
Finally, our future work will focus the further development
and integration of DT and 5G paradigms focusing FM related
scenarios (e.g., real-time data can be fed into a DT with the
help of 5G and serve to actively control the flow of people
and the operational capacity of the building).
ACK NOW LE DG ME NT S
This research and the L5IN project was funded by the Fed-
eral Ministry of Transport and Digital Infrastructure (BMVI),
grant number VB5GFHAMB.
REFERENCES
[1] K. Kensek, “BIM guidelines inform facilities management databases: a
case study over time,” Buildings, vol. 5, no. 3, pp. 899–916, 2015.
[2] P. Teicholz et al.,BIM for facility managers. John Wiley & Sons, 2013.
[3] C. Schuldt, H. Shoushtari, N. Hellweg, and H. Sternberg, “L5in:
Overview of an indoor navigation pilot project,” Remote Sensing, vol. 13,
no. 4, p. 624, 2021.
[4] M. Y. L. Chew, E. A. L. Teo, K. W. Shah, V. Kumar, and G. F. Hussein,
“Evaluating the roadmap of 5G technology implementation for smart
building and facilities management in singapore,” Sustainability, vol. 12,
no. 24, p. 10259, 2020.
[5] R. Alonso, M. Borras, R. H. Koppelaar, A. Lodigiani, E. Loscos, and
E. Y¨
ontem, “SPHERE: BIM digital twin platform,” in Multidisciplinary
Digital Publishing Institute Proceedings, vol. 20, no. 1, 2019, p. 9.
[6] M. Grieves and J. Vickers, “Digital twin: Mitigating unpredictable, un-
desirable emergent behavior in complex systems,” in Transdisciplinary
perspectives on complex systems. Springer, 2017, pp. 85–113.
[7] J. Posada, M. Zorrilla, A. Dominguez, B. Simoes, P. Eisert, D. Stricker,
J. Rambach, J. D¨
ollner, and M. Guevara, “Graphics and media tech-
nologies for operators in industry 4.0,” IEEE computer graphics and
applications, vol. 38, no. 5, pp. 119–132, 2018.
[8] Q. Lu, A. K. Parlikad, P. Woodall, G. Don Ranasinghe, X. Xie, Z. Liang,
E. Konstantinou, J. Heaton, and J. Schooling, “Developing a digital
twin at building and city levels: case study of west cambridge campus,”
Journal of Management in Engineering, vol. 36, no. 3, p. 05020004,
2020.
[9] L. Yang and M. Worboys, “Generation of navigation graphs for indoor
space,” International Journal of Geographical Information Science,
vol. 29, no. 10, pp. 1737–1756, 2015.
[10] S. Golodetz, “Automatic navigation mesh generation in configuration
space,” Overload Journal, pp. 22–27, 2013.
[11] D. E. Knuth, “A generalization of dijkstra’s algorithm,” Information
Processing Letters, vol. 6, no. 1, pp. 1–5, 1977.
[12] IFC4.3 Specification, buildingSMART, 2021, 4.3 RC2 - Release Candi-
date 2.
[13] L. Liu, B. Li, S. Zlatanova, and P. van Oosterom, “Indoor navigation
supported by the industry foundation classes (ifc): A survey,” Automation
in Construction, vol. 121, p. 103436, 2021.
[14] A. A. Diakit´
e and S. Zlatanova, “Spatial subdivision of complex in-
door environments for 3D indoor navigation,” International Journal of
Geographical Information Science, vol. 32, no. 2, pp. 213–235, 2018.
[15] M. Fu, R. Liu, B. Qi, and R. R. Issa, “Generating straight skeleton-
based navigation networks with industry foundation classes for indoor
way-finding,” Automation in Construction, vol. 112, p. 103057, 2020.
[16] Q. Wang and M.-K. Kim, “Applications of 3D point cloud data in
the construction industry: A fifteen-year review from 2004 to 2018,”
Advanced Engineering Informatics, vol. 39, pp. 306–319, 2019.
[17] W. Moussa, “Integration of digital photogrammetry and terrestrial laser
scanning for cultural heritage data recording,” Ph.D. dissertation, Uni-
versity of Stuttgart, 2014.
[18] E. Angelats, M. Par´
es, and P. Kumar, “Feasibility of smartphone
based photogrammetric point clouds for the generation of accessibility
maps,” International Archives of the Photogrammetry, Remote Sensing
& Spatial Information Sciences, vol. 42, no. 2, pp. 35–41, 2018.
[19] R. Staiger, “Terrestrial laser scanning technology, systems and applica-
tions,” in 2nd FIG Regional Conference Marrakech, Morocco, vol. 1,
2003, pp. 1–10.
[20] V. Stojanovic, R. Richter, J. D ¨
ollner, and M. Trapp, “Comparative vi-
sualization of BIM geometry and corresponding point clouds,” Building
Information Systems in the Construction Industry, p. 13, 2018.
[21] J. W. Ma, T. Czerniawski, and F. Leite, “Semantic segmentation of
point clouds of building interiors with deep learning: Augmenting
training datasets with synthetic BIM-based point clouds,” Automation
in Construction, vol. 113, p. 103144, 2020.
[22] F. Engelmann, T. Kontogianni, A. Hermans, and B. Leibe, “Exploring
spatial context for 3D semantic segmentation of point clouds,” in
Proceedings of the IEEE International Conference on Computer Vision
Workshops, 2017, pp. 716–724.
[23] P. Tang, D. Huber, B. Akinci, R. Lipman, and A. Lytle, “Automatic
reconstruction of as-built building information models from laser-
scanned point clouds: A review of related techniques,” Automation in
construction, vol. 19, no. 7, pp. 829–843, 2010.
[24] V. Stojanovic, M. Trapp, R. Richter, and J. D¨
ollner, “Generation of ap-
proximate 2D and 3D floor plans from 3D point clouds.” in VISIGRAPP
(1: GRAPP), 2019, pp. 177–184.
[25] J. D ¨
ollner, “Geospatial artificial intelligence: Potentials of machine
learning for 3D point clouds and geospatial digital twins,” PFG–
Journal of Photogrammetry, Remote Sensing and Geoinformation Sci-
ence, vol. 88, no. 1, pp. 15–24, 2020.
[26] J.-g. Park, “Indoor localization using place and motion signatures,” Ph.D.
dissertation, Massachusetts Institute of Technology, 2013.
[27] H. Shoushtari, T. Willemsen, and H. Sternberg, “Many ways lead to
the goal—possibilities of autonomous and infrastructure-based indoor
positioning,” Electronics, vol. 10, no. 4, p. 397, 2021.
[28] F. Li, C. Zhao, G. Ding, J. Gong, C. Liu, and F. Zhao, “A reliable
and accurate indoor localization method using phone inertial sensors,”
in Proceedings of the 2012 ACM conference on ubiquitous computing,
2012, pp. 421–430.
[29] T. Willemsen, “Fusionsalgorithmus zur autonomen positionssch¨
atzung
im geb¨
aude, basierend auf mems-inertialsensoren im smartphone,” Ph.D.
dissertation, HafenCity Universit¨
at Hamburg, 2016.
[30] C. Chen, P. Zhao, C. X. Lu, W. Wang, A. Markham, and N. Trigoni,
“Deep-learning-based pedestrian inertial navigation: Methods, data set,
and on-device inference,” IEEE Internet of Things Journal, vol. 7, no. 5,
pp. 4431–4441, 2020.
[31] S. Herath, H. Yan, and Y. Furukawa, “Ronin: Robust neural inertial
navigation in the wild: Benchmark, evaluations, & new methods,”
in 2020 IEEE International Conference on Robotics and Automation
(ICRA). IEEE, 2020, pp. 3146–3152.
[32] J. Liu, J. Luo, J. Hou, D. Wen, G. Feng, and X. Zhang, “A BIM based
hybrid 3D indoor map model for indoor positioning and navigation,”
ISPRS International Journal of Geo-Information, vol. 9, no. 12, p. 747,
2020.
[33] M. Deng, C. C. Menassa, and V. R. Kamat, “From BIM to digital twins:
a systematic review of the evolution of intelligent building represen-
tations in the aec-fm industry,” Journal of Information Technology in
Construction (ITcon), vol. 26, no. 5, pp. 58–83, 2021.
[34] J. D¨
ollner and B. Hagedorn, “Integrating urban GIS, CAD, and BIM
data by servicebased virtual 3D city models,” Urban and regional data
management-annual, pp. 157–160, 2007.
[35] K. Roper and R. Payant, The facility management handbook. Amacom,
2014.
[36] V. Stojanovic, M. Trapp, B. Hagedorn, J. Klimke, R. Richter, and
J. D¨
ollner, “Sensor data visualization for indoor point clouds.” Advances
in Cartography and GIScience of the ICA, vol. 2, pp. 1–8, 2019.
[37] J. D¨
ollner, B. Hagedorn, and J. Klimke, “Server-based rendering of large
3D scenes for mobile devices using G-buffer cube maps,” in Proceedings
of the 17th International Conference on 3D Web Technology, 2012, pp.
97–100.
[38] V. Stojanovic, M. Trapp, R. Richter, B. Hagedorn, and J. D¨
ollner,
“Towards the generation of digital twins for facility management based
on 3D point clouds,” in Proceeding of the 34th Annual ARCOM
Conference, vol. 2018, 2018, pp. 270–279.
[39] M. Borth, J. Verriet, and G. Muller, “Digital twin strategies for sos 4
challenges and 4 architecture setups for digital twins of sos,” in 2019
14th Annual Conference System of Systems Engineering (SoSE). IEEE,
2019, pp. 164–169.
[40] T. Birdal and S. Ilic, “A point sampling algorithm for 3D matching of
irregular geometries,” in 2017 IEEE/RSJ International Conference on
Intelligent Robots and Systems (IROS). IEEE, 2017, pp. 6871–6878.
[41] R. Pierdicca, M. Mameli, E. S. Malinverni, M. Paolanti, and E. Frontoni,
“Automatic generation of point cloud synthetic dataset for historical
building representation,” in International Conference on Augmented
Reality, Virtual Reality and Computer Graphics. Springer, 2019, pp.
203–219.
[42] V. Stojanovic, M. Trapp, R. Richter, and J. D¨
ollner, “A service-oriented
indoor point cloud processing pipeline,” The International Archives of
Photogrammetry, Remote Sensing and Spatial Information Sciences,
vol. 42, pp. 339–346, 2019.
[43] LAS Specification, https://www.ogc.org/standards/LAS, OGC, 2018, ver-
sion 1.0.
[44] R. Richter and J. D¨
ollner, “Out-of-core real-time visualization of massive
3D point clouds,” in Proceedings of the 7th International Conference
on Computer Graphics, Virtual Reality, Visualisation and Interaction in
Africa, 2010, pp. 121–128.
[45] P. van Oosterom, O. Martinez-Rubi, M. Ivanova, M. Horhammer,
D. Geringer, S. Ravada, T. Tijssen, M. Kodde, and R. Gonc¸alves,
“Massive point cloud data management: Design, implementation and
execution of a point cloud benchmark,” Computers & Graphics, vol. 49,
pp. 92–125, 2015.
[46] S. Agarwal and K. Rajan, “Analyzing the performance of nosql vs. sql
databases for spatial and aggregate queries,” in Free and Open Source
Software for Geospatial (FOSS4G) Conference Proceedings, vol. 17,
no. 1, 2017, p. 4.
[47] Y. Xie, J. Tian, and X. X. Zhu, “Linking points with labels in 3D: A
review of point cloud semantic segmentation,” IEEE Geoscience and
Remote Sensing Magazine, vol. 8, no. 4, pp. 38–59, 2020.
[48] M. Ester, H.-P. Kriegel, J. Sander, X. Xu et al., “A density-based
algorithm for discovering clusters in large spatial databases with noise.”
in Kdd, vol. 96, no. 34, 1996, pp. 226–231.
[49] E. Che, J. Jung, and M. J. Olsen, “Object recognition, segmentation,
and classification of mobile laser scanning point clouds: A state of the
art review,” Sensors, vol. 19, no. 4, p. 810, 2019.
[50] W. V. Toll, A. F. C. Iv, M. J. V. Kreveld, and R. Geraerts, “The medial
axis of a multi-layered environment and its application as a navigation
mesh,” ACM Transactions on Spatial Algorithms and Systems (TSAS),
vol. 4, no. 1, pp. 1–34, 2018.
[51] U. Isikdag, “BIM and IoT: A synopsis from GIS perspective,” The
International Archives of Photogrammetry, Remote Sensing and Spatial
Information Sciences, vol. 40, p. 33, 2015.
[52] W.-L. Lee, M.-H. Tsai, C.-H. Yang, J.-R. Juang, and J.-Y. Su, “V3DM+:
BIM interactive collaboration system for facility management,” Visual-
ization in Engineering, vol. 4, no. 1, pp. 1–15, 2016.
[53] Unity Game Engine, https://unity.com/, Unity Technologies, 2021.
[54] R. Cabello et al.,Three.js, https://threejs.org/, 2021.
[55] R. B. Haber and D. A. McNabb, “Visualization idioms: A conceptual
model for scientific visualization systems,” Visualization in scientific
computing, vol. 74, pp. 74–93, 1990.
[56] PostGIS, https://postgis.net/, PostGIS Project Steering Committee, 2021,
version 3.1.1.
[57] System Architecture for the 5G System, 3GPP, 2021, document TS 23.501
V16.8.0.
[58] Node.js, https://nodejs.org/en/, OpenJS Foundation, 2021.
[59] D. Varrazzo et al.,Psycopg, https://www.psycopg.org/, 2021.
[60] PostgreSQL, https://www.postgresql.org/, PostgreSQL Global Develop-
ment Group, 2021.
[61] G. Rauch et al.,Socket.IO, https://socket.io/, 2021.
[62] HTML 5.2 Specification, https://www.w3.org/TR/html52/, W3C, 2017,
version 5.2.
[63] E. W. Weisstein, Triangle point picking, https://mathworld.wolfram.
com/, Wolfram Research, Inc., 1999.
[64] Q.-Y. Zhou, J. Park, and V. Koltun, “Open3D: A modern library for 3D
data processing,” arXiv:1801.09847, 2018.
[65] Y. Chen and G. Medioni, “Object modelling by registration of multiple
range images,” Image and vision computing, vol. 10, no. 3, pp. 145–155,
1992.
[66] B. Skinner, T. Vidal-Calleja, J. V. Miro, F. De Bruijn, and R. Falque,
“3D point cloud upsampling for accurate reconstruction of dense 2.5D
thickness maps,” in ACRA, 2014.
[67] M. A. Fischler and R. C. Bolles, “Random sample consensus: a paradigm
for model fitting with applications to image analysis and automated
cartography,” Communications of the ACM, vol. 24, no. 6, pp. 381–395,
1981.
[68] SVG Specfication, https://www.w3.org/TR/SVG11/, W3C, 2011, version
1.1.
[69] GeoJSON Format Specfication, https://tools.ietf.org/html/rfc7946, Geo-
graphic JSON working group, 2016.
[70] R. Dechter and J. Pearl, “Generalized best-first search strategies and
the optimality of a,” Journal of the ACM (JACM), vol. 32, no. 3, pp.
505–536, 1985.
[71] A. Khan and K. Hornbæk, “Big data from the built environment,” in
Proceedings of the 2nd international workshop on Research in the large,
2011, pp. 29–32.
[72] M. U. Khalid, M. K. Bashir, and D. Newport, “Development of a
building information modelling BIM-based real-time data integration
system using a building management system (BMS),” in Building
Information Modelling, Building Performance, Design and Smart Con-
struction. Springer, 2017, pp. 93–104.
[73] A. Jain, D. Nong, T. X. Nghiem, and R. Mangharam, “Digital twins for
efficient modeling and control of buildings: An integrated solution with
scada systems,” in 2018 Building Performance Analysis Conference and
SimBuild, 2018.
[74] B. Tezel, Z. Aziz et al., “From conventional to it based visual man-
agement: a conceptual discussion for lean construction,” Journal of
information technology in construction, vol. 22, pp. 220–246, 2017.
