InterOpT: A new testing platform based on oneM2M standards for IoT systems

Full text

InterOpT: A new testing platform based on
oneM2M standards for IoT systems
Senay Tuna Demirel, Mehmet Demirel, Ibrahim Dogru
NETAS Telecommunication A.S.
Kurtkoy, Istanbul, Turkey
stuna,mdemirel,idogru@netas.com.tr
Resul Das
Firat University, Faculty of Technology
Department of Software Engineering, 23119, Elazig, Turkey
rdas@firat.edu.tr
Abstract—Nowadays, with the developing technologies, IoT
based solutions are increasing day by day. In the scope of
this study, a test platform has been developed to test the IoT
solutions developed by different companies according to oneM2M
standards. The main objective of the study is to provide a national
standardization for the control of IoT solutions. The developed
InterOpT (InterOpability Tool) testing platform software tool
is the automation system for controlling interoperability and
compliance with standards on IoT network components. Based
on the OneM2M IoT standard recognized by ETSI, this system
performs the validity tests defined by the RFC and analyzes and
automates the results. Provides interoperability and system audit
reports on IoT structures. Improved testing platform, web and
cloud-based service can offer. It can do traffic and stress tests
especially for the system and it can also do log collection analysis
automatically. The platform can support different embedded
IoT protocols such as HTTP, CoAP, MQTT. As a result, with
this project, we have developed a successful test automation
platform that can perform interoperability and compliance tests
by adhering to IoT protocols and OneM2M standards.
Index Terms—IoT test platform, M2M, OneM2M, ETSI, CoAP,
MQTT.
I. INTRODUCTION
The number of devices connected to the Internet in the
world reaches twice the human population. According to a
report released by Cisco, the IoT global market share is ex-
pected to be $ 7.1 trillion in 2020, while the number of devices
or objects connected to the Internet will reach 26 billion [1],
[2]. Household appliances, electronic devices and physical
devices all have the ability to detect the situation and events
around them with the help of sensors [3], [4]. These devices
can communicate with each other and with people who use
them [5]. The purpose of these devices in IoT environments
is to communicate with other IoT devices using the relevant
network communication protocols and to transmit information
by connecting to applications. The IoT platform fills the gap
between the sensors of the devices and the data networks [5],
[6]. The IoT platform connects the data network to the sensor
assembly, providing services that are useful to the needs of the
community by using the supportive applications to understand
the data flow from hundreds of sensors [7]. However, as the
number of IoT devices and applications connected to the
network increase, more and more communication protocols
are involved. The successful operation of this platform also
978-1-7281-1244-2/19/$31.00 ©2019 IEEE
requires rapid data flow by supporting the increase in network
traffic. These issues also reveal performance challenges. It is
also the challenge of securing end-to-end security of valuable
data, which should remain open to users. In addition, the IoT
platform architecture needs to be scalable so that the system
can continue to function without interruption, especially when
the number of devices, applications and protocols is designed.
In the developed system, functional validation, system verifi-
cation, traffic and stress testing requirements, finding deficits
defined on RFCs [8], [9] automating the system and providing
analysis of the results are critical functions. In addition to the
local test environment, it is a platform with a wide range of
applications that can be tested remotely over the cloud. In the
second section of this study, OneM2M standards [10] which
are considered and used in the developed test platform are
examined. In the third section, the developed IoT test software
platform and its architecture are explained in detail. In the
fourth section, the flow algorithm of the test run in the IoT
test platform and the oneM2M validity test are presented. In
the last section, the conclusion is presented.
II. ON EM2M S TANDARDS USED IN INTEROPT P ROJ EC T
OneM2M is the global standards initiative for Machine to
Machine Communications and the Internet of Things. One of
the most important goal of the OneM2M standard is to enable
M2M and IoT technology product developments, project work
and application solutions to be spoken in a common language.
The OneM2M standard defines the software service to be
as independent as possible from the application and network
layer and thus to provide diversity with the same software
service infrastructure. The OneM2M standard is the most com-
prehensive standard for IoT. Along with this standardization,
different M2M applications provide the opportunity to develop
new services that can use each other independently from the
infrastructure network. In this case, the M2M gateway or core
devices require that the compliance criteria of the standard be
included in the system after detection and verification [10].
NETA ¸S, which is a member of European Telecommu-
nication Standards Institute (ETSI), actively participates in
oneM2M working groups. Within the scope of the developed
project, firstly the IoT business models were examined and
the studies conducted in other countries were investigated.
OneM2M, 8 major standards institute (ARIB [Japan], ATIS

scenerio
Mailbox
(Flattening)
(Tag-Value Pair)
Validation of protocol parameters for incoming msgs(XML/JSON)
COAP to/from OneM2M Binding(server) HTTP to/from OneM2M Binding(server)
COAP(FreeCoap) HTTP(http_parser, libebb)
Test
Automation
Tool
Database
PostgreSQL
File
System
DB
wapper
Subscription &
Notification
Service Charging &
Accounting Group Management Security
CMDH Location Network Service
Triggering, mcn Discovery
Date Managment &
Repository Registration Device Managment Aplication Service
Layer Managment
Fig. 1. OneM2M compliant modules covered by the InterOpt test tool
[North America], CCSA [China], ETSI [Europe], TIA [North
America], TSDSI [India], TTA [Korea], TTC [Japan]) support
and has reached a position that is the standard of standards
[11], [12], [13], [14]. Firstly; 6LowPAN, ZigBee Wi-Fi focuses
on verifying the Bluetooth interfaces [15]. In the first place,
verification of CoAP and HTTP client-server system was
supported. Since narrowband radio systems have also begun to
use IoT standards, the developed and recommended InterOpt
test platform is becoming more and more important. Within
the scope of TUBITAK-TEYDEB project, an IoT test platform
software tool has been developed which has an efficient, useful
and high-performance structure. In this process, technical stan-
dards, specifications and academic publications were examined
in detail and scientific studies were used. As a result, the
technical specifications (RFC) have been transformed into
understandable requirements and feasibility studies have been
made. In this context, the standards / specifications given
below are examined in detail for InterOpt project [8], [10].
•oneM2M TS-001: "Functional Architecture"
•oneM2M TS-002: "Requirements"
•oneM2M TS-011: "Common Terminology"
•oneM2M TS-009: "HTTP Protocol Binding"
•oneM2M TS-010: "MQTT Protocol Binding"
•oneM2M TS-001: "Testing Framework"
•oneM2M TS-001: "OIC Interworking"
As shown in Figure 1, the above-mentioned standards [10]
and oneM2M compliant modules were considered in the test
platform developed. In IoT system solutions, necessary tests
are performed for performance, speed and system resources.
III. A NE W TE ST AU TOMATI ON PLATF OR M AN D IT S
CO MP ON EN TS FOR IOTSYSTEMS
With the determination of the IoT and communication stan-
dards, the IoT market is also accelerating in our country. With
this study, it is aimed to develop IoT test software products
that are able to compete with foreign alternatives in accordance
with industry standards, to contribute to the economy and to
decrease foreign dependency. With the InterOpt project, a web-
based IoT testing software automation system that provides
interoperability and standards compliance control in network
components is being developed. The compatibility with other
OneM2M projects in the world market is also tested. In the test
automation system developed in the JAVA programming soft-
ware platform, embedded IoT protocols such as HTTP, CoAP
and MQTT compatible with OneM2M standards are used to
transmit messages. OneM2M messages can be transmitted in
different formats such as XML and JSON.
Within the scope of the InterOpt project, it is possible to
make individual and project-specific adjustments to provide
a flexible structure to users. Besides, it is aimed to perform
functional verification (FV), system verification (SV), traffic,
stress and performance tests according to the suitability of the
system. Functional and system verification, traffic and stress
testing requirements, detection of defined openings on RFCs,
automaticisation of the system and analysis of results are the
critical functions considered in this test platform.
With the developed product, there is the possibility of
remote testing on both the local test environment and the
cloud. The project is also an automation system that provides
interoperability and compliance with standards in IoT network
components. As an IoT test platform, it will provide economic
and national gains with pioneering and domestic solutions in

OneM2M/IoT
Standarts
XML
Library
Test Tool
ReportingSystem
Analysis System
Scenario
Library
GitHub
Request
IoT Systems
Response
Test Suit
Fig. 2. The main components of the developed IoT test automation platform
Algorithm 1 The test running algorithm of IoT test automation platform
Initialize:
1) The XML file is sent to the desired destination server via the corresponding protocol to be sent by the testing tool.
2) The XML file is converted to a meaningful oneM2M message before it exits the test tool.
3) When the oneM2M message is received by the server, it is interpreted here to see if it is a valid message.
a) If the message is valid, go to step 4.
b) If the message is invalid, the server will return the appropriate error message. Go to step 8.
4) The server sends the message to the corresponding module.
5) The module performs the related operation by connecting to the database.
6) The message generated as a result of the transaction in the database is sent by the module to the sender module on
the server.
7) The server sends the response message to the test tool.
8) The test tool compares the value of the response code received in the reply message with the value it expects.
a) If the expected and received value is equal, the test is recorded as successful.
b) If the expected and received value is not equal, the test is recorded as unsuccessful.
9) Stop.
our country. In the test platform developed, there is compati-
bility and automation capability with automation systems like
Jenkins. However, logging of test records and log information
of operations, data analysis, visual graphs and statistical results
are also presented. The most innovative aspect of the project
is that it is the first product to provide standardization of
national IoT projects in our country. This product will be
used both in development time and after producing, supporting
next generation IoT standards (OneM2M), different embedded
systems and protocols (HTTP, COAP, MQTT). The platform
will provide service over the cloud, and will support the
compatibility with different automation systems. The most
important innovations of the project are inter-system com-
munication, traffic, co-analysis, reporting and automated log
collection. The statistical information to be obtained can be
reported to the authorized persons / organizations via the
cloud.
As shown in Figure 2, the developed IoT test platform
consists of 5 main components. Within the scope of the project
these components are named as follows.
1) IoT test engine module
2) Statistics and package analysis system module
3) Management console and reporting module
4) IoT interoperability test packages module
5) Protocol scope and extended functions module
Developed applications and systems; standard protocols

TABLE I
COM PARI SON O F DE VEL OPE D TE ST PL ATFO RM W ITH P OTE NTI AL S IMI LA R PRO DUC TS IN T HE M ARK ET
(CoAP, HTTP), verification of received message, operation of
common service items, secure protocol support (TLS / DTLS),
otorization, system decommissioning and retransmission, traf-
fic control mechanism, database connection, bypassing mes-
sage between many nodes should be able to make extensive
validation for situations. The basic features of the InterOpt
test platform based on these specifications can be explained
as follows. Also, Table I shows the similar IoT testing tools,
such as The Grinder [16], Gatling [17], Tsung [18] and Jmeter
[19], in terms of their specific characteristics with the InterOpT
test tool developed by NETA¸S.
•Support for OneM2M standards adopted by standard
institutions or organizations such as ETSI on IoT
•The platform can be used in the development of IoT
systems and testing of products,
•Easy integration between different M2M solutions,
•Scalable structure for testing of different IoT products
(gateways, end devices, central management platforms
etc.).
•Possibility to provide services in the cloud or local
networks, and also to be used on products for detailed
analysis.
•To be able to perform traffic tests at the limit limits of
the product.
•Supports existing IoT protocols as well as secure proto-
cols (TLS, DTLS, etc.).
•Support integration with automation systems and detailed
reporting features.
•The ability to modify the test library to allow different
standards to be tested.
•HTTP, CoAP and MQTT protocol support in all modules
of the test tool.
•The platform can be used both in the development phase
and in the product tests.
•Testing the limits of the product with traffic tests.
•Different format support like XML/JSON
IV. CRE ATING A TEST LIB RA RY AN D ON EM2M
VALIDATIO N TE ST
In the InterOpt project, the test library has been created
in accordance with OneM2M standards, including a variety of
variations, with reference to the published XSD files. Test files
are kept in the form of XML files to test the behavior of the
OneM2M modules in detail. XML files for each module are
kept in folders of that module. Also, thanks to a scenario file in
each folder, a meaningful test sequence is created when tests
for a module are run. Algorithm 1 presents the process steps
of the test run algorithm on the IoT test automation platform.
In the test validity of this project, the test messages sent
to the server are transmitted to the relevant modules. The
server runs the validity test first of all without any action.
In this way, if there is a faulty message, this message is
sent to the client without being forwarded to the subunits.
Thanks to this validity test in server software, both time and
transaction losses were prevented. If any faulty message was
transmitted to the relevant modules on the server, it would
be understood that the last module was faulty. In this case, it
would cause time and transaction losses. OneM2M compatible
systems work in RESTful architecture with IP based CoAP
and HTTP request-response architecture. OneM2M create, re-

Fig. 3. A part of the XML library of the InterOpT test tool
Fig. 4. A new test start screen on the IoT test platform
trieve, update, delete, notify request message types are created
in XML format and stored in the library. XML files are created
according to the XSD (XML Schema Definition) schema
specified by the OneM2M standard. Hence, mandatory and
optional parameters that must be present in request-response
messages and valid-invalid limit values of these parameters,
various types of messages such as false message type, different
message contents have been created and recorded in the XML
library. The prepared XML library contains different actions
and anticipated request-response streams in accordance with
different oneM2M end node types (such as AE-Application
EndPoint). A part of the XML library of the InterOpT test
tool is given in Figure 3. The starting screen used for testing
of an IoT product is shown in Figure 4, and the result screen
of the test during traffic is shown in Figure 5.
V. CONCLUSION
There is not any domestic company other than NETA¸S
which has produced or produced solutions by communicating
IoT and M2M solutions on a standard platform. Different
companies abroad, such as Eclipse, Korea Telekom, open their
products to other companies for testing purposes to see the
interoperability of oneM2M systems. However, these manual
tests are limited and there is a need for an automation system
that covers the complete standard compliance.
M2M gateway and IoT core projects designed by Turkish
engineers at NETA ¸S laboratories will be put on the market
for sale after they are verified with this system. Although
the developed platform has many advantages over similar
test tools developed worldwide, the scope will be expanded
with different test models to be added to the test library as

Fig. 5. Test result screen during traffic on IoT test platform
standards are developed and changed. According to the IOT
interoperability standards and protocols and oneM2M similar
test platforms capable compatibility test today, are beginning
to more recent developments. This shows the competitive side
of the project. The project will provide the applicability of
an international standardization model, will have a national
product and contribute to the software export of our country.
ACK NOW LE DG EM EN TS
This study was carried out within the framework of
TUBITAK-TEYDEB program by NETAS Telecommunication
Inc. within the framework of project number 3170163 called
"InterOpt: Interoperability and compliance control for stan-
dards of IoT network components". Assoc. Prof. Dr. Resul
Das worked as an academic consultant in the project. All
project employees are grateful to TUBITAK for their financial
support.
REFERENCES
[1] Cisco, “The Internet of Things Reference Model,” 2014.
[2] D. Evans, “How the Next Evolution of the Internet Is Changing
Everything,” 2011.
[3] J. Gubbi, R. Buyya, S. Marusic, and M. Palaniswami, “Internet of Things
(IoT): A vision, architectural elements, and future directions,” Future
Generation Computer Systems, vol. 29, pp. 1645–1660, Sept. 2013.
[4] P. Diogo, L. P. Reis, and N. V. Lopes, “Internet of Things: A system’s
architecture proposal,” in 2014 9th Iberian Conference on Information
Systems and Technologies (CISTI), (Barcelona, Spain), pp. 1–6, IEEE,
June 2014.
[5] M. Z. Gunduz and R. Das, “Nesnelerin interneti (IoT): Gelisimi,
bilesenleri ve uygulama alanlari,” Pamukkale University, Journal of
Engineering Sciences, vol. 24, no. 2, pp. 327–335, 2018.
[6] D. Miorandi, S. Sicari, F. De Pellegrini, and I. Chlamtac, “Internet of
things: Vision, applications and research challenges,” Ad Hoc Networks,
vol. 10, pp. 1497–1516, Sept. 2012.
[7] G. Cerullo, G. Mazzeo, G. Papale, B. Ragucci, and L. Sgaglione,
“Chapter 4 - IoT and Sensor Networks Security,” in Security and
Resilience in Intelligent Data-Centric Systems and Communication Net-
works (M. Ficco and F. Palmieri, eds.), Intelligent Data-Centric Systems,
pp. 77–101, Academic Press, 2018.
[8] S. Husain, A. Prasad, A. Kunz, A. Papageorgiou, and J. Song, “Recent
Trends in Standards Related to the Internet of Things and Machine-to-
Machine Communications,” Journal of information and communication
convergence engineering, vol. 12, pp. 228–236, Dec. 2014.
[9] I. Ishaq, D. Carels, G. K. Teklemariam, J. Hoebeke, F. V. d. Abeele,
E. D. Poorter, I. Moerman, and P. Demeester, “IETF Standardization in
the Field of the Internet of Things (IoT): A Survey,” Journal of Sensor
and Actuator Networks, vol. 2, pp. 235–287, Apr. 2013.
[10] oneM2M web platform, “oneM2m - Standards for M2m and the Internet
of Things,” Jan. 2018.
[11] J. Swetina, G. Lu, P. Jacobs, F. Ennesser, and J. Song, “Toward a
standardized common M2m service layer platform: Introduction to
oneM2m,” IEEE Wireless Communications, pp. 20–26, June 2014.
[12] P. Kess and H. Kropsu-Vehkaperä, “Standardization with IoT,” in
Proceedings of the MakeLearn and TIIM Joint International Confer-
ence, (Timisoara- Romania), pp. 1069–1076, Politehnica University of
Timisoara, Romania, May 2016.
[13] C. Gezer and E. Taskin, “An overview of oneM2m standard,” in 2016
24th Signal Processing and Communication Application Conference
(SIU), (Zonguldak, Turkey), pp. 1705–1708, IEEE, May 2016.
[14] O. Bello and S. Zeadally, “Toward efficient smartification of the Internet
of Things (IoT) services,” Future Generation Computer Systems, Oct.
2017.
[15] E. Borgia, “The Internet of Things vision: Key features, applications
and open issues,” Computer Communications, vol. 54, pp. 1–31, Dec.
2014.
[16] T. G. testing framework, “The Grinder, a Java Load Testing Framework,”
Dec. 2017.
[17] G. O.-S. L. Testing, “Gatling Open-Source Load Testing - Designed for
DevOps and Continuous Integration,” Dec. 2017.
[18] T. testing platform, “Tsung,” Dec. 2017.
[19] “Apache JMeter - Apache JMeter™,” Dec. 2017.