Available via license: CC BY-NC 4.0
Content may be subject to copyright.
International Journal of Electrical and Computer Engineering (IJECE)
Vol. 8, No. 5, October 2018, pp. 4023~4032
ISSN: 2088-8708, DOI: 10.11591/ijece.v8i5.pp4023-4032 4023
Journal homepage: http://iaescore.com/journals/index.php/IJECE
IT Service Management System Measurement using
ISO20000-1 and ISO15504-8: Developing a Solution-Mediated
Process Assessment Tool to Enable Transparent and SMS
Process Assessment
Imam Asrowardi1, Septafiansyah Dwi Putra2, Eko Subyantoro3, Norzaidi Haji Mohd Daud4
1,2,3Department of Informatics Management, Politeknik Negeri Lampung, Indonesia
4Arshad Ayub Graduate Business School, Universiti Teknologi MARA, Malaysia
Article Info
ABSTRACT
Article history:
Received Feb 19, 2018
Revised Apr 11, 2018
Accepted Apr 18, 2018
Information technology is about not only hardware, software, communication
infrastructure and communication infrastructure but also how to manage
services. Information technology plays an increasingly important role in
developing the structure and functions of public and private sectors.
Service measurement plays an important role in IT service management
(ITSM) that is one of the subfields of Services Computing science. ITSM is a
big part of service science, a science field that combines computer science,
operation research engineering, business strategy, management science, and
organizational theory. Performance measurement from each of IT services is
absolutely needed and is important in the continuous development of ITSM.
These research provide good technical knowledge about the measuring ITSM
with some requirements. In this paper we suggest the metrics in each service
processes enables organizations to predict a direction for active process
enhancement and to identify if the goal of process can achieve. This
objective process metrics based on ISO/IEC 15504-8 and PRM ISO/IEC
20000-4 refinement. The output of this research, in the form of metrics and
tools for any type organizational use.
Keyword:
Evaluation
IT-service
SLA
SMS
Tools
Copyright © 2018 Institute of Advanced Engineering and Science.
All rights reserved.
Corresponding Author:
Imam Asrowardi,
Informatics Management,
Politeknik Negeri Lampung, Indonesia.
Email: imam@polinela.ac.id
1. INTRODUCTION
Information technology is about not only hardware, software, communication infrastructure and
communication infrastructure but also how to manage services. Information technology plays an increasingly
important role in developing the structure and functions of public and private sectors. Almost billion of
money are being spent on the procurement of new technology with the aim to improve IT organization
(Service delivery) as well as the quality of services delivered to customers. Although IT service often brings
benefits to an organization, its implementation sometimes does fail [1].
Service measurement plays an important role in IT service management (ITSM) that is one of the
subfields of Services Computing science. ITSM is a big part of service science, a science field that combines
computer science, operation research engineering, business strategy, management science, and organizational
theory. Many IT service organizations consider the measurement of IT service management processes,
especially service processes, as a difficult task [1]. Difficulties are mainly due to the following four reasons:
1) IT organizations do not have a structured approach for measuring IT services and service management
processes, 2) tools used by service support teams do not enable effective measurement, 3) IT service
ISSN: 2088-8708
Int J Elec & Comp Eng, Vol. 8, No. 5, October 2018 : 4023 – 4032
4024
management standards and frameworks do not provide practical examples how to measure support processes,
and 4) there are too many options what to measure in service management [2]. IT service management
related about preparing, designing, implementing, developing, mainting IT services [2].
A lot of research was focusing on implementation and integration of IT service management
processes in organizations. However, these studies mainly focus on the business and process point of view
and technical perspective is missing. On the other hand, it is not mentioned explicitly on how to measure and
ensure that every steps could help to increase the efficiency and effectiveness of ITSM implementation.
Performance measurement from each of IT services is absolutely needed and is important in the continuous
development of ITSM. These research provide good technical knowledge about the measuring ITSM with
some requirements. This research will perform the collaboration and refinement related IT service, so that the
matrix produced can be more practical for the implementation.
The problem will solve in this research is about how to measure every steps in ITSM. The
measurement is to design a framework and model integrated to each of ITSM steps, from design to
deployment. The research problem represent in these questions:
1. What are the best practices or framework for measuring an ITSM process with an integration and many
progreess?
2. How metrics can be support the best practices or framework?
2. LITERATURE REVIEW
2.1. IT governance
IT governance recognized as an integral part of enterprise governance. It ―consists of the leadership
and organizational structures and processes that ensure that the organization‘s IT sustains and extends the
organization‘s strategies and objectives‖ [3]. IT Governance is a system that aims to control and direct IT
service activities given by the service provider. The system encompasses policy, objective, planning, process,
documentation and resources needed in a service cycle. The cycle consists of strategy, design, transition,
operation and continuous improvement [4]. ISO/IEC 20000-1 states that the components of service
management consist of [4]:
a. Strategy
1. Management responsibility.
2. Process governance by other parties.
3. Documentation management.
4. Resource management.
5. Service management system creation.
b. Design and transition
1. New service design or modification.
2. Design and development of a new or a modification of a service.
3. Transition of a new or a modification of a service.
c. Operation
1. Service delivery process: capacity management, service-level management, information security
management, service availability and continuity management, service report, service budgeting and
cost calculation.
2. Adjustment process: configuration management, change management, release and deployment
management.
3. Resolution process: problem management, incident and service request management.
d. Continuous service improvement
2.2. Service management system standard - ISO/IEC 20000:2011
ISO/IEC 20000:2009 is the first international standard for IT service management (ITSM). This
standard is based on and is intended to replace the British Standards BS 15000. This standard was first
published in December 2005 and like its predecessor. BS 15000, was originally developed to describe the
best practice guidance contained within the ITIL framework (Information Technology Infrastructure Library)
standard although it also supports ITSM framework and other approaches. The latest version is currently the
second edition of which is the development of ISO/IEC 20000:2011. ISO/IEC 20000:2011 consists of two
main parts in the service management certification standards. The first part is a standard specification for the
achievement of IT service management with the implementation rules for the management service. The first
part, ISO/IEC 20000-1, advocated the use of an integrated process approach to effectively provide the
appropriate managed service business and customer needs. The second part, ISO/IEC 20000-2, is a 'code of
Int J Elec & Comp Eng ISSN: 2088-8708
IT Service Management System Measurement using ISO20000-1 and ISO15504-8 ... (Imam Asrowardi)
4025
conduct' and describes the best practices for service management within the scope of ISO/IEC 20000-1 as
shown in Figure 1.
Figure 1. Service management system ISO/IEC 20000-1:2011
2.3. Refinement PRM ISO/IEC 20000- 1:2011
This process reference model (PRM) is a logical representation of the elements of the processes
within service management. Using the PRM in a practical application might require additional elements
suited to the environment and circumstances. The PRM specified in this part of ISO/IEC 20000 describes at
an abstract level the processes including the general service management system (SMS) processes implied by
ISO/IEC 20000-1. Each process of this PRM is described in terms of a purpose and outcomes. The PRM does
not attempt to place the processes in any specific environment nor does it pre-determine any level of process
capability required to fulfil the ISO/IEC 20000-1 requirements. Table 1 shows the example of refinement
PRM for ISO/IEC 2000-1:2011.
Table 1. Example of Refinement PRM for ISO/IEC 2000-1:2011
Process ID
SMS 2.
Name
SMS Governance of processes operated by other parties
Purpose
The purpose of the Governance of processes operated by other parties process is to
ensure the services supplied by other parties are managed to meet the service
requirements;
Outcomes
As a result of successful implementation of this process:
1.The objectives and requirements for service Management are identified and
established to satisfy business needs, the service provider's financial processes,
regulatory, contractual and statutory.
2.Services supplied by other parties are managed to meet the service requirements;
Base
Practice
SMS.2.1, Identify the objectives and requirements for service
management. The objectives and requirements for service management are identified
and established to satisfy business needs, the service provider's financial processes,
and regulatory, contractual and statutory requirements.[Outcome 1] SMS.2.2, Manage
services provided by other suppliers. Services supplied by other parties are managed to
meet the service requirements. [Outcome 2]
Inputs
12-01 Alternative party process requirements [Outcome 2]
Outputs
09-00 Report [Outcome 2]
12-01 Alternative party process requirements [Outcome 1]
09-02 Alternative parties performance evaluation report [Outcome 2]
ISSN: 2088-8708
Int J Elec & Comp Eng, Vol. 8, No. 5, October 2018 : 4023 – 4032
4026
2.4. Process assesment model ISO/IEC 15504-8
Process assessment model (PAM) is a method of measuring a process consisting of indicators of
process capability and process performance. Indicators of process capability is the ability of the process to
achieve the level of capability that is determined by the attributes of the process. Evidence of indicators of
process capability will support the assessment of the achievement of the process attributes.
A PAM comprises a set of indicators of process performance and process capability. The indicators
are used as a basis for collecting the objective evidence that enables an assessor to assign ratings. The set of
indicators included in this part of ISO/IEC 15504 is not intended to be an all-inclusive set nor is it intended to
be applicable in its entirety [5], [6]. Figure 2 shows the relationship between the PAM and its inputs [4].
Figure 2. Relationship between the PAM and its inputs [4]
Assessment activities performs to distinguish between levels 1 to a higher level. This is done
because the level 1 determines whether a process reaches its destination, and therefore it is very important to
achieve, and also the basis for reaching a higher level. The results obtained at each level will be categorized
into four classifications as shown in Table 2.
Table 1. The Four-Point Attribute Rating Scale
Rating and Designation
Description
N (Not achieved),
There is no evidence of achivement for defined attribute. The number
value achieved in this classification range 0-15 %.
P (Partially achieved/ achieved in part),
There is some achivement of the defined attribute. The number value
achieved in this classification range 15- 50%.
L (Largely achieved)
There is significant achivement of the defined attribute. The number
value achieved in this category ranges from 50-85 %.
F (Fully achieved / fully achieved),
In this classification, there is full achivement of the defined attribute.
The number value achieved in this category range from 85-100 %.
There are six capability levels [7]. At each level there is no ordering between the process attributes;
each attribute addresses a specific aspect of the capability level [7]. The list of process attributes was showing
in Table 3. PAM provide the basis for determining whether process attributes have achive:
a. Capability Level 1 — Indicators are specific for each process and assess whether the following Table 3
shows the attribute has been achieved the implemented process achieves its process purpose.
b. Capability Levels 2 to 5 — Assessment of capability is based from generic process indicators of
performance. These are called generic because they apply across all processes, but they are different from
one capability level to another.
Int J Elec & Comp Eng ISSN: 2088-8708
IT Service Management System Measurement using ISO20000-1 and ISO15504-8 ... (Imam Asrowardi)
4027
Table 2. Capability Levels and Process Attribute
Process Attribute
Capability Levels and Process Attributes
Level 0: Incomplete process, the process was not implementing and general failur to attain its
process purpose.
Level 1: Performed process, the purpose of process was achieve. The achivement may not be
tracked or planned.
PA 1.1
Process performance
Level 2: Managed process, the process delivers work products of acceptable quality and
defined by timescale. The primary of process related work products are appropriately
established, controlled and maintained.
PA 2.1
Performance management
PA 2.2
Work Products management
Level 3: Established process, the previously described managed process was implementing
using a defined process that is capable of achieving its process outcomes.
PA 3.1
Process definition
PA 3.2
Process deployment
Level 4: Predictable process, the defined process was performing consistently in practice
within defined limits to achieve its process outcomes
PA 4.1
Process measurement
PA 4.2
Process Control
Level 5: Optimizing process, the defined process was optimizing to meet curent and future
businnes needs. This process also consistently in practice within defined limits to achieve its
work products and outcomes.
PA 5.1
Process innovation
PA 5.2
Continuous optimization
3. RESEARCH METHODOLOGY AND IMPLEMENTATION
This research uses concurrent embedded mixed method as its methodology. Generally, this research
uses qualitative methodology by gathering qualitative data, literatures, and elaborating some best practices or
standards to design IT service governance. The first phase is to do literature analysis to evaluate the
assessment and the measurement process of IT service. The analysis is a qualitative method. But, on the last
part, a quantitative method is used to prove the qualitative method. The method is only a small part of the
whole qualitative process. It can be inferred that quantitative method is embedded in a qualitative method.
The said method is illustrated in Figure 3.
Figure 1. Research mehodology
After IT service measurement model is produced, quantitative validation is the next process on the
line. Validation is carried on by using the model in assessing a case. Specifically, framework used in this
research refers to a proactive research approach created by Cole R, et al. That approach has four process
steps that needs to be done in the research.
a. Problem identification is a process of defining a research problem and could cover the creation of initial
concept that involves research objects.
b. Intervention is a planning, developing, and action taking process to make a construction, model,
prototype, or other resources.
c. Evaluation is a process of observation and measurement of compatibility level and accuracy of produced
output in order to support the solution of said problem. This process is a validation of model or design by
using expert judgement.
d. Reflection and lessons learned. This is a process for produced output. The reflection and lessons learned
aims to report the output of the research and to find its contribution to the practice and the theory.
In framework development phase, we start with placing the initial requirements as the direction in
finding ITSM and IT service supporting theory literature. Theoretical model for designing metrics is showed
Qualitative
Quantitative
Analysis of Findings
Concurrent Embedded Design
ISSN: 2088-8708
Int J Elec & Comp Eng, Vol. 8, No. 5, October 2018 : 4023 – 4032
4028
in Figure 4. The outcome of the finding will be combined by the experiences from practicioner in this field
and will be inserted into a measurement metrics. Framework in this integration is showed in Figure 5.
Figure 2. Theoretical model for designing metrics
Figure 5. Framework element process
The next step in creating the metrics is to design Process Assessment Model (PAM) ISO/IEC
15504-8:2013 which is based on Process Reference Model (PRM) ISO/IEC 20000-4. PRM used in ISO/IEC
15504-8:2013 uses ISO/IEC 20000-1:2009 standard, so that researchers refines the PRM by improving its
compatibility with ISO/IEC 20000-1:2013. It became the base of the state-of-the-art of our research. PAM
Int J Elec & Comp Eng ISSN: 2088-8708
IT Service Management System Measurement using ISO20000-1 and ISO15504-8 ... (Imam Asrowardi)
4029
from the refinement is used by researcher in doing appropriate conformant assessment to the requirements of
ISO/IEC 15504-2 and ISO/IEC 20000-1:2013. The frame of mind in creating this evaluation tools is
illustrated in Figure 6.
The components involved in designing a evaluation tools consist of process dimension, SMS
processes list and capability dimension. Process dimension is a PRM process list that will be measured based
on ISO/IEC 20000-4; each process contains the list of assessment component and the purpose to support the
overall process of SMS.
Figure 6. Framework implementation
3.1. Element assesment process
ISO/IEC 15504-1 suggest the assessment process contains at least five specified activities: planning,
data collection, data validation, process attribute rating, and reporting. The assessment process must be
documented; in addition, the assessors must record the objective indicators of performance or capability used
to justify the ratings. A team carries out the process assessment with at least one competent assessor who has
the competencies. An assessment is carried out by assessing selected processes against the assessment
model(s) chosen for the assessment. The assessment model(s) have to be compatible with the requirements
defined in ISO/IEC 15504-2. The Process Reference Model is selected according to the application domain
of interest [6].
3.2. Implementation tools
A self-assessment by expert can identify process gaps that require improvements in advance of a
formal assessment; it can be done for a relatively small investment and assists enterprise management in
setting target capability levels.
1) Step 1 – Deciding process to asses scoping
Van Bon et al. (2008) explain the attributes to measure IT service process as a component.
However, van Bon et al. (2007b) that a characteristic of modern ITSM is an end-to-end approach. The
following attributes of an IT service can be used to measure quality:
1. Availability, ability of a service or service component to perform its required function at an agreed instant
or over an agreed period of time
2. Capacity, current and forecast demand for services also expected impact of agreed requirements for
availability, service continuity and service levels
3. Performance, the fulfillment of a claim, promise, or request of the services at planned intervals, with the
customer
4. Security, preservation of confidentiality, integrity and accessibility of information
5. Confidentiality, containing information whose unauthorized disclosure.
6. Scalability, capable of being easily IT service to be expand or upgradeon demand
7. Adjustability, to bring IT service or process to a more satisfactory state
8. Portability, the quality or state the IT service of being portable
We tried to elaborate ITSM process and the attribute component into a table.
Step 1 – Deciding
process to asses
scoping
Step 2 - Determine
Level 1 Capability
Step 3 - Determine
Capability for Levels 2
to 5
Step 4 - Record and
Summarise the
Capability Levels
Step 5 - Plan
Process Improvement
and Make
Recomendation
ISSN: 2088-8708
Int J Elec & Comp Eng, Vol. 8, No. 5, October 2018 : 4023 – 4032
4030
Table 3. ITSM Process and Metrics to Measure
ITSM Process
Metrics to Measure
Service Process Owner
SMS 1. Management Responsibility
Availability,
Chief Information Officer
SMS 2. Governance of Processes Operated
Performance,
Chief Information Officer
SMS 3. Documentation Management
Adjustability, security, confidentiality
Release Manager
SMS 4. Resource Management
Capacity, performance, availability
Capacity Manager
SMS 5. Establish the SMS
Availability, capacity, adjustabilit
Service Desk Manager
DTR 1. General
Performance, portability
Capacity Manager
DTR 2. Plan New or Changed Services
Capacity, performance, availability
Configuration Manager
DTR 3. Design and Development of New or
Changed
Capacity, performance,
Availability, scalability.
Configuration Manager
DTR 4. Transition of New or Changed Services
Capacity, performance, availability,
adjustability.
Configuration Manager and
Change Manager
SDE 1. Service level management
Availability, capacity
Release Manager
SDE 2. Service Reporting
Availability,capacity, performance,
security
Release Manager
SDE 3. Service Continuity and Avaibility
Management
Availability, performance
Release Manager
SDE 4. Budgeting and Accounting for Services
Capacity, adjustability
Release Manager
SDE 5. Capacity Management
Capacity, performance
Release Manager
SDE 6. Information Security Management
Security, confidentiality, avaibility
Release Manager /CSO
REL 1. Business Relationship Management
Avaibility, performance
Relationship Manager
REL 2. Supplier Management
Avaibility, performance
Relationship Manager
RES.1 Incident and Service Request
Management
Security, confidentiality, avaibility
Incident Manager
RES.2 Problem Management
Security, confidentiality, avaibility,
capacity
Problem Manager
CON.1 Configuration Management
Capacity, confidentiality, avaibility
Change Manager
CON.2 Change Management
Scalability, portability
Change Manager
CON.3 Release and Deployement Management
Scalability, portability, adjustability
Change Manager
The first step in the self-assessment is to decide what processes will to assess. Those processes
selected and to have record in the table as shown in Table 5.
Table 4. Capability Levels and Process Attribute
Process Name
Level 1
Level 2
Level 3
Level 4
Level 5
PA 1.1
PA 2.1
PA 2.2
PA 3.1
PA 3.2
PA 4.1
PA 4.2
PA 5.1
PA 5.2
Target
F
Rating by Criteria
Rating
Capability Level Achieved
At this stage, the target process capability level can be recorded. This will establish the level of
capability required of the process. In setting the target capability levels, consideration will give to the impact
on the business objectives of the enterprise if a specified level of capability will not achieve. The first
consideration is the impact on the enterprise if the process is non-existent or not working effectively or
efficiently.
2) Step 2 - Determine Level 1 Capability
The first step in the assessment of each process is to determine whether a process is actually being
performed and is achieving its outcomes. In the self-assessment worksheet Table 4 there is a table for each
process. The indicators at capability level 1 are specific for each process and assess whether the following
attribute has been achieved: The implemented process achieves its purpose.
3) Step 3 - Determine Capability for Levels 2 to 5
Above level 2, the assessment criteria are generic, i.e., the criteria are the same for each and
every process. Again, in each case, a judgment must be made as to whether the criteria have been met, and
that decision must be translated into a rating and recorded in the template for the process. This should be
repeated for each capability until a capability level is rated as ‗largely‘ or ‗fully achieved‘.
4) Step 4 - Record and Summarise the Capability Levels
The summary of assessment results should be recorded in Section 1. The capability level is
determined at the level where both capability indicators are either ‗largely‘ or ‘fully achieved‘. Those
processes selected should be recorded in the table as shown in Table 4.
Int J Elec & Comp Eng ISSN: 2088-8708
IT Service Management System Measurement using ISO20000-1 and ISO15504-8 ... (Imam Asrowardi)
4031
5) Step 5 - Plan Process Improvement and Make Recomendation
Based on the self-assessment, consideration should be given to the development of a plan of action
for process improvement. One option could be to commence an initial mprovement plan based on the self-
assessment. This could address the areas of highest importance to the enterprise‘s business goals and focus
on areas with gaps between the ‗current‘ and ‗target‘ process capability levels.
3.3. Implementation
We tried to the implement measurement metrics in an organizational unit of ICT service
management in a higher education institution. Scope definition of evaluation by using this metrics
encompasses main business processes of ICT services provided by the organization. The organization,
currently, is functioning as a system support in delivering the services related to those activities.
The ITSM proceses - capability level assessment is carried on by gradually evaluate whether the
processes are fulfilling the requirements on every level, from level 1 to level 5. There are category provisoins
of assessment result on every level, those are largely achieved (L) with range of score between 50-85%, and
fully achieved (F) with score range between 85%-100%, so that the process can be declared that it has
attained a capability level. The assessment is performed until the process achieve the expected level set by
the organization.
From the outputs of metrics as shown in Figure 7. We argue the true value of IT service provision
can only be realised when the services it provides are solutions to identified business needs that are both
practical and reliable. In order to achieve this, these services need to be well managed. This improvement in
the provision and management of services promotes the credibility of the industry while improving customer
loyalty and satisfaction. Service providers must ensure their capacity to provide and manage identified
services. This introspection is best performed in an environment that does not contain a customer.
Figure 3. The assesment result using proposed metrics
4. CONCLUSION
In this paper we suggest the metrics in each service processes enables organizations to predict a
direction for active process enhancement and to identify if the goal of process can achieve. This objective
process metrics based on ISO/IEC 15504-8 and PRM ISO/IEC 20000-4 refinement, which has not been
introduced in previous process assessment models, can be expected to measure process capability and to
identify the risk, problems, and condition of process performance by using these metrics. This metrics
provides a basis for use in IT service process improvement. From the results of this paper, following
advantage can be obtained: facilitates self-assessment, produces a process rating for every ITSM process, and
can raise formation's IT service management system process capability level. This metrics has already testing
by author with implementation and justification by ITSM Expert at Polinela (higher education institution).
The output of this research, in the form of metrics and tools for organizational use, can be downloaded in
http://bit.ly/ITSMtools2018.
ISSN: 2088-8708
Int J Elec & Comp Eng, Vol. 8, No. 5, October 2018 : 4023 – 4032
4032
REFERENCES
[1] K. Radha, et al., ―Service Level Agreements in Cloud Computing and Big Data‖, International Journal of
Electrical and Computer Engineering, vol. 5, pp. 158-165, 2015.
[2] M. A. M. Stambul and R. Razali, ―An Assessment Model of Information Security Implementation Levels‖, in
Electrical Engineering and Informatics (ICEEI), 2011 International Conference on, 2011.
[3] S. Buckby, et al., ―The Current State of Information Technology Governance Literature‖, Information Science
Reference (IGI Global), 2008.
[4] M. Mora, et al., ―An Extensive Review of IT Service Design in seven International ITSM Processes Frameworks:
Part I‖, International Journal of Information Technologies and Systems Approach, vol. 7, pp. 83-107, 2014.
[5] ISO/IEC JTC 1, Information technology — Process assessment —ISO/IEC 15504 Part8.
[6] ISO/IEC JTC 1, Information technology — Process assessment —ISO/IEC 15504 part 1.
[7] Isaca, ―COBIT Process Assessment Model (PAM): Using COBIT 5‖, ISACA, 2011.
