Content uploaded by Ebru Gökalp
Author content
All content in this area was uploaded by Ebru Gökalp on Nov 15, 2017
Content may be subject to copyright.
ClouDSS: A Decision Support System for Cloud Service
Selection
Umut Şener, Ebru Gökalp, P. Erhan Eren
Informatics Institute, Middle East Technical University
Ankara, Turkey
{sumut,egokalp,ereren}@metu.edu.tr
Abstract.
C
loud computing brings in significant technical advantages and en-
ables companies, especially small and medium size enterprises (SMEs), to elim-
inate up-front capital expenditures. This is due to the various benefits it pro-
vides, such as pay-as-you-go service model, flexibility of services, and on-
demand accessibility. The proliferation of cloud services leads to their wide
spread use and calls for comprehensive evaluation approaches in order to be
able to choose the most suitable alternatives. To this end, existing studies in the
literature generally provide solutions incorporating a single method for making
such decisions. Therefore, this study proposes a more comprehensive solution
in the form of a decision support system named as ClouDSS which employs
various Multi-Criteria Decision Making (MCDM) methods with the aim of op-
timizing cloud service selection decisions. ClouDSS has a default decision
model, which can be customized according to enterprise-specific requirements,
for evaluating the suitability of cloud services with respect to business needs.
After presenting the main components of ClouDSS, the employed cloud service
selection process is described in order to highlight the associated tasks, includ-
ing both objective and subjective evaluation approaches. Furthermore, the ap-
plicability of the proposed system is demonstrated through a case study.
Keywords: Economics of Cloud Computing, Service Selection, Decision Sup-
port System, SME, Multi-Criteria Decision Making
1 Introduction
Enterprises have been adopting Cloud Computing (CC) technologies which provide
various opportunities such as scalability, flexibility, and on-demand availability. In-
deed, CC provides financial benefits including reduced expenditures for existing ap-
plications as well as the availability of innovative IT at an affordable operating cost
[1]. Among the main drivers of CC are economics and simplification of software
delivery and operation [2]. Due to offered benefits such as competitive advantages,
significant cost savings, and enhanced business processes, CC is an attractive proposi-
tion for many Small and Medium Size Enterprises (SMEs) [3, 4].
2
Despite the high rate of IT related advances, the growth of CC adoption by SMEs is
relatively slow. Since the CC adoption related concerns are multifaceted, the assess-
ment and selection of a variety of available cloud services with similar functions in
the market have become a major challenge [5]. Practitioners in SMEs are faced with
complex decisions regarding the selection of most suitable CC services for their busi-
ness activities. This is because the decision includes a comprehensive analysis of all
potential criteria influencing the CC service adoption and utilization. These criteria
may vary depending on the business structures of SMEs, and may include improved
efficiency, increased availability, fast deployment, and elastic scalability, security
concern, privacy issues, and information loss [6–8]. Therefore, the CC service selec-
tion for SMEs is a complicated decision-making process, which may benefit from
multi criteria decision making (MCDM) methods. Although there has been a growing
number of studies regarding CC adoption in SMEs [4, 9, 10], a literature review [11]
indicates that only few of them are related to the use of MCDM approaches for CC
adoption in SMEs.
The decision support system (DSS) concept is described as “computer based in-
formation systems that provide interactive information support to managers during
the decision-making process” [12]. DSSs are interactive and well-integrated systems
which provide managers with data, tools, and models to facilitate semi-structured
decisions that are unique, rapidly changing, and not easily specified in advance. The
information system architecture is relatively less complex for the case of SMEs, but
lacks computer aided decision-making capability. Therefore, the development of
computerized decision support for SMEs will contribute to their innovation and pros-
perity [13].
The aim of this study is to analyze existing work related to cloud service selection
decisions in SMEs and to develop a DSS providing a collection of MCDM methods
for supporting such decisions. Accordingly, the literature is reviewed systematically
to identify studies related to the cloud service selection approaches in SMEs. Then, by
analyzing the existing studies, and considering the strengths and weaknesses of them,
a DSS named as ClouDSS is proposed. The aim of ClouDSS is to provide a compre-
hensive approach for assessing cloud service alternatives in order to find the best
selection for a given company maximizing the economic benefits of CC technologies.
The remainder of the paper is structured as follows: Section 2 provides background
information which covers economic benefits of cloud computing for SMEs, a brief
description of MCDM methods for cloud service selection, and a systematic literature
review of existing proposed DSSs for cloud service selection decision of SMEs. Sec-
tion 3 presents the components of ClouDSS together with the description of the cloud
service selection process. Consequently, a case study that demonstrates the applicabil-
ity of ClouDSS is presented in Section 4, followed by the conclusion of the study.
3
2 Literature Review
2.1 Economic Benefits of Cloud Computing for SMEs
CC provides the capability to provision on-demand services at a cheaper cost than on-
premises alternatives, with reduced complexity, improved scalability, and broader
availability. In CC, various services such as computing, storage, and software are
available and accessed over the internet.
SMEs have a significant role in terms of supporting national economies. Because
small companies have flexible organizational structures which are easily adaptable to
new economic conditions or market trends. Although SMEs are capable of creating
innovation, their technical capacities constitute a barrier regarding opportunities and
profits resulting from economies of scale obtained by large companies [14]. In addi-
tion, SMEs have limited financial capabilities, and new expenditures may cause fatal
results in business, therefore they try to carry out cost-effective hardware and soft-
ware investments. CC addresses these issues and provides on-demand and flexible
solutions for SMEs, at lower cost levels, thereby reducing potential risks of invest-
ments as well as boosting productivity and creativity in businesses.
The economic benefits of CC for SMEs are identified as follows [15]: strategic
flexibility (the ability of quick deployment for entering a new market), reduced cost
(no up-front and maintenance costs due to pay-per-use), software availability (reduced
or no licensing fees), scalability (practically endless resources and automatic scaling
based on changing demand), skills and staffing (reduced need for highly-skilled per-
sonnel), energy efficiency (reduced utility cost), and system redundancy (data recov-
ery for better action plan in case of system failure). The quick deployment ability of
cloud services and reduced Total Cost of Ownership of cloud solutions such as SaaS
seem to be more appropriate for SMEs than large organizations [16]. Accordingly,
SMEs are in need of selecting and deploying CC solutions based on their specific
business requirements.
2.2 Cloud Service Selection By Using MCDM
CC service selection is a complicated decision-making process requiring the use of
MCDM approach for identifying the most suitable cloud services among available
alternatives [17–19]. As stated in [20], MCDM methods are commonly applied to
study complex problems, since they provide a well-structured approach in the opera-
tions research domain, and their efficiency is proven in solving complicated and mul-
ti-dimensional decision making problems [17]. MCDM includes a set of methods for
making comparisons, prioritizing multiple alternatives, and selecting the best-fit
choice. Among these methods which include Min-Max, Max-Min, ELECTRE,
PROMETHEE, TOPSIS, Compromise Programming, Analytic Hierarchy Process
(AHP), Fuzzy AHP, Data Envelopment Analysis (DEA), and Goal Programming, the
most widely used one is AHP. It is also quite suitable for cloud service selection deci-
sions [18].
4
2.3 DSSs Developed for Cloud Service Selection Decision
A systematic literature review conducted by following the method given in Kitchen-
ham [21] is presented in this section. DSSs developed for cloud service selection for
SMEs is selected as the research topic and the starting point of the search. The search
query is defined as {{"decision" OR "decision-making" OR "DSS" OR “decision
support system” OR "Service Selection"} AND "cloud"}. Web of Science
(www.webofknowledge.com) and Aisel (aisel.aisnet.org) are selected as databases for
the search. In Web of Science, 36 papers are collected, while in Aisel only 33 papers
are identified. First of all, SSCI, SCI, and AIS index journals, and conference pro-
ceedings, series, meetings, and reviews are selected among the resulting papers. Be-
fore reading papers fully, keywords, titles, and abstracts of the studies are checked to
assess whether they are related to the research topic. Then, the publication date is
selected as between 2000 and 2017. A significant portion of the collected studies is
related to the decision of cloud services adoption. They mainly investigate the identi-
fication of significant decision criteria instead of proposing a DSS. After this elimina-
tion, only eight studies remain and they propose a DSS solution which is based on a
single model such as AHP, Fuzzy AHP, and DEA:
• The service selection based on user feedback [22] is proposed as a decision model
for cloud selection. However, this model covers the subjective assessment of cus-
tomers and the assessment of third-party organization. Therefore, this model ap-
pears to be inconvenient for SMEs.
• Karim et. al [23] propose an AHP-based decision model for cloud service selec-
tion.
• Wilson et. al [24], Godse and Mulik [25], Garg et. al [18] propose a DSS based on
AHP ranking. But it does not provide additional assessment methods.
• Whaiduzzaman et. al [17] investigate the available MCDM methods. But, they do
not present a decision model or DSS.
• Rehman et.al [26] propose a scenario based MCDM for IaaS selection and com-
pare the results of 7 MCDM methods. However, they utilize matlab functions and
usage of the model requires domain knowledge, which can be difficult for SMEs to
use.
• Eisa et. al [27] investigate the trends in cloud service selection. They present dif-
ferent online assessment tools such as RightCloudz, Intel Cloud Finder, and
Cloudorado. They give a comparison of these tools in terms of their capabilities.
MCDM methods are not directly incorporated into their proposed solution.
As a result of the systematic literature review, it can be concluded that there is a lim-
ited number of studies proposing DSS for cloud service selection. The analysis of
existing studies reveals that they provide solutions utilizing a maximum of two deci-
sion models and their structures are not customizable according to enterprise specific
requirements. Therefore, in order to provide a more comprehensive solution, this
study proposes a customizable DSS for selecting the most suitable cloud services. The
system is intended to be also accessible for users with limited domain knowledge
regarding CC and decision making approaches. The proposed solution is described
next.
5
3 Development of ClouDSS
The system architecture of ClouDSS comprises three main components of a typical
DSS: Data Management, Model Management, and Knowledge Management, as
shown in Figure 1. The proposed ClouDSS is designed as a DSS for cloud service
selection process which contains a set of semi-structured decisions requiring individ-
ual judgment. It focuses on determining the best cloud service alternative based on
both objective and subjective evaluation by using MCDM methods including AHP,
Fuzzy AHP, linear optimization, goal programming, etc. The unique aspect of
ClouDSS is that it provides
Fig. 1. System Architecture of ClouDSS
Table 1. Default Assessment Criteria Set of ClouDSS
Criteria ID Assessment Criteria Attributes
AC1 Functionality Operations and functions set
Requirement set (memory, CPU, bandwidth)
Fitness for business purposes
Data migration and export capabilities
Business partners’ requirements
AC2 Security & Privacy Conformance (Legal Requirements / Standards)
Reputation (trust toward providers)
Enterprise specific requirements (encrypted data
storage, enhanced security level, PII controls)
Disaster Recoverability
Ease of monitoring
6
AC3 Performance System Uptime
Reliability
Response Time
Elasticity
AC4 Usability User-friendly interface
Ease of use
AC5 Economic Value Price of the product
Additional operating cost of the product
Cost of the downtime
Total cost of Ownership (i.e. Implementation
cost, personnel training cost, licensing fees, etc.)
various techniques within a single system, and Decision Makers (DMs) can access the
system over the internet for making cloud service selection decisions efficiently and
comprehensively.
3.1 Identification of the Criteria Set for MCDM
The criteria set for adopting a cloud-based enterprise solution has already been identi-
fied in our earlier study [28], based on an extensive literature search. In that study, the
factors are ranked by employing the AHP method with 20 experts. Based on these
results, the highest ranking factors are chosen as the assessment criteria set to be used
in ClouDSS (Table 1). Each criterion consists of several attributes that enable DMs to
evaluate cloud service alternatives. While the default assessment criteria set consists
of five main items including functionality, security&privacy, performance, usability
and economic value, additional criteria can be chosen from an extended collection
available in ClouDSS.
3.2 Cloud Service Selection Process in ClouDSS
The cloud service selection process and the interrelated set of tasks performed in con-
junction with the ClouDSS modules are shown in Figure 1. The DM accesses
ClouDSS after registering and entering the company information such as company
size, sector, number of employees, and business structure. After the user is authenti-
cated successfully, the DM selects the type of cloud service, such as Enterprise In-
formation Systems, Enterprise Content Management Systems. The DM can make an
objective evaluation by obtaining a feature comparison table for the cloud service
alternatives, including objective parameters such as languages provided, hourly pay-
as-you-go (yes/no), and SLA level. The DM can also make a subjective evaluation,
which means finding the best-fit cloud service alternative by weighing multiple crite-
ria based on his intuition, judgement, and experience regarding cloud services. If the
DM wants to make an objective comparison, he selects the set of features in order to
compare the service alternatives to be shown in the comparison table. He also selects
a suitable user profile for user reviews matching his own requirements, such as user
experience, user review rating, and company size the user works at, and the system
displays the reviews. As a result, he obtains the feature comparison table which is in a
7
matrix form showing features versus service alternatives. Reviews of other users are
also shown at the end of the table for each cloud service. Once the DM makes the
cloud service selection decision based on this table, the option of making additional
subjective evaluation before making the final decision is also offered to the DM. If he
selects this option, MCDM is performed after the selection of model (the default
model is AHP), criteria from the criteria set (the default criteria are functionality,
security &privacy, performance, usability, and economic value), and service alterna-
tives from the cloud services set. Then, the system requests the user to perform a pair-
wise comparison of the selected criteria followed by a comparison of alternatives for
each selected criteria. For example, if the best-fit solution is to be chosen among sev-
en alternatives by using the default criteria set containing the five criteria, six pairwise
comparison matrices are requested to be filled by the DM (one for pair-wise compari-
son of criteria and five for pair-wise comparison of alternatives for each criteria).
Upon completion of comparisons, ClouDSS displays the results report including the
scores for each alternative and offers to perform additional assessments by using dif-
ferent models. If the DM selects an additional assessment, available models are dis-
played for selection. After selecting the additional model, the assessment is conducted
and the resulting report is obtained. As a result, the process is concluded by making
the cloud service selection decision. ClouDSS consists of five modules as described
below.
MCDM for Cloud Service Selection: This module includes algorithms imple-
menting MCDM Models such as AHP, Fuzzy AHP, DEA and Goal-Programming to
provide optimized decision making.
Cloud Services Information Fetcher: This module includes APIs developed for
extracting up-to-date information about cloud services by constantly checking their
provider’s websites to find out if there is any new information. The collected data is
stored as cloud services data.
Guidance for Parameters and Criteria Selection: This module is responsible for
providing assistance to DMs with specifying parameters used for objective evaluation
and criteria for subjective evaluation. This module also represents parameters and
criteria in a uniform way so that users with little knowledge about cloud technologies
can easily understand and specify their requirements.
User Review for Cloud Services: This module aims to manage user review data
related to cloud services. IT experts or other DMs using the services provide reviews
for cloud services, which are rated by other DMs based on usefulness and correctness.
Cloud Services Monitoring: Some quantitative Key Performance Indicator (KPI)
values regarding performance and reliability of cloud services, such as response time
and system up-time, are obtained, monitored and managed by this module which reg-
ularly checks cloud providers. The real-time values obtained periodically for this kind
of KPIs are stored as KPI data. The DM can select the time interval in which the val-
ues are collected. ClouDSS gives real-life measures for these KPIs in order to in-
crease the decision quality. However, for some cloud services, it is not possible to
monitor them as they may not have interfaces for monitoring purposes.
8
4 Case Study
The applicability of ClouDSS is presented by employing a usage scenario in this sec-
tion. The SMEs need to assess the different aspects of the alternative cloud solutions
before implementing; therefore they need a set of methods in order to evaluate the
different aspect of the alternatives and to select the best-fit solution among them.
Fig. 2. Cloud Service Selection Process in ClouDSS
9
A small company considers implementing a cloud-based Enterprise Content Man-
agement (ECM) solution. This decision is made by following the subjective evalua-
tion path described in Fig. 2. Six Decision Makers (DMs) in a given SME try to de-
termine the best alternative among three cloud service alternatives X, Y, and Z, with
respect to the requirement set provided in Table 1. Decision makers employ pairwise
comparisons of the AHP methodology to obtain the following: i) Prioritize the as-
sessment criteria independently, ii) Prioritize the feasible products independently, iii)
Merge the results of the prioritization to identify the best solution.
The default decision model in ClouDSS is AHP. The decision criteria together with
their descriptions are provided by the ClouDSS user interface. If the company has
additional requirements apart from the criteria in the default decision model, the deci-
sion model can be enhanced by selecting those items from the criteria pool in
ClouDSS. After finalizing the decision criteria, the ClouDSS construct judgment ma-
trix is formed based on the AHP method.
The judgment matrix consists of the pairs that the decision makers compare. In this
case, six experts compare each item of the comparison pairs to each other, and ex-
press their individual rankings for the comparison by using Saaty’s Scale [29]. This
scale allows the decision makers to convert their linguistic judgment into a numerical
measure which represents the relative importance of items in each pair. The scale is
from “1”, which represents “equally important”, to “9” which represents “extremely
important”.
Table 2. The Weights of the Products Based on Each Criterion
AC1 AC2 AC3 AC4 AC5 Overall Priority
Alternative X 0.36 0.53 0.47 0.37 0.37 0.42 1
Alternative Y 0.35 0.32 0.22 0.20 0.39 0.30 2
Alternative Z 0.29 0.14 0.31 0.43 0.25 0.28 3
ClouDSS checks the consistency of each judgment matrix in order to prevent incon-
sistent judgments of the experts, and once the consistency ratio is calculated as over
10%, a notification is sent to the corresponding user to revise his judgment. After the
consistency check, the weight of each criterion is determined. The resulting weights
obtained by combining the comparison results of six DMs are given in Table 2. Ac-
cording to the AHP ranking, the highest weight is assigned to X. But the weights of Y
and Z are very close to each other. Therefore, the company may prefer to conduct an
additional analysis such as DEA, in order to evaluate them, as described next.
The DM investigates the most cost-effective product among the three different
cloud-based enterprise solutions given above, and can employ Input-oriented DEA
decision model to select the best alternative. That means, it is investigated whether the
selected product can still increase its output (i.e., net income, etc.) or decrease its
input when compared to the “ideal” cloud product.
• Input 1: Amount of Subscription Payment per Year
• Input 2: Number of IT Personnel Hired
10
Table 3. The Input and Output of the DEA model
Cloud
Alternatives
Input 1
( million $)
Input 2 Output 1
(thousand)
Output 2
(thousand $)
Output 3
Alternative X
2 50 10 100 24
Alternative Y
1.6 44 8 80 16
Alternative Z
1.2 30 6 90 12
• Output 1: Average Number of Customers of the Enterprise
• Output 2: Expected Net Income from Investment
• Output 3: Expected Number of Business Partnership
According to the given inputs and outputs, for calculation efficiency of the Alterna-
tive X, the following linear optimization model is constructed.
Linear Optimization Model for Alternative X efficiency:
Minimize θ; minimize input resources (1)
Constraints:
2λ
A1
+ 1.6 λ
A2
+ 1.2λ
A3
<=2*θ (2)
50λ
A1
+ 44λ
A2
+ 30λ
A3
<=50* θ (3)
10λ
A1
+ 8λ
A2
+ 6λ
A3
>=10 (4)
100λ
A1
+ 80λ
A2
+ 90λ
A3
>=100 (5)
24λ
A1
+ 16 λ
A2
+ 12λ
A3
>=24 (6)
λ
A1
+ λ
A2
+ λ
A3
=1 (7)
λ
A1
, λ
A2
, λ
A3
>=0 (8)
Once this model is solved for Alternative X, λ
A1
=1, λ
A2
=0 λ
A3
=0, and; the efficiency
coefficient of Alternative X is calculated as “1”, which means Alternative X is found
as the efficient product. Similarly, each product efficiency can be calculated by the
DEA method.
As a result, Alternative X has the highest rank among others in AHP and it is also
found as efficient according to DEA. Therefore, the decision to choose Alternative X,
as suggested by AHP is further verified by DEA as an efficient selection.
5 Conclusion
CC provides significant benefits to SMEs both financially and technically. There are
various aspects of CC which are important for its adoption. Accordingly, the selection
of suitable cloud services turns into a complex decision problem requiring a compre-
hensive approach for making an optimum decision. Furthermore, each SME may be
operating under a unique set of circumstances which makes this decision even more
difficult. Therefore, a DSS that is capable of collecting relevant data as well as
providing a set of suitable methods becomes important in helping SMEs with cloud
service selection decisions. To this end, this study proposes ClouDSS which is a DSS
for cloud service selections.
The conducted systematic literature review reveals that there is a limited number of
studies proposing DSS for cloud service selection. Upon analyzing them, the system
architecture for ClouDSS is constructed in order to provide a more comprehensive
11
solution. Its system architecture containing data, model and knowledge management
components is described. Furthermore, the cloud service selection process by using
ClouDSS is presented in order to delineate the set of corresponding tasks. The ap-
plicability of the proposed system is demonstrated by providing a case study.
ClouDSS provides a set of assessment methods within a single system without the
need of expertise or knowledge in the domain of cloud technologies and decision
making approaches. The main contribution of the study is that it proposes a compre-
hensive DSS while a limited number of existing studies provides solutions utilizing a
maximum of two decision models. ClouDSS offers both objective and subjective
evaluation approaches for cloud service selection decision. For subjective evaluation,
10 MCDM methods are available to support decisions for identifying the best alterna-
tives according to enterprise-specific requirements. Another significant contribution is
that it provides customization of criteria for subjective evaluation and parameters for
objective evaluation, as well as the capabilities for searching and filtering of cloud
service alternatives and user reviews. Furthermore, it collects real-life measurements
for quantitative KPIs and up-to-date service information in order to increase the deci-
sion quality. Finally, it also provides guidance to DMs for specifying parameters and
criteria through easy to understand representations. While ClouDSS has been de-
signed by considering the needs of SMEs, the solution is suitable for use in large en-
terprises as well.
As part of future work, additional case studies are planned in order to further assess
the applicability of ClouDSS. Furthermore, its usability will be investigated by con-
ducting System Usability Scale (SUS) tests with DMs who are planning to adopt a
cloud service.
6 References
1. Sultan, N.A.: Reaching for the “cloud”: How SMEs can manage. Int. J. Inf. Manage. 31,
272–278 (2011).
2. Erdogmus, H.: Cloud computing: Does nirvana hide behind the nebula? IEEE Softw. 26, 4–
6 (2009).
3. Dillon, S., Vossen, G.: SaaS Cloud Computing in Small and Medium Enterprises : A
Comparison between Germany and New Zealand. Int. J. Inf. Technol. Commun. Converg. 3,
1–16 (2009).
4. Carcary, M., Doherty, E., Conway, G., McLaughlin, S.: Cloud Computing Adoption
Readiness and Benefit Realization in Irish SMEs—An Exploratory Study. Inf. Syst. Manag.
31, 313–327 (2014).
5. Sultan, N.A.: Reaching for the “cloud”: How SMEs can manage. Int. J. Inf. Manage. 31,
272–278 (2011).
6. Dutta, A., Peng, G.C.A., Choudhary, A.: Risks in enterprise cloud computing: the
perspective of IT experts. J. Comput. Inf. Syst. 53, 39–48 (2013).
7. Daniel, W.K., Chen, D., Liu, Q., Wang, F., Wei, Z.: Emerging issues in cloud storage
security: Encryption, key management, data redundancy, trust mechanism. In: International
Conference, MISNC. pp. 297–310. Springer (2014).
8. Wu, W.-W., Lan, L.W., Lee, Y.-T.: Factors hindering acceptance of using cloud services in
university: a case study. Electron. Libr. 31, 84–98 (2013).
12
9. El-Gazzar, R.F.: A literature review on cloud computing adoption issues in enterprises. In:
International Working Conference on Transfer and Diffusion of IT. pp. 214–242. Springer
(2014).
10. Oliveira, T., Thomas, M., Espadanal, M.: Assessing the determinants of cloud computing
adoption: An analysis of the manufacturing and services sectors. Inf. Manag. 51, 497–510
(2014).
11. Yang, H., Tate, M.: A descriptive literature review and classification of cloud computing
research. Commun. Assoc. Inf. Syst. 31, 35–60 (2012).
12. O’Brien, J.A., Marakas, G.: Introduction to information systems. McGraw-Hill, Inc. (2005).
13. Szabo, S., Ferencz, V., Pucihar, A.: Trust, Innovation and Prosperity. Qual. Innov. Prosper.
17, 1–8 (2013).
14. Isma’, S. Al, ili, N.A., Li, M., Shen, J., He, Q.: Cloud computing adoption decision
modelling for SMEs: a conjoint analysis. Int. J. Web Grid Serv. 12, 296 (2016).
15. Talukder, A.K., Zimmerman, L., Prahalad, H.A.: Cloud Computing. 34, (2010).
16. Seethamraju, R.: Adoption of Software as a Service (SaaS) Enterprise Resource Planning
(ERP) Systems in Small and Medium Sized Enterprises (SMEs). Inf. Syst. Front. 17, 475–
492 (2015).
17. Whaiduzzaman, M., Gani, A., Anuar, N.B., Shiraz, M., Haque, M.N., Haque, I.T.: Cloud
service selection using multicriteria decision analysis. Sci. World J. 2014, (2014).
18. Garg, S.K., Versteeg, S., Buyya, R.: SMICloud: A framework for comparing and ranking
cloud services. Proc. - 2011 4th IEEE Int. Conf. Util. Cloud Comput. UCC 2011. 210–218
(2011).
19. Lee, S., Seo, K.-K.: A Hybrid Multi-Criteria Decision-Making Model for a Cloud Service
Selection Problem Using BSC, Fuzzy Delphi Method and Fuzzy AHP. Wirel. Pers.
Commun. 86, 57–75 (2016).
20. Dyer, J.: Multiple criteria decision analysis: state of the art surveys. Int. Ser. Oper. Res.
Manag. Sci. 78, 265–292 (2005).
21. Kitchenham, B.: Procedures for performing systematic reviews. Keele, UK, Keele Univ. 33,
28 (2004).
22. Qu, L., Wang, Y., Orgun, M. a: Cloud Service Selection Based on the Aggregation of User
Feedback and Quantitative Performance Assessment. 2013 IEEE Int. Conf. Serv. Comput.
152–159 (2013).
23. Karim, R., Ding, C., Miri, A.: An End-to-End QoS Mapping Approach for Cloud Service
Selection. 2013 IEEE Ninth World Congr. Serv. 341–348 (2013).
24. Wilson, B.M.R., Khazaei, B., Hirsch, L.: Towards a cloud migration decision support
system for small and medium enterprises in Tamil Nadu. CINTI 2016 - 17th IEEE Int.
Symp. Comput. Intell. Informatics Proc. 341–346 (2017).
25. Godse, M., Mulik, S.: An approach for selecting Software-as-a-Service (SaaS) product.
CLOUD 2009 - 2009 IEEE Int. Conf. Cloud Comput. 155–158 (2009).
26. Rehman, Z.U., Hussain, O.K., Hussain, F.K.: Iaas cloud selection using MCDM methods.
Proc. - 9th IEEE Int. Conf. E-bus. Eng. ICEBE 2012. 246–251 (2012).
27. Eisa, M., Younas, M., Basu, K., Zhu, H.: Trends and directions in cloud service selection.
Proc. - 2016 IEEE Symp. Serv. Syst. Eng. SOSE 2016. 423–432 (2016).
28. Şener, U., Gökalp, E., Erhan Eren, P.: Cloud-based enterprise information systems:
Determinants of adoption in the context of organizations. (2016).
29. Saaty, T.L.: The Analytical Hierarchy Process, Planning, Priority. In: Resource Allocation.
RWS Publications, USA (1980).
