Next Article in Journal
On Spectral Properties of Doubly Stochastic Matrices
Next Article in Special Issue
Using Data Envelopment Analysis and Multi-Criteria Decision-Making Methods to Evaluate Teacher Performance in Higher Education
Previous Article in Journal
Exceptional Set for Sums of Symmetric Mixed Powers of Primes
Previous Article in Special Issue
Assessing Factors for Designing a Successful B2C E-Commerce Website Using Fuzzy AHP and TOPSIS-Grey Methodology
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:

Implementing a Novel Use of Multicriteria Decision Analysis to Select IIoT Platforms for Smart Manufacturing

Roberto Contreras-Masse
Alberto Ochoa-Zezzatti
Vicente García
Luis Pérez-Dominguez
1 and
Mayra Elizondo-Cortés
Department of Computer Science, ITCJ, Universidad Autonoma de Ciudad Juárez (UACJ), Ciudad Juárez 32310, Mexico
Instituto Tecnológico de Ciudad Juárez (ITCJ), Ciudad Juárez 32500, Mexico
Departamento de Ingeniería en Sistemas DIMEI-FI, Universidad Nacional Autonoma de Mexico (UNAM), Ciudad de México 04510, Mexico
Author to whom correspondence should be addressed.
Current address: Av. Tecnológico No. 1340 Fracc. El Crucero C.P. 32500 Ciudad Juárez, Chih, Mexico.
Symmetry 2020, 12(3), 368;
Submission received: 4 February 2020 / Revised: 21 February 2020 / Accepted: 27 February 2020 / Published: 2 March 2020
(This article belongs to the Special Issue Uncertain Multi-Criteria Optimization Problems)


Industry 4.0 is having a great impact in all smart efforts. This is not a single product but is composed of several technologies, one of them being Industrial Internet of Things (IIoT). Currently, there are very varied implementation options offered by several companies, and this imposes a new challenge to companies that want to implement IIoT in their processes. This challenge suggests using multi-criteria analysis to make a repeatable and justified decision, requiring a set of alternatives and criteria. This paper proposes a new methodology and comprehensive criteria to help organizations to take an educated decision by applying multi-criteria analysis. Here, we suggest a new original use of PROMETHEE-II with a full example from weight calculation up to IIoT platform selection, showing this methodology as an effective study for other organizations interested in selecting an IIoT platform. The criteria proposed stands out from previous work by including not only technical aspects, but economic and social criteria, providing a full view of the problem analyzed. A case of study was used to prove this proposed methodology and finds the minimum subset to reach the best possible ranking.

1. Introduction

Industry 4.0 is having a high impact in all industries. This is not a unique product, but it is composed of several technologies. Boston Consulting Group has defined nine technological pillars for Industry 4.0: cloud, additive manufacturing, simulation, big data and analysis, autonomous robots, augmented reality, integration of horizontal and vertical systems, cybersecurity and industrial internet of things (IIOT) [1]. IIOT has been used not only in the manufacturing industry, but has expanded to other industries such as health, travel and transportation, energy, gas and oil, etc. This is one of the main reasons that IIOT is known as the Internet of Things (IoT) [2]. IIoT is a key intelligent factor that allows factories to act intelligently. By adding sensors and actuators to objects, the object becomes intelligent because it can interact with people, other objects, generate data, generate transactions, and react to environmental data [3,4]. Cities do not ignore this trend, since there is a plan to turn cities into smart cities in certain countries [5].
The decision processes that companies must follow should be supported by methods that consider pros and cons of plural points of view that affect the decision process. Researchers and practitioners have developed over time the techniques that today are part of the domain of Multiple Criteria Decision Analysis (MCDA), which, very simplistically, requires three basic elements: a finite set of actions or alternatives, at least two criteria, and at least one decision making method. [6]. The MCDA has been the object of study and nowadays there are a lot of methods for decision-making in disciplines such as waste management, industrial engineering, strategies, manufacturing, even natural resource management and environmental impact [7].The purpose of this manuscript is precisely to propose a method of MCDA with the corresponding criteria for the selection of an IIoT(Industria Internet of Things) platform, which can serve as a starting point to companies and individuals embarked on implementation projects of Industry 4.0. Our conceptual model to solve the problem is shown in Figure 1.

1.1. Literature Review

Industria Internet of Things (IIoT) continues to evolve. Due to the instrinsic complexity, it is good practice to look at architectural references. IIoT have five main requirements on a general basis [8]: (1) Enable communication and connectivity between devices and data processing; (2) Establish a mechanism to manage devices, including tasks such as adding or deleting devices, updating software and configurations; (3) Gather all the data produced by the devices and then analyze them to provide a meaningful perspective to the companies or users; (4) Facilitate scalability to handle the increased flow of “data pipes” (hereinafter referred to as data pipelines) and the flow of data, and handle an increasing number of devices; (5) Protect the data by adding the necessary functions to provide privacy and trust between the devices and the users. Table 1 shows the summary of the various multi-layer architectures found in the literature.
Technical architecture provides an extreme value to users because it can be implemented with different products. Therefore, it is understandable that several companies offer IIoT platforms that can be useful for our architectures. Commercial providers aim for flexible options offered, and consumers are responsible for using each component in the best way they consider. The main commercial players identified are, in alphabetical order: Amazon Web Services, Bosch IoT Suite, Google Cloud Platform, IBM Blue Mix (now Watson IoT), Microsoft Azure IoT, and Oracle Integrated Cloud [19]. The leading players identified in 2014 by Gartner Group were AWS and Microsoft, but, in 2018, Google enters the leaders quadrant. IBM, for its commercial relevance, is considered, although it has become a niche player, along with Oracle. Although Bosch IoT does not appear in the panorama detected by Gartner, we include it for being used in several industries. Each of these suppliers has similar characteristics among them but have different value propositions.

1.1.1. MCDA as a Tool to Select an IIoT Platform

Making a decision introduces problems to individuals. One of the problems is the integration of heterogeneous data and the uncertainty factor surrounding a decision, and the criteria that usually conflict with each other [7,20]. To carry out a MCDA process, a series of tasks is proposed, based on the three generic steps suggested by [21]: (i) identify the objective or goal, (ii) select the criteria, parameters, factors, and attributes, (iii) selection of alternatives, (iv) association of attributes with the criteria, (v) selection of weight methods to represent the importance of each criterion, and (vi) the method of aggregation. Ref. [21] included a step that is left out of these proposed tasks, but which should be considered in the discussion before executing the selected action. This step is to understand and compare the preferences of the person making the decision.
The MCDA can be classified according to the basis of the problem, by type, by category, or by the methods used to make the analysis. Figure 2 shows a taxonomy adapted from [22]; the methods included in this taxonomy are not exhaustive. The MCDA is a collection of systematic methodologies for comparisons, classification, and selection of multiple alternatives, each one with multiple attributes and is dependent on an evaluation matrix. Generally, it used to detect and quantify the decisions and considerations from interested parties (stakeholders) about various monetary factors and non-monetary factors to compare an alternative course of action [7,22]. The major division that exists in MCDA lies in the category of methodologies. First, the group considers discrete values with a limited number of known alternatives that involve some compensation or trade-off. This group is called Multiple Attributes Decision Making (MADM). The other group is the Multiple Objectives Decision Making (MODM), and its variable decision values are within a continuous domain with infinite or very numerous options that satisfy the restrictions, preferences, or priorities [20]. In addition, there is another classification according to the way of adding criteria, and it is divided into the American school, which aggregates into a single criterion, and into the European or French school that uses outranking methods. It can be considered a mixture of both schools and they are indirect approaches, such as the Peer Criteria Comparison methods (PCCA) [23].

1.1.2. Use of MCDA to Select IIoT Platforms or Technology Platforms—Related Work

When finding the available alternatives of the market, a new question will arise to find the method that helps to select the appropriate option. To answer this last question, a review of the literature is made looking for: (a) MCDA methods applied to the selection of IIoT platforms and (b) knowing the criteria taken into account.
In the literature, there is little information on the subject in recent years. Table 2 shows the summary of the work found. The selected methods are focused on AHP, TOPSIS, and Fuzzy logic in AHP and TOPSIS. The outranking methods were not implemented but were considered as an option or for future work by some authors [24,25]. The selection of an IIoT platform is neither dominated by a single criterion nor is there a single alternative. Ref. [26] considered AWS, Azure, Bosch, IBM Watson, and Google Cloud within their options, which coincide with some of the alternatives considered in this manuscript. Therefore, it is interesting to review the criteria they included for MCDA, as summarized in Table 2.
Criteria found in literature are purely technical with some hints of economy, and can be found as part of the characteristics of IIoT architecture [32]. However, when implementing an IIoT platform, non-technical aspects should also be considered. As the platform to be considered has its foundation in the cloud, it is valid to review the criteria included in previous MCDA exercises to select a cloud provider, looking for non-technical aspects.
The criteria for selecting a cloud proposed in the CSMIC Framework v 2.1 of 2014 (Cloud Services Measurement Initiative Consortium (CSMIC) was created by Carnegie Mellon University to develop Service Measurement Index (SMI). it can be found at as the Index of Measure of Service (SMI) includes topics of interest to the organization, financial, and usability, together with the technical issues [31]. Some of these criteria can be included to complement the analysis having the technical point of view and the business point of view.
Finally, there is a question about which methods are suitable for these types of problems, noting that the previous work includes AHP, ANP, TOPSIS, and Fuzzy Logic, but they are left aside for future research methods such as PROMETHEE and ELECTRE. There are many more methods available in MCDA scope. Following the decision tree to select an MCDA method written by [23], which considers 56 methods, the number of options can be easily reduced. In the case of selecting an IIoT platform that has different criteria, the problem has the characteristics of classification or ranking, ordering the options from best to worst. This technique is useful in real life, since they hardly conform and subject themselves to a single option, but they have to consider their primary option and another option as backup, assuming that the first option is not viable.
Of the 29 methods suggested by the decision tree, those used in the literature are included for this type of problem. However, although it would be a very interesting exercise to compare the 29 methods with each other, it is beyond the scope of this article. We propose to use PROMETHEE II, which has been widely used in different industries: stock exchange assets, selecting electric vehicles, biology growth models, drainage models, to mention few, but it has not been used in IIoT platform selection [33,34,35]. The fact that this prodigious method has not yet been used in the field of IIoT encourages us to explore the use of this method, which will be novel. Furthermore, PROMETHEE methods are recognized as one of the best and most popular methods for outrank, allowing the experts who evaluate the alternatives to possibly not have complete and thorough knowledge of all the criteria and also allow them to express the importance of their preferences clearly [34]. These characteristics cover in a good way the aspect that the roles of experts involved in a decision of IIoT platform are multidisciplinary, having in several occasions roles that are not experts in the field of technology, but experts in another area, such as social or economic.

2. Materials and Methods

In our experience, companies that want to implement IIoT show great enthusiasm for the initiative, but, on several occasions, they have a misconception of what IIoT entails. IIoT concepts are technical and of great interest to engineers and systems architects, but the business factors, cost aspects, methods of payment, and commercial conditions, and all of them are of great interest for senior management represented by the Chief Officers, referred to often as the CxO Level. In addition, the wide offer that exists in the market where suppliers have different prices and service schemes makes it difficult to compare one between the other, or at least difficult to do a linear comparison.
Our proposal identifies and suggests the criteria required for IIoT Platform selection for a MCDA exercise with PROMETHEE-II method, enabling organizations to compare results and make a well-founded decision. This work does not provide a universal and definitive solution, but, rather, it proposes the methodology that any organization, be it small or large, can use to decide on the IIoT platform that best suits their circumstances and needs. Following the general MCDA process depicted in Figure 3, the decision objective is the selection of an IIoT platform.
The selection of criteria must be consistent with the decision and each criterion must be independent of one another. Each criterion must also be measured on the same scale and applicable to all alternatives. Table 3 summarizes the criteria to be used together with its definition. Criteria that are qualitative, i.e., based on expert judgement, can be measured by text to number scale. For calculating criteria weights, we propose to use the Analytic Hierarchy Process and the Saaty scale [24,27]. Criteria that are quantitative should consider equal scenarios, such as the cost of data transmission, which for all alternatives should be calculated with the same number of devices, same message size, and same number of messages per day.
The selected criteria are divided into three major areas of interest: technical, economic, and social. This is a major enhancement over previous works found in the literature. To identify to what area each criterion belongs, we use a relationship matrix, where we identify if the criterion has a high, medium, or low relationship with each of the areas. The selected criteria are also classified as quantitative and qualitative according to their nature, and are summarized in Table 3.
The existing alternatives for the IIoT platform considered in this paper appear in the literature, or are widely used in the industry and are recognized as market leaders of cloud providers, such Gartner’s Magic Quadrant. Figure 4 shows how in 2014 there were 15 competitors, while, in 2018, only six remained. However, it is easy to observe the leaders, dominated by AWS, Microsoft and the recently newcomer, Google. Thus, the alternatives included in this exercise are: AWS IoT Platform, Microsoft Azure IoT Platform, and Google Cloud IoT Platform. The alternatives and criteria is arranged in a matrix style, as shown in Table 4.
Our proposal includes profiles of people who must participate in the expert judgement exercise, something that has not been found in literature. It is important that they are not only dedicated to technology in order to enrich the exercise. The Table 5 lists the desirable profiles of people we suggest, who should be involved in a MCDA exercise as experts. It is important to note that not all roles must necessarily be participating, as these positions may vary between organizations.


Our proposed methodology, shown in Figure 5, consists of several tasks in order to found the best alternative. The first task (Activity 1) is to define a decision matrix, taking in consideration sub tasks. It is required to find the alternatives available in the market (Activity 1.1). A good source of information is to rely in recognized entities such as Gartner Consulting (Activity 1.1.1), which has been recognized as a trusted source of information to perform studies to find who are the leaders, challengers, niche players, and visionaries; other sources may be Forrester and IDC, but, for our study, we took Gartner. In next activity, criteria is defined (Activity 1.2) supported by elaborating a relationship matrix (Activity 1.2.1) as presented in Figure 6 using the defined criteria proposed in Table 3. It consists of fourteen items available, named C i , where i = 1 , 2 , , n , and n = 14 , arranged in the three main areas.
Each criterion was marked with the level of relationship it has with each group proposed in Low, Medium, and High. It may happen that a criterion has a high relationship with two or more groups. This indicates that the criterion could be classified in any group, or it needs to be broken down in finer criteria.
When evaluating the relationship each of the criteria proposed has with the three groups suggested, it is clear that the technical group will have {Available Regions, Communication Protocols, Device Management, Display, Managed Integration, Security, and Variety of Data Analytics}. The same treatment occurs for economic and social groups. The criterion having high and medium relationship could be argued to have a certain degree of impact in the related groups, but the highest relationship is taken to classify the criterion. It was found that there is no criterion with a high relationship in two or more groups. In addition, the relationship matrix suggests which group may have more impact during decisions, which has to be verified later. In this relationship matrix, the technical group is the one with the most elements (seven), then Economic group with four elements, and, finally, the social group got three elements.
The resulting decision matrix will have 14 criteria, grouped in three categories (technical, economic and social) with three alternatives presented in Table 4, as we are considering as feasible alternatives only the leaders from Figure 4. The structure of the criteria broken down into groups is presented in Figure 7.
Then, Activity 2 starts, where experts will need to grade each criterion in pairwise fashion, using Saaty scale [38] (Activity 2.1) for pairwise comparison (Table 6) to assign a level of importance of C i over C j . Experts’ answers are recorded in a square matrix x = [ n × n ] . Each element x i j will have a numeric value translated from Saaty scale and, as it is pairwise, the reciprocal x j i = 1 / x i j when i j ; when i = j , then x i j = 1 . In other words, x i j corresponds to the importance of C i over C j .
When designing the tool to grab expert’s answers, consider the number of pairwise comparisons required. These can be easily calculated by
N u m C o m p a r i s o n s = n 2 n 2
After having recorded all answers, it is required to calculate weights w, for each C i . To proceed, the matrix values need to first be normalized by obtaining the sum of each column and then dividing each cell by the sum of its corresponding column.
From this normalized matrix, criteria weights w are obtained by the sum on each row element i = 1 n x i j , when j = 1 , 2 , , n . However, it is important to verify if weights found are trustworthy and can be applied later. This is achieved by calculating the Consistency Ratio ( C R ). C R will measure how consistent the judgements are relative to a large sample of pure random judgements, known as Random Index ( R I ). When C R < 0.1 , then the weights are acceptable. In the case C R > 0.1 , it indicates that the judgements are untrustworthy because they are closer to random distribution and the exercise must be repeated. Random distribution, also known as Saaty random consistency index, is well documented by Saaty [38] and widely used in literature. As a reference, Table 7 shows values for RI, based on a number of criteria [39].
C R is found by
C R = C I R I
where C I is Consistency Index and R I is the Random Index. CI is calculated as
C I = λ m a x n n 1
It is required to multiply each value for its corresponding criteria weight and then sum each row to obtain a weighted sum value ( W S M ) . Then, each of this weighted sum values is divided by the corresponding criteria weight ( C W ). The result is a new column with λ i = W S M i C W i values.
To calculate λ m a x , just sum up the results of each λ and divide it by the number of rows in the matrix
λ m a x = i = 1 n ( λ i ) num of rows
If C R < 0.1 , then calculated weights are accepted (trustworthy) and experts can proceed to grade each alternative S k for each C i . We propose a qualitative criterion to use qualitative conversion from 1 to 5. Each word from low, below low, average, good, and excellent has a corresponding value, in this case {1, 2,3,4,5}.
Activity 3 consists of evaluating the alternatives using the decision matrix with the weights found and validated. It is required to define a criterion goal. They can be Maximize (also known as direct criteria, or beneficial criteria) or Minimize (also known as indirect criteria or non beneficial criteria). This goal setting is important as it will define the normalization method in Activity 4.
A quantitative criterion just needs to enter the value as it is found. For a qualitative criterion, the expert enters a perception of the criterion that in turn will be translated into a numeric value. We propose to use 1 to 5 values, as shown in Table 8.
After all decision matrix is evaluated, a PROMETHEE-II method can be applied. PROMTHEE-II stands for a Preference Ranking Organization Method for Enrichment Evaluations. Version I is just a partial ranking, reason enough not to use it in our methodology, while version II is a full ranking. PROMETHEE-II is an extensively documented method, and the reader can find information about this method in [40,41].
Finally, all alternatives are ranked, and the best option for the organization (Activity 5) can be obtained.

3. Results

Calculating weights, consistency, and selecting the best alternative can be difficult to follow. It is better to show an example. In our work, we follow our proposed methodology to obtain the best option to select an IIoT platform calculating the weighted criteria with the three platform vendors located in the leader quadrant from Gartner’s magic quadrant (Figure 4). Those are: AWS, Azure, and GCP.

3.1. Weight Criteria Calculation

The first step in our methodology says to calculate the weights required for platform selection. In order to achieve this, there are two things to do: (1) Weight calculation coming from experts judgement (participants came from Table 5) and (2) Validate consistency.
Each expert must answer how important is c r i t e r i o n i over c r i t e r i o n j . Using Saaty scale [38] for pairwise comparison (Table 6), experts can express the importance between two criteria. In our proposed methodology, each expert consulted should answer [ ( 14 2 ) 14 ] / 2 = 91 comparisons, as there are 14 criteria. This is 91 items.
By following criteria abbreviations proposed in Table 3, and having recorded experts’ judgement for each pairwise comparison, Table 9 shows the matrix with answers given.
We need to obtain the sum of each column. The sum of each column will be used to normalize Table 9 resulting in Table 10. Then, in Table 11 are shown the weighted values for all criteria.
To determine if weights are trustworthy, we calculated Consistency Index and Consistency ratio. In order to achieve this, calculation of weighted values need to be found by ( x i j × w i ) , as is shown in Table 12. The Table 13 shows the values obtained when calculating W V S , the ratio of each W V S w i , λ m a x and Equation (5) shows Consistency Index C I calculation.
Consistency Index in our experiment is calculated as
C I = λ m a x n n 1 = 15.37023 14 ( 14 1 ) = 0.105402
Using the random index for N = 14 from Table 7, Consistency ratio is computed as
C R = C I R I ( n ) = 0.105402 1.59 = 0.06671
As CR < 0.1, the weights for each criterion are consistent and trustworthy; therefore, they are accepted to use in our decision process.

3.2. IIoT Platform Selection

Among the three cloud platform vendors considered for this excercise: AWS, Azure, and Google Cloud Platform (GCP), listed in alphabetical order. Each vendor brings IIoT capacity, different services, and price schema not directly comparable among vendors. Each organization must have their goals, and will answer the weight criteria process differently, so it is not possible to determine which vendor is better than another in an absolute fashion. For that reason, this scenario is a good fit for our methodology.
Each alternative (let us call them S i ) needs to be graded on each of the criterion proposed. It is convenient to have it on a table, with criteria identified (in this case, we use abbreviations suggested in our methodology) and specify if criterion is qualitative, i.e., requires a numeric value contained in criterion domain, or it is qualitative and requires converting the appreciation of expert grading into a pre-established numeric value, as shown in Table 14.
For criterion, “Available regions (TAr), AWS has 22 available regions worldwide (, Azure offers 55 regions (, and GCP offers 21 ( Criterion Communication ports (TCp), AWS offers three options (HTTP, Websockets, MQTT), Azure offers four (HTTP, AMQP, MQTT, Websockets), and GCP offers two (HTTP, MQTT). Criterion Cost (EC) is the most cumbersome to compare and calculate. AWS uses a mix schema to estimate IIoT costs. Azure is based on messages, and GCP has a traffic consumption schema. As it can be seen, this is not comparable directly, so we estimated costs based on a same scenario for all three vendors.
The scenario consists of 1000 devices, sending a message of 8 Kb with a rate of two messages per minute. All estimations are per month. Our compared estimations using each vendor calculator are summarized in Table 15.
Training cost (ETc) takes into consideration the cost of certification, being AWS $150.00, Azure $165.00, and GCP $200.00 (at the time of writing this paper). The rest of the criteria are evaluated from a qualitative form. Table 16 contains the grades provided and M a x ( x i j ) and M i n ( x i j ) . In order to save space, we use S 1 as AWS, S 2 as Azure, and S 3 as GCP.
To normalize the table, we need to consider if we are maximizing or minimizing. The resulting normalized matrix is in Table 17. As a courtesy to the reader, we exemplify the operation using the first cell of the matrix. The operation executed to normalize values (Maximizing) is
X 1 , 1 M i n ( x i j ) M a x ( x i j ) M i n ( x i j ) = 22 21 55 21 = 0.023
For criterion looking for minimization, the equation changes, such as EC calculation (top row):
M a x ( x i j ) X 1 , 1 M a x ( x i j ) M i n ( x i j ) = 3037.5 138.82 3037.5 138.82 = 1
The next step is to calculate differences from normalized Table 17 using a pairwise comparison as shown in Table 18. The sample operation is
S 1 S 2 = ( 0.029 1 ) = 0.971
Next, calculate preference function values, resulting in Table 19. The operation is
P i ( a , b ) 0 then P i ( a , b ) = 0 ; 0.971 0 then = 0
Next, we calculate the weighted preferences, using preference function and weights found in Table 11. Each cell has the value w P ( a , b ) and results are in Table 20 by doing
w i P i ( a , b ) = 0.033 × 0 = 0
The aggregated preference is shown in Table 21.
Next, using the aggregated preference values, we calculate the entering and leaving flows. Table 22 has the arranged values; the right-most column contains the leaving flow ( φ + ), and the bottom row shows the entering flow ( φ ).
Leaving flow φ + and entering flow φ are calculated as follows:
φ + = 1 n 1 b = 1 n π ( a , b ) = ( 0.152428017 + 0.55062249 ) 3 1 = 0.351525254
φ = 1 n 1 b = 1 n π ( b , a ) = ( 0.44937751 + 0.107132361 ) 3 1 = 0.278254935
As we are using PROMETHEE-II, we need to calculate net flow Φ . The best way to do it is to build another table with each alternative and its corresponding leaving and entering flows. Add the column for net flow ( Φ = φ + φ ) and order the net flows from highest to lowest to rank all alternatives available. Table 23 shows the results.

4. Discussion

The methodology proposed to find the best alternative within a decision matrix, using all criteria, and applied to an example, finds the best solution. However, as part of this research, we decided to execute two validations. The first one uses the proposed methodology with criteria subsets. The second consists of running the full criteria (14 elements) with three different methods: TOPSIS, and its use has been reported in literature for similar problems, MOORA and Dimensional Analysis (DA), using the same alternatives and values in decision matrix.
Our proposed methodology with criteria subsets shows a good consistency in the alternative selected, except when we used five criteria. When use seven or ten criteria, the result is exactly the same, as shown in Table 24 and Figure 8.
In addition, we found that there is a change of index values when adding criteria. Figure 9 depicts how alternative AWS lowers when adding criteria, and alternative GCP increases. It can be observed also how alternative Azure remains not only as the best alternative, but also consistent in the index value.
Now, comparing TOPSIS, MOORA, and DA against our proposed methodology, the results are consistent, as all algorithms selected the same alternative with same number of criteria considered. Table 25 and Figure 10 show that all three other methods selected the same alternative as our methodology.
Becasue TOPSIS has been used in similar problems, we decided to do an additional comparison. By running TOPSIS against the same criteria subsets, we can observe that the selected alternative is the same for all cases, as shown in Table 26.
As it can be observed, when the number of criteria varies, only in one case, the one with fewest criteria subset, the result changes while the rest remains constant. This suggests that there should be a criteria subset that could provide the best selection option. We analyzed another set of scenarios, in order to identify the minimum criteria subset. To achieve this, it is interesting to take a look at resulting indexes, to identify: 1) where is the major gap among alternatives ranked, and 2) what is the trend by expanding the number of criteria. Ordering the criteria weights, it can be found that some criteria provide a very low percentage in the mix (we assume for every scenario ( w i ) = 1 ) (Table 27).
If all criteria weights were equally important (baseline), for each criterion, its deviation from that baseline is identified. Positive deviation means more importance, while a negative deviation means lower importance. By using this reasoning, we found six criteria candidates that could lead us to the minimum subset. Figure 11 shows the subset chosen {Security, Communication Protocols, Managed Integration, Device Management, and Display and Cost}.
To verify that this is a significant subset, this subset called “Top” is evaluated in PROMETHEE-II, and, to double check our selection, we add three more scenarios: top subset with equal weights (TopEq), bottom criteria (Bot), and bottom criteria with equal weights (BotEq). The results in Figure 12 show an extremely well performance selecting the alternative S 2 (Azure) with the largest separation from the other two alternatives, being AWS and GCP with negative numbers, in both Top and TopEq scenarios. In comparison with our full criteria set (weighted in the graph), Top and TopEq scenarios show the largest distance between alternative S 2 and S 1 (AWS). In addition, it can be observed that Bot and BotEq scenarios are very close each other, with less separation between S 2 and S 1 . Finally, the criteria subset labeled Top provides the highest rank index, suggesting that the selected subset is feasible to be the minimum set required.

5. Conclusions

As technology in IIoT and the cloud advances, there will be new options available in the market for the organizations. In addition, there are aspects that are relevant, not only technical, but economical and social. The three alternatives evaluated for this paper are aligned to leaders identified by Gartner up to 2018; however, it doesn’t assure they will be the only ones in the near-, mid- or long-term.
The criteria proposed follows and adapt for today’s vision. People must have double deep abilities—which are technical and business. This is one of the reasons to add to technical criteria the angles of economics and social view. As per our literature review, economics and social views have not been considered. Our contribution to industry provides these two missing aspects.
Cost is one of the most difficult and confusing comparisons, if there is not a good scenario to run against each price schema. However, as it is shown in Table 11, cost is not the main driver to take a decision in IIoT. Security has the highest weight and this is understandable as an organization’s IIoT implementations and solutions will transmit sensitive data. Communication protocols are the second most important criterion, and the reasoning behind is the flexibility required for different sensors available in the market. Device management and display are very close in importance, which is logical as organizations need to deploy from dozens to thousands of devices for a solution, and having a dashboard to locate and get information about devices is important.
Of economic and social criteria, the most significant are cost and available resources, respectively; longevity in the market was the least important criterion. This can be read as organizations possibly being open to experiments and learning with newcomers.
It is the best to have different experts from different backgrounds or responsibility within the organization. The roles suggested in this methodology (Table 5) cover a large part of main organization areas. We decided to include not only the IT department, but operations, financial, human resources, and business unit leaders. This proves to be aligned with the criteria suggested. By inviting the ability to participate in different roles, the weighting criteria become more accurate; therefore, the selection process will be better. We do not suggest to have a single expert to provide an opinion on criteria weighting. As people may have different understanding or could be biased towards a specific criteria, having more than one expert is preferred, and our proposed set of roles provides the options to select the experts.
Use of Saaty scale and method to evaluate criteria importance was proven to be effective. However, we discover that the validation of opinions is even more important, in order to provide trustworthy weights for the selection criteria. In our experiment, consistency ratio was 0.06, which is acceptable and allows for continuing with the process. Organizations must use these kinds of validations when choosing what would be more important over other criteria.
As it was discovered in the literature review (Table 2), most work related to cloud and IIoT has focused on AHP and TOPSIS. However, selecting an IIoT platform cannot have a single alternative winner; it is better to have all alternatives ranked. Our experience states, in some cases, that the vendor selected cannot deliver or does not meet other organizations’ requirements such as terms, legal contracts, conditions, or timing. When this happens, it would be a waste of time to redo the whole MCDA process again. This is why PROMETHEE-II has been proven to be effective as it can rank from top to bottom the alternatives available. In our exercise, Azure was the first option, followed by AWS and GCP.
It is important to notice that PROMETHEE-II and our methodology will not say which platform or technology is better, from an absolute standpoint, but which platform or technology is better suited for the organization based on the weights and grades provided by experts within the organization.
The paper demonstrated that our proposed methodology is effective at finding the best alternative to select an IIoT platform vendor as it has been performed consistently with five, seven, and ten criteria subsets, as well as comparing results against other methods. In addition, it contributes to the field of IIoT, as it provides a novel method to solve the problem many organizations are or will face at any time. Combining Saaty weight method and PROMEHEE-II, decision makers have a good tool to perform the selection. However, if it is limited to the technical aspects, the result may be biased and miss important aspects of the market. For example, if the technology is very good, the platform is the most complete and least expensive, but if there are not engineers or developers available, or training classes cost a fortune, implementing this platform will be a difficult and expensive project, with hidden costs not detected since inception. This is the reason and justification to include economic and social aspects in the criteria, as our methodology proposes.
IIoT platform selection should not be left to IT departments or CIO or CTO. Doing that will miss the point of view of other important leaders that will use, maintain or benefit from the selected platform. The Chief Operation Officer, leaders from business units, interdisciplinary teams, and even human resources and finance should participate in the MCDA process, as they bring ideas and considerations that sometimes are ignored unintentionally. Our proposed methodology provides a suggested list of key persons that should participate, something that was not found in the literature, and is very valuable for the decision process.
As a side discovery, comparing price schemes among vendors is not an easy task. We saw it as very useful to have a common scenario to run against the price schemes. To build a common scenario, it is required to have a close to reality idea of usage, number of devices, message size, and frequency of communication. Trying to compare price schemes without this scenario could lead to incorrect information entered into the grading matrix of the PROMETHEE-II part (Table 16).
The process of doing calculations and operations is laborious, due to the nature of algorithms used in our proposed methodology. This inspires us to continue the future work enhancing the methodology, creating a software to facilitate the computation. Another key aspect is the importance grading from Saaty’s process. Filling the matrix with reciprocal values could lead to human error easily. This also highlights, as part of our future work, to develop a graphical user interface that experts can use in a friendly fashion to enter the importance between criteria and fully automate our methodology when multiple experts participate in the process.
Future research work will focus on the fact that, by 2047, the year with the greatest incidence of a paradigm change in Generation Z in Industry 4.0, each tender that will require detailing the side effect of environmental impact can be carried out by an intelligent system using multi-criteria analysis to determine the best option for an alternative in a set of parts supplying resolution possibilities, where decision-making is decisive for its adequate solution, as can be seen in the following Figure 13.
The decision-making in this century will allow for extending in the Z generation to societies with a specific competitive value such as Bouganville, Brunei, Chuuk, East Timor, Rapa Nui, Sarawak, and Tuva that will have more symbolic capital with a combination of low population and diverse natural resources. Where manual work or traditional manufacturing will generate valuable cultural artifacts such as a French poodle made with balloons, and of which there will be no mass production, something that will be an avant-garde model for the Z generation and their descendants.
Finally, our future work will explore the use and implementation of other techniques to find the minimum criteria required to select the optimum IIoT platform, applying machine learning and data mining techniques. In addition, we plan to expand data acquisition from different experts around the globe in the roles identified previously. This is planned to be achieved by publishing a tool accessed via a web browser to collect the importance of each criterion in pairwise comparison.

Author Contributions

All of the authors jointly contributed to the finalization of the paper: R.C.-M. defined the criteria proposed and provided the MCDA options; A.O.-Z. supervised the overall process and provided resources; M.E.-C. directed the method of ranking; L.P.-D. reviewed the paper; R.C.-M. developed the methodology; V.G. critically reviewed the concept and design of the paper. All authors have read and agreed to the published version of the manuscript.


This research received no external funding

Conflicts of Interest

The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.


The following abbreviations are used in this manuscript:
IoTInternet of Things
IIoTIndustrial Internet of Things
MQTTMessage Queue Telemetry Transport
HTTPHypertext Transfer Protocol
AMQPAdvanced Message Queuing Protocol
S1Type of Azure IoT Hub
AWSAmazon Web Services
GCPGoogle Cloud Platform
MCDAMultiple Criteria Decision Analysis


  1. Rüßmann, M.; Lorenz, M.; Gerbert, P.; Waldner, M.; Justus, J.; Engel, P.; Harnisch, M. Industry 4.0: The future of productivity and growth in manufacturing industries. Boston Consult. Group 2015, 9, 54–89. [Google Scholar]
  2. Hatzivasilis, G.; Fysarakis, K.; Soultatos, O.; Askoxylakis, I.; Papaefstathiou, I.; Demetriou, G. The Industrial Internet of Things as an enabler for a Circular Economy Hy- LP: A novel IIoT protocol, evaluated on a wind park’s SDN/NFV-enabled 5G industrial network. Comput. Commun. 2018. [Google Scholar] [CrossRef]
  3. Höller, J.; Tsiatsis, V.; Mulligan, C.; Karnouskos, S.; Avesand, S.; Boyle, D. IoT Architecture—State of the Art. In From Machine-To-Machine to the Internet of Things; Academic Press: Oxford, UK, 2014. [Google Scholar] [CrossRef]
  4. Lanotte, R.; Merro, M. A semantic theory of the Internet of Things. Inf. Comput. 2018, 259, 72–101. [Google Scholar] [CrossRef] [Green Version]
  5. Rathore, M.M.; Ahmad, A.; Paul, A.; Rho, S. Urban planning and building smart cities based on the Internet of Things using Big Data analytics. Comput. Netw. 2016, 101, 63–80. [Google Scholar] [CrossRef]
  6. Figueira, J.; Greco, S.; Ehrgott, M. Multiple Criteria Decision analysis: State of the Art Surveys; Springer Science & Business Media: Berlin, Germany, 2005; Volume 78. [Google Scholar]
  7. Huang, I.B.; Keisler, J.; Linkov, I. Multi-criteria decision analysis in environmental sciences: Ten years of applications and trends. Sci. Total Environ. 2011, 409, 3578–3594. [Google Scholar] [CrossRef] [PubMed]
  8. Weyrich, M.; Ebert, C. Reference architectures for the internet of things. IEEE Softw. 2016, 33, 112–116. [Google Scholar] [CrossRef]
  9. Vasilomanolakis, E.; Daubert, J.; Luthra, M.; Gazis, V.; Wiesmaier, A.; Kikiras, P. On the Security and Privacy of Internet of Things Architectures and Systems. In Proceedings of the 2015 International Workshop on Secure Internet of Things, SIoT 2015, Vienna, Austria, 21–25 September 2015. [Google Scholar] [CrossRef] [Green Version]
  10. Gironés, T.; Canovas Solbes, J.; Parra-Boronat, A. An Integrated IoT Architecture for Smart Metering. IEEE Commun. Mag. 2016, 54, 50–57. [Google Scholar] [CrossRef]
  11. Krishnamurthy, R.; Cecil, J.; Perera, D. IMECE2017-72293 an Internet of Things (iot) Based Frameworks for Colloborative Manufacturing. In Proceedings of the International Mechanical Engineering Congress & Exposition, Tampa, FL, USA, 3–9 November 2017. [Google Scholar]
  12. Ray, P.P. A survey on Internet of Things architectures. J. King Saud Univ. Comput. Inf. Sci. 2018, 30, 291–319. [Google Scholar] [CrossRef] [Green Version]
  13. Ferreira, H.G.C.; Dias Canedo, E.; De Sousa, R.T. IoT architecture to enable intercommunication through REST API and UPnP using IP, ZigBee and arduino. In Proceedings of the International Conference on Wireless and Mobile Computing, Networking and Communications, Lyon, France, 7–9 Octpber 2013. [Google Scholar] [CrossRef]
  14. Gazis, V.; Goertz, M.; Huber, M.; Leonardi, A.; Mathioudakis, K.; Wiesmaier, A.; Zeiger, F. Short paper: IoT: Challenges, projects, architectures. In Proceedings of the IEEE 2015 18th International Conference on Intelligence in Next, Generation Networks, Paris, France, 17–19 Febuary 2015; pp. 145–147. [Google Scholar]
  15. Firdous, F.; Mohd Umair, M.; Alikhan Siddiqui, D.; Mohd Umair, A. IoT Based Home Automation System over the Cloud. Int. J. Innov. Adv. Comput. Sci. (IJIACS) 2018, 7, 512–517. [Google Scholar]
  16. Nitti, M.; Pilloni, V.; Giusto, D.; Popescu, V. IoT Architecture for a sustainable tourism application in a smart city environment. Mob. Inf. Syst. 2017. [Google Scholar] [CrossRef]
  17. Contreras-Castillo, J.; Zeadally, S.; Guerrero Ibáñez, J.A. A seven-layered model architecture for Internet of Vehicles. J. Inf. Telecommun. 2017. [Google Scholar] [CrossRef] [Green Version]
  18. Rahimi, H.; Zibaeenejad, A.; Safavi, A.A. A Novel IoT Architecture Based on 5G-IoT and Next, Generation Technologies. In Proceedings of the 2018 IEEE 9th Annual Information Technology, Electronics and Mobile Communication Conference (IEMCON), Vancouver, BC, Canada, 1–3 November 2018. [Google Scholar]
  19. Dumitru, R.L. IoT Platforms: Analysis for Building Projects. Inf. Econ. 2017. [Google Scholar] [CrossRef]
  20. Zanakis, S.H.; Solomon, A.; Wishart, N.; Dublish, S. Multi-attribute decision-making: A simulation comparison of select methods. Eur. J. Oper. Res. 1998, 107, 507–529. [Google Scholar] [CrossRef]
  21. Henig, M.I.; Buchanan, J.T. Solving MCDM problems: Process concepts. J. Multi-Criteria Decis. Anal. 1996, 5, 3–21. [Google Scholar] [CrossRef]
  22. Whaiduzzaman, M.; Gani, A.; Anuar, N.B.; Shiraz, M.; Haque, M.N.; Haque, I.T. Cloud service selection using multicriteria decision analysis. Sci. World J. 2014, 2014, 459375. [Google Scholar] [CrossRef]
  23. Watrobski, J.; Jankowski, J.; Pawel, Z.; Karczmarczyk, A.; Ziolo, M. Generalised framework for multi-criteria method selection. Omega 2018, 86, 107–124. [Google Scholar] [CrossRef]
  24. Silva, E.M.; Jardim-Goncalves, R. Multi-criteria analysis and decision methodology for the selection of internet-of-things hardware platforms. In Doctoral Conference on Computing, Electrical and Industrial Systems; Springer: Cham, Switzerland, 2017; pp. 111–121. [Google Scholar]
  25. Soltani, S.; Martin, P.; Elgazzar, K. A hybrid approach to automatic IaaS service selection. J. Cloud Comput. 2018, 7, 12. [Google Scholar] [CrossRef] [Green Version]
  26. Kondratenko, Y.; Kondratenko, G.; Sidenko, I. Multi-criteria decision-making for selecting a rational IoT platform. In Proceedings of the 2018 IEEE 9th International Conference on Dependable Systems, Services and Technologies (DESSERT), Kiev, Ukraine, 24–27 May 2018; pp. 147–152. [Google Scholar]
  27. Uslu, B.; Eren, T.; Gür, S.; Özcan, E. Evaluation of the Difficulties in the Internet of Things (IoT) with Multi-Criteria Decision-Making. Processes 2019, 7, 164. [Google Scholar] [CrossRef] [Green Version]
  28. Singla, C.; Mahajan, N.; Kaushal, S.; Verma, A.; Sangaiah, A.K. Modelling and Analysis of Multi-objective Service Selection Scheme in IoT-Cloud Environment. In Cognitive Computing for Big Data Systems Over IoT; Springer: Berlin, Germany, 2018; pp. 63–77. [Google Scholar]
  29. Alelaiwi, A. Evaluating distributed IoT databases for edge/cloud platforms using the analytic hierarchy process. J. Parallel Distrib. Comput. 2019, 124, 41–46. [Google Scholar] [CrossRef]
  30. Silva, E.M.; Agostinho, C.; Jardim-Goncalves, R. A multi-criteria decision model for the selection of a more suitable Internet-of-Things device. In Proceedings of the IEEE 2017 International Conference on Engineering, Technology and Innovation (ICE/ITMC), Funchal, Portugal, 27–29 June 2017; pp. 1268–1276. [Google Scholar]
  31. Garg, S.K.; Versteeg, S.; Buyya, R. A framework for ranking of cloud computing services. Future Gener. Comput. Syst. 2013, 29, 1012–1023. [Google Scholar] [CrossRef]
  32. Guth, J.; Breitenbucher, U.; Falkenthal, M.; Leymann, F.; Reinfurt, L. Comparison of IoT platform architectures: A field study based on a reference architecture. In Proceedings of the 2016 Cloudification of the Internet of Things, CIoT 2016, Paris, France, 23–25 November 2016. [Google Scholar] [CrossRef]
  33. Mitkova, V.; Mlynarovič, V. Investment Opportunities Identification Based on Macroeconomic Development, the Multiple Criteria Decision Approach. Symmetry 2019, 11, 827. [Google Scholar] [CrossRef] [Green Version]
  34. Wątróbski, J.; Małecki, K.; Kijewska, K.; Iwan, S.; Karczmarczyk, A.; Thompson, R.G. Multi-criteria analysis of electric vans for city logistics. Sustainability 2017, 9, 1453. [Google Scholar] [CrossRef] [Green Version]
  35. Palevičius, V.; Podviezko, A.; Sivilevičius, H.; Prentkovskis, O. Decision-aiding evaluation of public infrastructure for electric vehicles in cities and resorts of Lithuania. Sustainability 2018, 10, 904. [Google Scholar] [CrossRef] [Green Version]
  36. Google Cloud Platform Breaks into Leader Category in Gartner’s Magic Quadrant. Available online: (accessed on 28 January 2020).
  37. Amazon and Microsoft Top Gartner’s IaaS Magic Quadrant. Available online: (accessed on 28 January 2020).
  38. Saaty, T.L. Decision making with the analytic hierarchy process. Int. J. Serv. Sci. 2008, 1, 83–98. [Google Scholar] [CrossRef] [Green Version]
  39. Setiawan, A.; Sediyono, E.; Moekoe, D.A. Application of AHP method in determining priorities of conversion of unusedland to food land in Minahasa Tenggara. Int. J. Comput. Appl. 2014, 89, 8. [Google Scholar] [CrossRef]
  40. Vulević, T.; Dragović, N. Multi-criteria decision analysis for sub-watersheds ranking via the PROMETHEE method. Int. Soil Water Conserv. Res. 2017, 5, 50–55. [Google Scholar] [CrossRef]
  41. Brans, J.P.; De Smet, Y. PROMETHEE methods. In Multiple Criteria Decision Analysis; Springer: Berlin, Germany, 2016; pp. 187–219. [Google Scholar]
Figure 1. Conceptual model to select IIoT platforms.
Figure 1. Conceptual model to select IIoT platforms.
Symmetry 12 00368 g001
Figure 2. Taxonomy of MCDA (adapted from [22]).
Figure 2. Taxonomy of MCDA (adapted from [22]).
Symmetry 12 00368 g002
Figure 3. Process for multiple criteria analysis.
Figure 3. Process for multiple criteria analysis.
Symmetry 12 00368 g003
Figure 4. Gartner Cloud Providers Leaders Magic Quadrant 2014 vs. 2018 (adapted from [36,37], own creation).
Figure 4. Gartner Cloud Providers Leaders Magic Quadrant 2014 vs. 2018 (adapted from [36,37], own creation).
Symmetry 12 00368 g004
Figure 5. Methodology proposed to select an IIoT Platform.
Figure 5. Methodology proposed to select an IIoT Platform.
Symmetry 12 00368 g005
Figure 6. Relationship matrix to find the criteria and area belonging.
Figure 6. Relationship matrix to find the criteria and area belonging.
Symmetry 12 00368 g006
Figure 7. Criteria breakdown into groups.
Figure 7. Criteria breakdown into groups.
Symmetry 12 00368 g007
Figure 8. Comparison of results using different criteria subsets with the same methodology.
Figure 8. Comparison of results using different criteria subsets with the same methodology.
Symmetry 12 00368 g008
Figure 9. Comparison of resulting indexes in the proposed methodology.
Figure 9. Comparison of resulting indexes in the proposed methodology.
Symmetry 12 00368 g009
Figure 10. Comparing different methodologies against our proposed methodology.
Figure 10. Comparing different methodologies against our proposed methodology.
Symmetry 12 00368 g010
Figure 11. Importance comparison based on distance to baseline.
Figure 11. Importance comparison based on distance to baseline.
Symmetry 12 00368 g011
Figure 12. Comparison of results with different criteria subsets.
Figure 12. Comparison of results with different criteria subsets.
Symmetry 12 00368 g012
Figure 13. Conceptual diagram of an Intelligent Model that can adequately determine the best multi-criteria selection of a component supply model associated with Industry 4.0.
Figure 13. Conceptual diagram of an Intelligent Model that can adequately determine the best multi-criteria selection of a component supply model associated with Industry 4.0.
Symmetry 12 00368 g013
Table 1. IIoT architectures.
Table 1. IIoT architectures.
2Devices and Communication[9]
3Devices, Communication and Application[10,11,12]
4Devices, Communication, Transport and Application[9,12,13,14,15,16]
5Devices, Local processing, Communication, Transport and Applications[12]
7Business, Management, Communication, Processing, Acquisition, User
interaction and Security
8Physical devices, Communication, Edge or Fog processing, Data storage,
Applications, Collaboration and process, Security
Table 2. Previous work related to select technology.
Table 2. Previous work related to select technology.
2019IoT ChallengesAHP, ANPCommunication, Technology, Privacy and security,
Legal regulations, Culture
2018Cloud service for IoTFAHP, FTOPSISAvailability, Privacy, Capacity, Speed, Cost[28]
2018Platform IoTFuzzySecurity, Device management, Integration level,
Processing level, Database functionality, Data
collection protocols, Visualization, Analytics
2018IaaSTOPSISCost, Computing required, Storage capacity,
Operating system
2018Distributed IoT DatabasesAHPUsability, Prtability, Support[29]
2017IoT DeviceAHPEnergy consumption, Implementation time,
Difficulty of implementation, Cost, Clock device
2017IoT PlatformAHPEnergy, Cost, Computing speed, Data memory,
Program memory, device weight
2013Ranking cloud servicesAHPResponsibility, Agility, Service assurance, Cost,
Performance, Security and privacy, Usability
Analytic Hierarchy Process (AHP), Fuzzy Analytic Hierarchy Process (FAHP), Analytic Network Process (ANP), Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS), Fuzzy TOPSIS (FTOPSIS), Internet of Things (IoT), Infrastructure as a Service (IaaS).
Table 3. Criteria for IIoT Platform selection process.
Table 3. Criteria for IIoT Platform selection process.
(Qualitative (Q) or Quantitative (C).
All are maximization except when noted
Minimization (min) )
Available region
In cloud-based solutions, it is important to identify the
regions where the provider is present and that are suited
to the geographical situation of the industry.
Managed Integration
The platform has the ability to offer an integration engine
with services and applications.
IoT devices can communicate telemetry and receive messages
with different protocols such as HTTP, MQTT, AMQP, CoAP,
or even private.
The security of the platform must include security for the
transmission, registration of devices, avoiding apocryphal
devices, authentication and authorization, preferably from
start to finish.
Device Management
Devices that can be connected, device identification, device
monitoring, send software updates to devices and specify
alert conditions. The digital twin refers to the digital replica
of the physical asset.
It allows that the data and the behavior of the devices can
be seen by humans. It is better if a native and customizable
dashboard is offered to show the relevant data to each person.
Variety of Data
The data collected must be analyzed in different ways. It is
important to consider the data flow, real-time analysis, batch,
and machine learning algorithms available on the platform.
Longevity in market
Years that the provider has in the market. It is expected that
the reputation of a supplier will increase over the years.
Calculate the monthly cost (30 days average) for the devices
that will be connected. Use constant message size and the
frequency of constant message sending.
Free Cost
The providers offer a free amount of messages that are
subtracted from the monthly consumption.
Training Cost
Providers can offer access to training with cost or free, and
staff certification plans.
Community support
Informative resources about the platform, including the
available documentation of the provider and external
resources of the expert community (blogs, tutorials,
discussion forums, etc.)
Available Resources
Availability of human resources with expert knowledge in
the platform.
Providers offer training and certifications, which can be
complicated to follow and hinder the learning curve. One
measure may be the estimated time to complete the courses
and certifications.
Table 4. Our resulting decision matrix (activity 1).
Table 4. Our resulting decision matrix (activity 1).
AlternativeCriterion C 1 Criterion C 2 Criterion C 1 4
AWS ( S 1 )
Azure ( S 2 )
GCP ( S 3 )
Table 5. Roles involved in the IIoT Platform selection.
Table 5. Roles involved in the IIoT Platform selection.
CIOChief Information Officer
In terms usually is, it is the most important person responsible
for technology in any company. Their tasks range from buying IT
equipment to directing the workforce to the use of technology.
T, E, S
CTOChief Technology Officer
The technology director reports to the CIO, which means that it acts
as support for IIoT projects. That said, in larger organizations, the
work may be too much for just one person, so the CTO has this
CInOChief Innovation Officer
This role has been recently created and is the one that can counteract
the wild instinct oriented to sales of the business units of a
company and design an organizational environment more favorable
to innovation.
T, S
CSOChief Security Officer
He is the main person responsible for the information security
program of an organization and should be consulted before any
deployment of technology.
COOChief Operations Officer
Oversees the business operations of an organization and work to
create an operations strategy and communicate it to employees. He is
very involved in the day to day of the company and will be one of the
main impacted in an IIoT project.
CMOChief Marketing Officer
The technology and the business aspects of the company are
converging. This convergence of technology and marketing reflects
the need for the traditional Commercial Director to adapt to a digital
world and, therefore, participates in any IIoT project in which they are
working, to express their opinion so as to obtain commercial benefit
for the company.
CFOChief Financial Officer
In all the projects of the company, there must be the support of the
Finance Director, who controls the economic resources of the company.
In an IIoT project, he is interested in the investment required, and
especially in the return of investment to exercise.
HROHuman Resources Officer
It is the person who needs to know if the necessary skills to the
project exist in the market, how easy it is to obtain them, and the
sources where they can be obtained. Among his responsibilities
are the personnel development plans and the recruitment of human
BULBusiness Unit Leaders
The deputy directors and managers who report within each hierarchy
are key personnel that can provide good opinions and issue a more
tactical than strategic judgement. By being more focused on specific
projects, their knowledge and sensitivity also become specific, giving
value to expert judgements.
T, E, S
Table 6. Saaty scale for pairwise comparison (adapted from [38]).
Table 6. Saaty scale for pairwise comparison (adapted from [38]).
Intensity of ImportanceDefinitionExplanation
1Equal importanceTwo elements contribute equally to the objective
3Moderate importanceExperience and judgement slightly favor one element
over another
5Strong importanceExperience and judgement strongly favor one element
over another
7Very strong importanceOne element is favored very strongly over another, its
dominance is demonstrated in practice
9Extreme importanceThe evidence favoring one activity over another is of
the highest possible order of affirmation
2, 4, 6, 8Intermediate valuesImportance between above and below value
Table 7. Random index [39].
Table 7. Random index [39].
NRandom Index (RI)
Table 8. Perception to value.
Table 8. Perception to value.
Below Average2
Table 9. Expert’s judgement pairwise comparison recorded.
Table 9. Expert’s judgement pairwise comparison recorded.
TAr1 1 / 2 1/21/51/21/51/221/211/21/21/21/2
x i j
Table 10. Normalized matrix.
Table 10. Normalized matrix.
Table 11. Weights w i calculated for each criterion.
Table 11. Weights w i calculated for each criterion.
Criterion C i Weight Calculated w i
Table 12. Computed weighted values.
Table 12. Computed weighted values.
Table 13. Computed consistency.
Table 13. Computed consistency.
Criterion C i Weight Value ∑ (WVS)Ratio WVS / w i
λ m a x = 15.37023
Table 14. Pre-define values for qualitative labels.
Table 14. Pre-define values for qualitative labels.
Qualitative LabelPre-Defined Value
Below Avg2
Table 15. Cost estimations by vendor.
Table 15. Cost estimations by vendor.
$ 3.46 Connectivity
$86.40 of messaging
$36.00 device shadow
$ 4.32 rules triggered
$ 8.64 rules actions
2880 meessages/device
2,880,000 msg/day
S1 node provides 400,00
unlimited access
Need 8 X S1 nodes
675,000 MB/month
Total Cost: $138.32Total Cost: $180.00Total Cost: $3,037.50
Table 16. Graded alternatives.
Table 16. Graded alternatives.
S 1 AWS224353455138.823150454
S 2 Azure555444554182.535165553
S 3 GCP2133335433037.54200333
M a x ( x i j ) 5554545553037.55200554
M i n ( x i j ) 213333443138.823150333
Table 17. Normalized table.
Table 17. Normalized table.
S 1 0.0290.50100111010.511
S 2 1110.51110.50.98510.7110
S 3 0000010000.50000
Table 18. Calculated differences from normalized matrix.
Table 18. Calculated differences from normalized matrix.
S 1 S 2 −0.971−0.5−10.5−1−100.50.015−10.3−0.501
S 1 S 3 0.0290.5010−1111−0.510.511
S 2 S 1 0.9710.51−0.5110−0.5−0.0151−0.30.50−1
S 2 S 3 1110.51010.50.9850.50.7110
S 3 S 1 −0.029−0.50−101−1−1−10.5−1−0.5−1−1
S 3 S 2 −1−1−1−0.5−10−1−0.5−0.985−0.5−0.7−1−10
Table 19. Preference function computation results.
Table 19. Preference function computation results.
S 1 S 2 0000.50000.50.015100.3001
S 1 S 3 0.0230.50100111010.511
S 2 S 1 0.9710.51011000100.500
S 2 S 3 1110.51010.50.9850.50.7110
S 3 S 1 0000010000.50000
S 3 S 2 00000000000000
Table 20. Weighted preferences.
Table 20. Weighted preferences.
w i 0.0330.0990.1290.1600.0950.0920.0660.0250.0830.0310.0420.0430.0560.046
S 1 S 2 0.0000.0000.0000.0800.0000.0000.0000.0130.0010.0000.0130.0000.0000.046
S 1 S 3 0.0010.0500.0000.1600.0000.0000.0660.0250.0830.0000.0420.0220.0560.046
S 2 S 1 0.0320.0500.1290.0000.0950.0920.0000.0000.0000.0310.0000.0220.0000.000
S 2 S 3 0.0330.0990.1290.0800.0950.0000.0660.0130.0820.0150.0290.0430.0560.000
S 3 S 1 0.0000.0000.0000.0000.0000.0920.0000.0000.0000.0150.0000.0000.0000.000
S 3 S 2 0.0000.0000.0000.0000.0000.0000.0000.0000.0000.0000.0000.0000.0000.000
Table 21. Aggregated preference.
Table 21. Aggregated preference.
S q S b π ( a , b )
S 1 S 2 0.152428017
S 1 S 3 0.55062249
S 2 S 1 0.44937751
S 2 S 3 0.740439621
S 3 S 1 0.107132361
S 3 S 2 0
Table 22. Entering and leaving flows.
Table 22. Entering and leaving flows.
AWSAzureGCP φ +
AWS 0.1524280170.550622490.351525254
Azure0.44937751 0.7404396210.594908565
GCP0.1071323610 0.053566181
φ 0.2782549350.0762140090.645531056
Table 23. Ranking of alternatives.
Table 23. Ranking of alternatives.
Leaving flow φ + Entering flow φ Net Flow Φ Rank
Table 24. Ranking with our proposed methodology with criteria subsets (1 is highest).
Table 24. Ranking with our proposed methodology with criteria subsets (1 is highest).
5 Criteria7 Criteria10 CriteriaFull Criteria (14)
Table 25. Proposed methodology validation with three more algorithms using full criteria.
Table 25. Proposed methodology validation with three more algorithms using full criteria.
Table 26. Ranking with our proposed methodology with criteria subsets (1 is highest).
Table 26. Ranking with our proposed methodology with criteria subsets (1 is highest).
5 Criteria7 Criteria10 CriteriaFull Criteria (14)
Table 27. Ordered weighted preferences.
Table 27. Ordered weighted preferences.

Share and Cite

MDPI and ACS Style

Contreras-Masse, R.; Ochoa-Zezzatti, A.; García, V.; Pérez-Dominguez, L.; Elizondo-Cortés, M. Implementing a Novel Use of Multicriteria Decision Analysis to Select IIoT Platforms for Smart Manufacturing. Symmetry 2020, 12, 368.

AMA Style

Contreras-Masse R, Ochoa-Zezzatti A, García V, Pérez-Dominguez L, Elizondo-Cortés M. Implementing a Novel Use of Multicriteria Decision Analysis to Select IIoT Platforms for Smart Manufacturing. Symmetry. 2020; 12(3):368.

Chicago/Turabian Style

Contreras-Masse, Roberto, Alberto Ochoa-Zezzatti, Vicente García, Luis Pérez-Dominguez, and Mayra Elizondo-Cortés. 2020. "Implementing a Novel Use of Multicriteria Decision Analysis to Select IIoT Platforms for Smart Manufacturing" Symmetry 12, no. 3: 368.

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop