Implementing a Novel Use of Multicriteria Decision Analysis to Select IIoT Platforms for Smart Manufacturing

Abstract

Industry 4.0 is having a great impact in all smart efforts. This is not a single product but is composed of several technologies, one of them being Industrial Internet of Things (IIoT). Currently, there are very varied implementation options offered by several companies, and this imposes a new challenge to companies that want to implement IIoT in their processes. This challenge suggests using multi-criteria analysis to make a repeatable and justified decision, requiring a set of alternatives and criteria. This paper proposes a new methodology and comprehensive criteria to help organizations to take an educated decision by applying multi-criteria analysis. Here, we suggest a new original use of PROMETHEE-II with a full example from weight calculation up to IIoT platform selection, showing this methodology as an effective study for other organizations interested in selecting an IIoT platform. The criteria proposed stands out from previous work by including not only technical aspects, but economic and social criteria, providing a full view of the problem analyzed. A case of study was used to prove this proposed methodology and finds the minimum subset to reach the best possible ranking.

1. Introduction

Industry 4.0 is having a high impact in all industries. This is not a unique product, but it is composed of several technologies. Boston Consulting Group has defined nine technological pillars for Industry 4.0: cloud, additive manufacturing, simulation, big data and analysis, autonomous robots, augmented reality, integration of horizontal and vertical systems, cybersecurity and industrial internet of things (IIOT) [1]. IIOT has been used not only in the manufacturing industry, but has expanded to other industries such as health, travel and transportation, energy, gas and oil, etc. This is one of the main reasons that IIOT is known as the Internet of Things (IoT) [2]. IIoT is a key intelligent factor that allows factories to act intelligently. By adding sensors and actuators to objects, the object becomes intelligent because it can interact with people, other objects, generate data, generate transactions, and react to environmental data [3,4]. Cities do not ignore this trend, since there is a plan to turn cities into smart cities in certain countries [5].

The decision processes that companies must follow should be supported by methods that consider pros and cons of plural points of view that affect the decision process. Researchers and practitioners have developed over time the techniques that today are part of the domain of Multiple Criteria Decision Analysis (MCDA), which, very simplistically, requires three basic elements: a finite set of actions or alternatives, at least two criteria, and at least one decision making method. [6]. The MCDA has been the object of study and nowadays there are a lot of methods for decision-making in disciplines such as waste management, industrial engineering, strategies, manufacturing, even natural resource management and environmental impact [7].The purpose of this manuscript is precisely to propose a method of MCDA with the corresponding criteria for the selection of an IIoT(Industria Internet of Things) platform, which can serve as a starting point to companies and individuals embarked on implementation projects of Industry 4.0. Our conceptual model to solve the problem is shown in Figure 1.

Figure 1. Conceptual model to select IIoT platforms.

1.1. Literature Review

Industria Internet of Things (IIoT) continues to evolve. Due to the instrinsic complexity, it is good practice to look at architectural references. IIoT have five main requirements on a general basis [8]: (1) Enable communication and connectivity between devices and data processing; (2) Establish a mechanism to manage devices, including tasks such as adding or deleting devices, updating software and configurations; (3) Gather all the data produced by the devices and then analyze them to provide a meaningful perspective to the companies or users; (4) Facilitate scalability to handle the increased flow of “data pipes” (hereinafter referred to as data pipelines) and the flow of data, and handle an increasing number of devices; (5) Protect the data by adding the necessary functions to provide privacy and trust between the devices and the users. Table 1 shows the summary of the various multi-layer architectures found in the literature.

Table 1. IIoT architectures.

Num.	Layers	References
2	Devices and Communication	[9]
3	Devices, Communication and Application	[10,11,12]
4	Devices, Communication, Transport and Application	[9,12,13,14,15,16]
5	Devices, Local processing, Communication, Transport and Applications	[12]
7	Business, Management, Communication, Processing, Acquisition, User interaction and Security	[15,17]
8	Physical devices, Communication, Edge or Fog processing, Data storage, Applications, Collaboration and process, Security	[18]

Technical architecture provides an extreme value to users because it can be implemented with different products. Therefore, it is understandable that several companies offer IIoT platforms that can be useful for our architectures. Commercial providers aim for flexible options offered, and consumers are responsible for using each component in the best way they consider. The main commercial players identified are, in alphabetical order: Amazon Web Services, Bosch IoT Suite, Google Cloud Platform, IBM Blue Mix (now Watson IoT), Microsoft Azure IoT, and Oracle Integrated Cloud [19]. The leading players identified in 2014 by Gartner Group were AWS and Microsoft, but, in 2018, Google enters the leaders quadrant. IBM, for its commercial relevance, is considered, although it has become a niche player, along with Oracle. Although Bosch IoT does not appear in the panorama detected by Gartner, we include it for being used in several industries. Each of these suppliers has similar characteristics among them but have different value propositions.

1.1.1. MCDA as a Tool to Select an IIoT Platform

Making a decision introduces problems to individuals. One of the problems is the integration of heterogeneous data and the uncertainty factor surrounding a decision, and the criteria that usually conflict with each other [7,20]. To carry out a MCDA process, a series of tasks is proposed, based on the three generic steps suggested by [21]: (i) identify the objective or goal, (ii) select the criteria, parameters, factors, and attributes, (iii) selection of alternatives, (iv) association of attributes with the criteria, (v) selection of weight methods to represent the importance of each criterion, and (vi) the method of aggregation. Ref. [21] included a step that is left out of these proposed tasks, but which should be considered in the discussion before executing the selected action. This step is to understand and compare the preferences of the person making the decision.

The MCDA can be classified according to the basis of the problem, by type, by category, or by the methods used to make the analysis. Figure 2 shows a taxonomy adapted from [22]; the methods included in this taxonomy are not exhaustive. The MCDA is a collection of systematic methodologies for comparisons, classification, and selection of multiple alternatives, each one with multiple attributes and is dependent on an evaluation matrix. Generally, it used to detect and quantify the decisions and considerations from interested parties (stakeholders) about various monetary factors and non-monetary factors to compare an alternative course of action [7,22]. The major division that exists in MCDA lies in the category of methodologies. First, the group considers discrete values with a limited number of known alternatives that involve some compensation or trade-off. This group is called Multiple Attributes Decision Making (MADM). The other group is the Multiple Objectives Decision Making (MODM), and its variable decision values are within a continuous domain with infinite or very numerous options that satisfy the restrictions, preferences, or priorities [20]. In addition, there is another classification according to the way of adding criteria, and it is divided into the American school, which aggregates into a single criterion, and into the European or French school that uses outranking methods. It can be considered a mixture of both schools and they are indirect approaches, such as the Peer Criteria Comparison methods (PCCA) [23].

Figure 2. Taxonomy of MCDA (adapted from [22]).

1.1.2. Use of MCDA to Select IIoT Platforms or Technology Platforms—Related Work

When finding the available alternatives of the market, a new question will arise to find the method that helps to select the appropriate option. To answer this last question, a review of the literature is made looking for: (a) MCDA methods applied to the selection of IIoT platforms and (b) knowing the criteria taken into account.

In the literature, there is little information on the subject in recent years. Table 2 shows the summary of the work found. The selected methods are focused on AHP, TOPSIS, and Fuzzy logic in AHP and TOPSIS. The outranking methods were not implemented but were considered as an option or for future work by some authors [24,25]. The selection of an IIoT platform is neither dominated by a single criterion nor is there a single alternative. Ref. [26] considered AWS, Azure, Bosch, IBM Watson, and Google Cloud within their options, which coincide with some of the alternatives considered in this manuscript. Therefore, it is interesting to review the criteria they included for MCDA, as summarized in Table 2.

Table 2. Previous work related to select technology.

Year	Application	MCDA	Criteria	Ref.
2019	IoT Challenges	AHP, ANP	Communication, Technology, Privacy and security, Legal regulations, Culture	[27]
2018	Cloud service for IoT	FAHP, FTOPSIS	Availability, Privacy, Capacity, Speed, Cost	[28]
2018	Platform IoT	Fuzzy	Security, Device management, Integration level, Processing level, Database functionality, Data collection protocols, Visualization, Analytics variety	[26]
2018	IaaS	TOPSIS	Cost, Computing required, Storage capacity, Operating system	[25]
2018	Distributed IoT Databases	AHP	Usability, Prtability, Support	[29]
2017	IoT Device	AHP	Energy consumption, Implementation time, Difficulty of implementation, Cost, Clock device	[24]
2017	IoT Platform	AHP	Energy, Cost, Computing speed, Data memory, Program memory, device weight	[30]
2013	Ranking cloud services	AHP	Responsibility, Agility, Service assurance, Cost, Performance, Security and privacy, Usability	[31]

Analytic Hierarchy Process (AHP), Fuzzy Analytic Hierarchy Process (FAHP), Analytic Network Process (ANP), Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS), Fuzzy TOPSIS (FTOPSIS), Internet of Things (IoT), Infrastructure as a Service (IaaS).

Criteria found in literature are purely technical with some hints of economy, and can be found as part of the characteristics of IIoT architecture [32]. However, when implementing an IIoT platform, non-technical aspects should also be considered. As the platform to be considered has its foundation in the cloud, it is valid to review the criteria included in previous MCDA exercises to select a cloud provider, looking for non-technical aspects.

The criteria for selecting a cloud proposed in the CSMIC Framework v 2.1 of 2014 (Cloud Services Measurement Initiative Consortium (CSMIC) was created by Carnegie Mellon University to develop Service Measurement Index (SMI). it can be found at https://spark.adobe.com/page/PN39b/) as the Index of Measure of Service (SMI) includes topics of interest to the organization, financial, and usability, together with the technical issues [31]. Some of these criteria can be included to complement the analysis having the technical point of view and the business point of view.

Finally, there is a question about which methods are suitable for these types of problems, noting that the previous work includes AHP, ANP, TOPSIS, and Fuzzy Logic, but they are left aside for future research methods such as PROMETHEE and ELECTRE. There are many more methods available in MCDA scope. Following the decision tree to select an MCDA method written by [23], which considers 56 methods, the number of options can be easily reduced. In the case of selecting an IIoT platform that has different criteria, the problem has the characteristics of classification or ranking, ordering the options from best to worst. This technique is useful in real life, since they hardly conform and subject themselves to a single option, but they have to consider their primary option and another option as backup, assuming that the first option is not viable.

The candidate methods found are COMET, NAIADE II, EVAMIX, MAUT, MAVT, SAW, SMART, TOPSIS, UTA, VIKOR, Fuzzy SAW, Fuzzy TOPSIS, Fuzzy VIKOR, PROMETHEE I, PAMSSEM II, Fuzzy PROMETHEE II, AHP + TOPSIS, AHP + VIKOR, fuzzy AHP + TOPSIS, AHP + Fuzzy TOPSIS, Fuzzy ANP + Fuzzy TOPSIS, AHP, ANP, MACBETH, DEMATEL, REMBRANDT, Fuzzy AHP, and Fuzzy ANP.

Of the 29 methods suggested by the decision tree, those used in the literature are included for this type of problem. However, although it would be a very interesting exercise to compare the 29 methods with each other, it is beyond the scope of this article. We propose to use PROMETHEE II, which has been widely used in different industries: stock exchange assets, selecting electric vehicles, biology growth models, drainage models, to mention few, but it has not been used in IIoT platform selection [33,34,35]. The fact that this prodigious method has not yet been used in the field of IIoT encourages us to explore the use of this method, which will be novel. Furthermore, PROMETHEE methods are recognized as one of the best and most popular methods for outrank, allowing the experts who evaluate the alternatives to possibly not have complete and thorough knowledge of all the criteria and also allow them to express the importance of their preferences clearly [34]. These characteristics cover in a good way the aspect that the roles of experts involved in a decision of IIoT platform are multidisciplinary, having in several occasions roles that are not experts in the field of technology, but experts in another area, such as social or economic.

2. Materials and Methods

In our experience, companies that want to implement IIoT show great enthusiasm for the initiative, but, on several occasions, they have a misconception of what IIoT entails. IIoT concepts are technical and of great interest to engineers and systems architects, but the business factors, cost aspects, methods of payment, and commercial conditions, and all of them are of great interest for senior management represented by the Chief Officers, referred to often as the CxO Level. In addition, the wide offer that exists in the market where suppliers have different prices and service schemes makes it difficult to compare one between the other, or at least difficult to do a linear comparison.

Our proposal identifies and suggests the criteria required for IIoT Platform selection for a MCDA exercise with PROMETHEE-II method, enabling organizations to compare results and make a well-founded decision. This work does not provide a universal and definitive solution, but, rather, it proposes the methodology that any organization, be it small or large, can use to decide on the IIoT platform that best suits their circumstances and needs. Following the general MCDA process depicted in Figure 3, the decision objective is the selection of an IIoT platform.

Figure 3. Process for multiple criteria analysis.

The selection of criteria must be consistent with the decision and each criterion must be independent of one another. Each criterion must also be measured on the same scale and applicable to all alternatives. Table 3 summarizes the criteria to be used together with its definition. Criteria that are qualitative, i.e., based on expert judgement, can be measured by text to number scale. For calculating criteria weights, we propose to use the Analytic Hierarchy Process and the Saaty scale [24,27]. Criteria that are quantitative should consider equal scenarios, such as the cost of data transmission, which for all alternatives should be calculated with the same number of devices, same message size, and same number of messages per day.

Table 3. Criteria for IIoT Platform selection process.

Area	Criterion (Abbreviation)	Definition (Qualitative (Q) or Quantitative (C). All are maximization except when noted Minimization (min) )	Type
Technical (T)	Available region (TAr)	In cloud-based solutions, it is important to identify the regions where the provider is present and that are suited to the geographical situation of the industry.	C
	Managed Integration (TMi)	The platform has the ability to offer an integration engine with services and applications.	Q
	Communication Protocols (TCp)	IoT devices can communicate telemetry and receive messages with different protocols such as HTTP, MQTT, AMQP, CoAP, or even private.	C
	Security (TS)	The security of the platform must include security for the transmission, registration of devices, avoiding apocryphal devices, authentication and authorization, preferably from start to finish.	Q
	Device Management (TDm)	Devices that can be connected, device identification, device monitoring, send software updates to devices and specify alert conditions. The digital twin refers to the digital replica of the physical asset.	Q
	Display (TD)	It allows that the data and the behavior of the devices can be seen by humans. It is better if a native and customizable dashboard is offered to show the relevant data to each person.	Q
	Variety of Data Analytics (TAi)	The data collected must be analyzed in different ways. It is important to consider the data flow, real-time analysis, batch, and machine learning algorithms available on the platform.	Q
	Longevity in market (EM)	Years that the provider has in the market. It is expected that the reputation of a supplier will increase over the years.	Q
Economic (E)	Cost (EC)	Calculate the monthly cost (30 days average) for the devices that will be connected. Use constant message size and the frequency of constant message sending.	C(Min)
	Free Cost (EFc)	The providers offer a free amount of messages that are subtracted from the monthly consumption.	Q
	Training Cost (ETc)	Providers can offer access to training with cost or free, and staff certification plans.	C(Min)
	Community support (SCs)	Informative resources about the platform, including the available documentation of the provider and external resources of the expert community (blogs, tutorials, discussion forums, etc.)	Q
Social (S)	Available Resources (SHr)	Availability of human resources with expert knowledge in the platform.	Q
	Training (ST)	Providers offer training and certifications, which can be complicated to follow and hinder the learning curve. One measure may be the estimated time to complete the courses and certifications.	C

The selected criteria are divided into three major areas of interest: technical, economic, and social. This is a major enhancement over previous works found in the literature. To identify to what area each criterion belongs, we use a relationship matrix, where we identify if the criterion has a high, medium, or low relationship with each of the areas. The selected criteria are also classified as quantitative and qualitative according to their nature, and are summarized in Table 3.

The existing alternatives for the IIoT platform considered in this paper appear in the literature, or are widely used in the industry and are recognized as market leaders of cloud providers, such Gartner’s Magic Quadrant. Figure 4 shows how in 2014 there were 15 competitors, while, in 2018, only six remained. However, it is easy to observe the leaders, dominated by AWS, Microsoft and the recently newcomer, Google. Thus, the alternatives included in this exercise are: AWS IoT Platform, Microsoft Azure IoT Platform, and Google Cloud IoT Platform. The alternatives and criteria is arranged in a matrix style, as shown in Table 4.

Figure 4. Gartner Cloud Providers Leaders Magic Quadrant 2014 vs. 2018 (adapted from [36,37], own creation).

Table 4. Our resulting decision matrix (activity 1).

Alternative	Criterion ${\mathit{C}}_1$	Criterion ${\mathit{C}}_2$	…	Criterion ${\mathit{C}}_{1}4$
AWS ($S_1$)
Azure ($S_2$)
GCP ($S_3$)

Our proposal includes profiles of people who must participate in the expert judgement exercise, something that has not been found in literature. It is important that they are not only dedicated to technology in order to enrich the exercise. The Table 5 lists the desirable profiles of people we suggest, who should be involved in a MCDA exercise as experts. It is important to note that not all roles must necessarily be participating, as these positions may vary between organizations.

Table 5. Roles involved in the IIoT Platform selection.

Role	Description	Interest
CIO	Chief Information Officer In terms usually is, it is the most important person responsible for technology in any company. Their tasks range from buying IT equipment to directing the workforce to the use of technology.	T, E, S
CTO	Chief Technology Officer The technology director reports to the CIO, which means that it acts as support for IIoT projects. That said, in larger organizations, the work may be too much for just one person, so the CTO has this responsibility.	T
CInO	Chief Innovation Officer This role has been recently created and is the one that can counteract the wild instinct oriented to sales of the business units of a company and design an organizational environment more favorable to innovation.	T, S
CSO	Chief Security Officer He is the main person responsible for the information security program of an organization and should be consulted before any deployment of technology.	T
COO	Chief Operations Officer Oversees the business operations of an organization and work to create an operations strategy and communicate it to employees. He is very involved in the day to day of the company and will be one of the main impacted in an IIoT project.	E
CMO	Chief Marketing Officer The technology and the business aspects of the company are converging. This convergence of technology and marketing reflects the need for the traditional Commercial Director to adapt to a digital world and, therefore, participates in any IIoT project in which they are working, to express their opinion so as to obtain commercial benefit for the company.	E
CFO	Chief Financial Officer In all the projects of the company, there must be the support of the Finance Director, who controls the economic resources of the company. In an IIoT project, he is interested in the investment required, and especially in the return of investment to exercise.	E
HRO	Human Resources Officer It is the person who needs to know if the necessary skills to the project exist in the market, how easy it is to obtain them, and the sources where they can be obtained. Among his responsibilities are the personnel development plans and the recruitment of human resources.	S
BUL	Business Unit Leaders The deputy directors and managers who report within each hierarchy are key personnel that can provide good opinions and issue a more tactical than strategic judgement. By being more focused on specific projects, their knowledge and sensitivity also become specific, giving value to expert judgements.	T, E, S

Methods

Our proposed methodology, shown in Figure 5, consists of several tasks in order to found the best alternative. The first task (Activity 1) is to define a decision matrix, taking in consideration sub tasks. It is required to find the alternatives available in the market (Activity 1.1). A good source of information is to rely in recognized entities such as Gartner Consulting (Activity 1.1.1), which has been recognized as a trusted source of information to perform studies to find who are the leaders, challengers, niche players, and visionaries; other sources may be Forrester and IDC, but, for our study, we took Gartner. In next activity, criteria is defined (Activity 1.2) supported by elaborating a relationship matrix (Activity 1.2.1) as presented in Figure 6 using the defined criteria proposed in Table 3. It consists of fourteen items available, named $C_i$, where $i=1,2,\dots ,n$, and $n=14$, arranged in the three main areas.

Figure 5. Methodology proposed to select an IIoT Platform.

Figure 6. Relationship matrix to find the criteria and area belonging.

Each criterion was marked with the level of relationship it has with each group proposed in Low, Medium, and High. It may happen that a criterion has a high relationship with two or more groups. This indicates that the criterion could be classified in any group, or it needs to be broken down in finer criteria.

When evaluating the relationship each of the criteria proposed has with the three groups suggested, it is clear that the technical group will have {Available Regions, Communication Protocols, Device Management, Display, Managed Integration, Security, and Variety of Data Analytics}. The same treatment occurs for economic and social groups. The criterion having high and medium relationship could be argued to have a certain degree of impact in the related groups, but the highest relationship is taken to classify the criterion. It was found that there is no criterion with a high relationship in two or more groups. In addition, the relationship matrix suggests which group may have more impact during decisions, which has to be verified later. In this relationship matrix, the technical group is the one with the most elements (seven), then Economic group with four elements, and, finally, the social group got three elements.

The resulting decision matrix will have 14 criteria, grouped in three categories (technical, economic and social) with three alternatives presented in Table 4, as we are considering as feasible alternatives only the leaders from Figure 4. The structure of the criteria broken down into groups is presented in Figure 7.

Figure 7. Criteria breakdown into groups.

Then, Activity 2 starts, where experts will need to grade each criterion in pairwise fashion, using Saaty scale [38] (Activity 2.1) for pairwise comparison (Table 6) to assign a level of importance of $C_i$ over $C_j$. Experts’ answers are recorded in a square matrix $x=[n\times n]$. Each element $x_{ij}$ will have a numeric value translated from Saaty scale and, as it is pairwise, the reciprocal $x_{ji}=1/x_{ij}$ when $i\ne j$; when $i=j$, then $x_{ij}=1$. In other words, $x_{ij}$ corresponds to the importance of $C_i$ over $C_j$.

Table 6. Saaty scale for pairwise comparison (adapted from [38]).

Intensity of Importance	Definition	Explanation
1	Equal importance	Two elements contribute equally to the objective
3	Moderate importance	Experience and judgement slightly favor one element over another
5	Strong importance	Experience and judgement strongly favor one element over another
7	Very strong importance	One element is favored very strongly over another, its dominance is demonstrated in practice
9	Extreme importance	The evidence favoring one activity over another is of the highest possible order of affirmation
2, 4, 6, 8	Intermediate values	Importance between above and below value

When designing the tool to grab expert’s answers, consider the number of pairwise comparisons required. These can be easily calculated by

$$NumComparisons=\frac{n^2-n}{2}$$

(1)

After having recorded all answers, it is required to calculate weights w, for each $C_i$. To proceed, the matrix values need to first be normalized by obtaining the sum of each column and then dividing each cell by the sum of its corresponding column.

From this normalized matrix, criteria weights w are obtained by the sum on each row element ${\sum }_{i=1}^n{x}_{ij}$, when $j=1,2,\dots ,n$. However, it is important to verify if weights found are trustworthy and can be applied later. This is achieved by calculating the Consistency Ratio ($CR$). $CR$ will measure how consistent the judgements are relative to a large sample of pure random judgements, known as Random Index ($RI$). When $CR<0.1$, then the weights are acceptable. In the case $CR>0.1$, it indicates that the judgements are untrustworthy because they are closer to random distribution and the exercise must be repeated. Random distribution, also known as Saaty random consistency index, is well documented by Saaty [38] and widely used in literature. As a reference, Table 7 shows values for RI, based on a number of criteria [39].

Table 7. Random index [39].

N	Random Index (RI)
1	0.00
2	0.00
3	0.58
4	0.90
5	1.12
6	1.24
7	1.32
8	1.41
9	1.45
10	1.49
11	1.51
12	1.48
13	1.56
14	1.57
15	1.59

$CR$ is found by

$$CR=\frac{CI}{RI}$$

(2)

where $CI$ is Consistency Index and $RI$ is the Random Index. CI is calculated as

$$CI=\frac{{\lambda }_{max}-n}{n-1}$$

(3)

It is required to multiply each value for its corresponding criteria weight and then sum each row to obtain a weighted sum value ($WSM)$. Then, each of this weighted sum values is divided by the corresponding criteria weight ($CW$). The result is a new column with ${\lambda }_i=\frac{WS{M}_i}{C{W}_i}$ values.

To calculate ${\lambda }_{max}$, just sum up the results of each $\lambda$ and divide it by the number of rows in the matrix

$${\lambda }_{max}=\frac{{\sum }_{i=1}^n\left({\lambda }_i\right)}{\mathrm{num}\mathrm{of}\mathrm{rows}}$$

(4)

If $CR<0.1$, then calculated weights are accepted (trustworthy) and experts can proceed to grade each alternative $S_k$ for each $C_i$. We propose a qualitative criterion to use qualitative conversion from 1 to 5. Each word from low, below low, average, good, and excellent has a corresponding value, in this case {1, 2,3,4,5}.

Activity 3 consists of evaluating the alternatives using the decision matrix with the weights found and validated. It is required to define a criterion goal. They can be Maximize (also known as direct criteria, or beneficial criteria) or Minimize (also known as indirect criteria or non beneficial criteria). This goal setting is important as it will define the normalization method in Activity 4.

A quantitative criterion just needs to enter the value as it is found. For a qualitative criterion, the expert enters a perception of the criterion that in turn will be translated into a numeric value. We propose to use 1 to 5 values, as shown in Table 8.

Table 8. Perception to value.

Perception	Value
Excellent	5
Good	4
Average	3
Below Average	2
Low	1

After all decision matrix is evaluated, a PROMETHEE-II method can be applied. PROMTHEE-II stands for a Preference Ranking Organization Method for Enrichment Evaluations. Version I is just a partial ranking, reason enough not to use it in our methodology, while version II is a full ranking. PROMETHEE-II is an extensively documented method, and the reader can find information about this method in [40,41].

Finally, all alternatives are ranked, and the best option for the organization (Activity 5) can be obtained.

3. Results

Calculating weights, consistency, and selecting the best alternative can be difficult to follow. It is better to show an example. In our work, we follow our proposed methodology to obtain the best option to select an IIoT platform calculating the weighted criteria with the three platform vendors located in the leader quadrant from Gartner’s magic quadrant (Figure 4). Those are: AWS, Azure, and GCP.

3.1. Weight Criteria Calculation

The first step in our methodology says to calculate the weights required for platform selection. In order to achieve this, there are two things to do: (1) Weight calculation coming from experts judgement (participants came from Table 5) and (2) Validate consistency.

Each expert must answer how important is $criterio{n}_i$ over $criterio{n}_j$. Using Saaty scale [38] for pairwise comparison (Table 6), experts can express the importance between two criteria. In our proposed methodology, each expert consulted should answer $[\left({14}^2\right)-14]/2=91$ comparisons, as there are 14 criteria. This is 91 items.

By following criteria abbreviations proposed in Table 3, and having recorded experts’ judgement for each pairwise comparison, Table 9 shows the matrix with answers given.

Table 9. Expert’s judgement pairwise comparison recorded.

	TAr	TMi	TCp	TS	TDm	D	TAi	EM	EC	EFc	ETc	SCs	SHr	ST
TAr	1	$1/2$	1/2	1/5	1/2	1/5	1/2	2	1/2	1	1/2	1/2	1/2	1/2
TMi	2	1	1	1	1	1	1	5	1	3	3	5	1	3
TCp	2	1	1	1	3	1	1	5	1	3	5	5	3	5
TS	5	1	1	1	1	5	5	2	1	5	3	5	5	5
TDm	2	1	1/3	1	1	3	3	3	1	3	3	2	1	1
D	5	1	1	1/5	1/3	1	1	3	1	3	4	3	2	3
TAi	2	1	1	1/5	1/3	1	1	3	1	2	1	2	1	2
EM	1/2	1/5	1/5	1/2	1/3	1/3	1/3	1	1/2	1	1/3	1/3	1/3	1/3
EC	2	1	1	1	1	1	1	2	1	1	2	2	2	3
EFc	1	1/3	1/3	1/5	1/3	1/3	1/2	1	1	1	1/2	1/3	1/2	1/3
ETc	2	1/3	1/5	1/3	1/3	1/55	1	3	1/2	2	1	1	1/2	1
SCs	2	1/5	1/5	1/5	1/2	1/3	1/2	3	1/2	3	1	1	1	1
SHr	2	1	1/3	1/5	1	1/2	1	3	1/2	2	2	1	1	1
ST	2	1/3	1/5	1/5	1	1/3	1/2	3	1/3	3	1	1	1	1
$\mathbf{\sum }{\mathit{x}}_{\mathit{i}\mathit{j}}$	30.5	9.9	8.3	7.23	11.67	15.283	17.33	39	10.83	33	27.33	29.167	19.83	27.167

We need to obtain the sum of each column. The sum of each column will be used to normalize Table 9 resulting in Table 10. Then, in Table 11 are shown the weighted values for all criteria.

Table 10. Normalized matrix.

	TAr	TMi	TCp	TS	TDm	TD	TAi	EM	EC	EFc	ETc	SCs	SHr	ST
TAr	0.033	0.051	0.060	0.028	0.043	0.013	0.029	0.051	0.046	0.030	0.018	0.017	0.025	0.018
TMi	0.066	0.101	0.120	0.138	0.086	0.065	0.058	0.128	0.092	0.091	0.110	0.171	0.050	0.110
TCp	0.066	0.101	0.120	0.138	0.257	0.065	0.058	0.128	0.092	0.091	0.183	0.171	0.151	0.184
TS	0.164	0.101	0.120	0.138	0.086	0.327	0.288	0.051	0.092	0.152	0.110	0.171	0.252	0.184
TDm	0.066	0.101	0.040	0.138	0.086	0.196	0.173	0.077	0.092	0.091	0.110	0.069	0.050	0.037
TD	0.164	0.101	0.120	0.028	0.029	0.065	0.058	0.077	0.092	0.091	0.146	0.103	0.101	0.110
TAi	0.066	0.101	0.120	0.028	0.029	0.065	0.058	0.077	0.092	0.061	0.037	0.069	0.050	0.074
EM	0.016	0.020	0.024	0.069	0.029	0.022	0.019	0.026	0.046	0.030	0.012	0.011	0.017	0.012
EC	0.066	0.101	0.120	0.138	0.086	0.065	0.058	0.051	0.092	0.030	0.073	0.069	0.101	0.110
EFc	0.033	0.034	0.040	0.028	0.029	0.022	0.029	0.026	0.092	0.030	0.018	0.011	0.025	0.012
ETc	0.066	0.034	0.024	0.046	0.029	0.016	0.058	0.077	0.046	0.061	0.037	0.034	0.025	0.037
SCs	0.066	0.020	0.024	0.028	0.043	0.022	0.029	0.077	0.046	0.091	0.037	0.034	0.050	0.037
SHr	0.066	0.101	0.040	0.028	0.086	0.033	0.058	0.077	0.046	0.061	0.073	0.034	0.050	0.037
ST	0.066	0.034	0.024	0.028	0.086	0.022	0.029	0.077	0.031	0.091	0.037	0.034	0.050	0.037

Table 11. Weights $w_i$ calculated for each criterion.

Criterion ${\mathit{C}}_{\mathit{i}}$	Weight Calculated ${\mathit{w}}_{\mathit{i}}$
TAr	0.033054398
TMi	0.099114871
TCp	0.129047676
TS	0.159817455
TDm	0.094698157
TD	0.091812783
TAi	0.066103106
EM	0.025301927
EC	0.082932622
EFc	0.030639156
ETc	0.042044181
SCs	0.043080184
SHr	0.056348976
ST	0.046004508

To determine if weights are trustworthy, we calculated Consistency Index and Consistency ratio. In order to achieve this, calculation of weighted values need to be found by $(x_{ij}\times w_i)$, as is shown in Table 12. The Table 13 shows the values obtained when calculating $WVS$, the ratio of each $\frac{WVS}{w_i}$, ${\lambda }_{max}$ and Equation (5) shows Consistency Index $CI$ calculation.

Table 12. Computed weighted values.

	TAr	TMi	TCp	TS	TDm	TD	TAi	EM	EC	EFc	ETc	SCs	SHr	ST
TAr	0.033	0.050	0.065	0.032	0.047	0.018	0.033	0.051	0.041	0.031	0.021	0.022	0.028	0.023
TMi	0.066	0.099	0.129	0.160	0.095	0.092	0.066	0.127	0.083	0.092	0.126	0.215	0.056	0.138
TCp	0.066	0.099	0.129	0.160	0.284	0.092	0.066	0.127	0.083	0.092	0.210	0.215	0.169	0.230
TS	0.165	0.099	0.129	0.160	0.095	0.459	0.331	0.051	0.083	0.153	0.126	0.215	0.282	0.230
TDm	0.066	0.099	0.043	0.160	0.095	0.275	0.198	0.076	0.083	0.092	0.126	0.086	0.056	0.046
TD	0.165	0.099	0.129	0.032	0.032	0.092	0.066	0.076	0.083	0.092	0.168	0.129	0.113	0.138
TAi	0.066	0.099	0.129	0.032	0.032	0.092	0.066	0.076	0.083	0.061	0.042	0.086	0.056	0.092
EM	0.017	0.020	0.026	0.080	0.032	0.031	0.022	0.025	0.041	0.031	0.014	0.014	0.019	0.015
EC	0.066	0.099	0.129	0.160	0.095	0.092	0.066	0.051	0.083	0.031	0.084	0.086	0.113	0.138
EFc	0.033	0.033	0.043	0.032	0.032	0.031	0.033	0.025	0.083	0.031	0.021	0.014	0.028	0.015
ETc	0.066	0.033	0.026	0.053	0.032	0.023	0.066	0.076	0.041	0.061	0.042	0.043	0.028	0.046
SCs	0.066	0.020	0.026	0.032	0.047	0.031	0.033	0.076	0.041	0.092	0.042	0.043	0.056	0.046
SHr	0.066	0.099	0.043	0.032	0.095	0.046	0.066	0.076	0.041	0.061	0.084	0.043	0.056	0.046
ST	0.066	0.033	0.026	0.032	0.095	0.031	0.033	0.076	0.028	0.092	0.042	0.043	0.056	0.046

Table 13. Computed consistency.

Criterion ${\mathit{C}}_{\mathit{i}}$	Weight Value ∑ (WVS)	Ratio $\mathbf{WVS}/{\mathit{w}}_{\mathit{i}}$
TAr	0.494310596	14.95445755
TMi	1.543958531	15.57746603
TCp	2.022150174	15.66979151
TS	2.577562734	16.12816779
TDm	1.501905104	15.85991904
TD	1.413764592	15.39834154
TAi	1.012396031	15.31540793
EM	0.386173129	15.26259755
EC	1.291838682	15.57696654
EFc	0.454059119	14.81956987
ETc	0.636805226	15.14609676
SCs	0.651477099	15.12243086
SHr	0.855083138	15.17477683
ST	0.69821939	15.17719502
		${\lambda }_{max}$ = 15.37023

Consistency Index in our experiment is calculated as

$$CI=\frac{{\lambda }_{max}-n}{n-1}=\frac{15.37023-14}{(14-1)}=0.105402$$

(5)

Using the random index for $N=14$ from Table 7, Consistency ratio is computed as

$$CR=\frac{CI}{RI\left(n\right)}=\frac{0.105402}{1.59}=0.06671$$

(6)

As CR < 0.1, the weights for each criterion are consistent and trustworthy; therefore, they are accepted to use in our decision process.

3.2. IIoT Platform Selection

Among the three cloud platform vendors considered for this excercise: AWS, Azure, and Google Cloud Platform (GCP), listed in alphabetical order. Each vendor brings IIoT capacity, different services, and price schema not directly comparable among vendors. Each organization must have their goals, and will answer the weight criteria process differently, so it is not possible to determine which vendor is better than another in an absolute fashion. For that reason, this scenario is a good fit for our methodology.

Each alternative (let us call them $S_i$) needs to be graded on each of the criterion proposed. It is convenient to have it on a table, with criteria identified (in this case, we use abbreviations suggested in our methodology) and specify if criterion is qualitative, i.e., requires a numeric value contained in criterion domain, or it is qualitative and requires converting the appreciation of expert grading into a pre-established numeric value, as shown in Table 14.

Table 14. Pre-define values for qualitative labels.

Qualitative Label	Pre-Defined Value
Low	1
Below Avg	2
Average	3
Good	4
Excellent	5

For criterion, “Available regions (TAr), AWS has 22 available regions worldwide (https://aws.amazon.com/about-aws/global-infrastructure/?p=ngi&loc=1), Azure offers 55 regions (https://azure.microsoft.com/en-us/global-infrastructure/regions/), and GCP offers 21 (https://cloud.google.com/about/locations/). Criterion Communication ports (TCp), AWS offers three options (HTTP, Websockets, MQTT), Azure offers four (HTTP, AMQP, MQTT, Websockets), and GCP offers two (HTTP, MQTT). Criterion Cost (EC) is the most cumbersome to compare and calculate. AWS uses a mix schema to estimate IIoT costs. Azure is based on messages, and GCP has a traffic consumption schema. As it can be seen, this is not comparable directly, so we estimated costs based on a same scenario for all three vendors.

The scenario consists of 1000 devices, sending a message of 8 Kb with a rate of two messages per minute. All estimations are per month. Our compared estimations using each vendor calculator are summarized in Table 15.

Table 15. Cost estimations by vendor.

AWS	Azure	GCP
$ 3.46 Connectivity $86.40 of messaging $36.00 device shadow $ 4.32 rules triggered $ 8.64 rules actions	2880 meessages/device 2,880,000 msg/day S1 node provides 400,00 msg/day unlimited access Need 8 X S1 nodes	675,000 MB/month $0.0045/MB
Total Cost: $138.32	Total Cost: $180.00	Total Cost: $3,037.50

Training cost (ETc) takes into consideration the cost of certification, being AWS $150.00, Azure $165.00, and GCP $200.00 (at the time of writing this paper). The rest of the criteria are evaluated from a qualitative form. Table 16 contains the grades provided and $Max\left(x_{ij}\right)$ and $Min\left(x_{ij}\right)$. In order to save space, we use $S_1$ as AWS, $S_2$ as Azure, and $S_3$ as GCP.

Table 16. Graded alternatives.

${\mathit{S}}_{\mathit{i}}$	TAr	TMi	TCp	TS	TDm	TD	TAi	EM	EC	EFc	ETc	SCs	SHr	ST
$S_1$ AWS	22	4	3	5	3	4	5	5	138.82	3	150	4	5	4
$S_2$ Azure	55	5	4	4	4	5	5	4	182.53	5	165	5	5	3
$S_3$ GCP	21	3	3	3	3	5	4	3	3037.5	4	200	3	3	3
$Max\left(x_{ij}\right)$	55	5	4	5	4	5	5	5	3037.5	5	200	5	5	4
$Min\left(x_{ij}\right)$	21	3	3	3	3	4	4	3	138.82	3	150	3	3	3

To normalize the table, we need to consider if we are maximizing or minimizing. The resulting normalized matrix is in Table 17. As a courtesy to the reader, we exemplify the operation using the first cell of the matrix. The operation executed to normalize values (Maximizing) is

$$\frac{X_{1,1}-Min\left(x_{ij}\right)}{Max\left(x_{ij}\right)-Min\left(x_{ij}\right)}=\frac{22-21}{55-21}=0.023$$

Table 17. Normalized table.

${\mathit{S}}_{\mathit{i}}$	TAr	TMi	TCp	TS	TDm	TD	TAi	EM	EC	EFc	ETc	SCs	SHr	ST
$S_1$	0.029	0.5	0	1	0	0	1	1	1	0	1	0.5	1	1
$S_2$	1	1	1	0.5	1	1	1	0.5	0.985	1	0.7	1	1	0
$S_3$	0	0	0	0	0	1	0	0	0	0.5	0	0	0	0

For criterion looking for minimization, the equation changes, such as EC calculation (top row):

$$\frac{Max\left(x_{ij}\right)-X_{1,1}}{Max\left(x_{ij}\right)-Min\left(x_{ij}\right)}=\frac{3037.5-138.82}{3037.5-138.82}=1$$

The next step is to calculate differences from normalized Table 17 using a pairwise comparison as shown in Table 18. The sample operation is

$$S_1-S_2=(0.029-1)=-0.971$$

Table 18. Calculated differences from normalized matrix.

${\mathit{S}}_{\mathit{a}}-{\mathit{S}}_{\mathit{b}}$	TAr	TMi	TCp	TS	TDm	TD	TAi	EM	EC	EFc	ETc	SCs	SHr	ST
$S_1-S_2$	−0.971	−0.5	−1	0.5	−1	−1	0	0.5	0.015	−1	0.3	−0.5	0	1
$S_1-S_3$	0.029	0.5	0	1	0	−1	1	1	1	−0.5	1	0.5	1	1
$S_2-S_1$	0.971	0.5	1	−0.5	1	1	0	−0.5	−0.015	1	−0.3	0.5	0	−1
$S_2-S_3$	1	1	1	0.5	1	0	1	0.5	0.985	0.5	0.7	1	1	0
$S_3-S_1$	−0.029	−0.5	0	−1	0	1	−1	−1	−1	0.5	−1	−0.5	−1	−1
$S_3-S_2$	−1	−1	−1	−0.5	−1	0	−1	−0.5	−0.985	−0.5	−0.7	−1	−1	0

Next, calculate preference function values, resulting in Table 19. The operation is

$$P_i(a,b)\le 0\mathrm{then}{P}_i(a,b)=0;-0.971\le 0\mathrm{then}=0$$

Table 19. Preference function computation results.

${\mathit{S}}_{\mathit{a}}-{\mathit{S}}_{\mathit{b}}$	TAr	TMi	TCp	TS	TDm	TD	TAi	EM	EC	EFc	ETc	SCs	SHr	ST
$S_1-S_2$	0	0	0	0.5	0	0	0	0.5	0.0151	0	0.3	0	0	1
$S_1-S_3$	0.023	0.5	0	1	0	0	1	1	1	0	1	0.5	1	1
$S_2-S_1$	0.971	0.5	1	0	1	1	0	0	0	1	0	0.5	0	0
$S_2-S_3$	1	1	1	0.5	1	0	1	0.5	0.985	0.5	0.7	1	1	0
$S_3-S_1$	0	0	0	0	0	1	0	0	0	0.5	0	0	0	0
$S_3-S_2$	0	0	0	0	0	0	0	0	0	0	0	0	0	0

Next, we calculate the weighted preferences, using preference function and weights found in Table 11. Each cell has the value $wP(a,b)$ and results are in Table 20 by doing

$$w_i{P}_i(a,b)=0.033\times 0=0$$

Table 20. Weighted preferences.

	TAr	TMi	TCp	TS	TDm	TD	TAi	EM	EC	EFc	ETc	SCs	SHr	ST
$w_i$	0.033	0.099	0.129	0.160	0.095	0.092	0.066	0.025	0.083	0.031	0.042	0.043	0.056	0.046
$S_1-S_2$	0.000	0.000	0.000	0.080	0.000	0.000	0.000	0.013	0.001	0.000	0.013	0.000	0.000	0.046
$S_1-S_3$	0.001	0.050	0.000	0.160	0.000	0.000	0.066	0.025	0.083	0.000	0.042	0.022	0.056	0.046
$S_2-S_1$	0.032	0.050	0.129	0.000	0.095	0.092	0.000	0.000	0.000	0.031	0.000	0.022	0.000	0.000
$S_2-S_3$	0.033	0.099	0.129	0.080	0.095	0.000	0.066	0.013	0.082	0.015	0.029	0.043	0.056	0.000
$S_3-S_1$	0.000	0.000	0.000	0.000	0.000	0.092	0.000	0.000	0.000	0.015	0.000	0.000	0.000	0.000
$S_3-S_2$	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.000

The aggregated preference is shown in Table 21.

Table 21. Aggregated preference.

${\mathit{S}}_{\mathit{q}}-{\mathit{S}}_{\mathit{b}}$	$\mathit{\pi }(\mathit{a},\mathit{b})$
$S_1-S_2$	0.152428017
$S_1-S_3$	0.55062249
$S_2-S_1$	0.44937751
$S_2-S_3$	0.740439621
$S_3-S_1$	0.107132361
$S_3-S_2$	0

Next, using the aggregated preference values, we calculate the entering and leaving flows. Table 22 has the arranged values; the right-most column contains the leaving flow (${\phi }^{+}$), and the bottom row shows the entering flow (${\phi }^{-}$).

Table 22. Entering and leaving flows.

	AWS	Azure	GCP	${\mathit{\phi }}^{+}$
AWS		0.152428017	0.55062249	0.351525254
Azure	0.44937751		0.740439621	0.594908565
GCP	0.107132361	0		0.053566181
${\phi }^{-}$	0.278254935	0.076214009	0.645531056

Leaving flow ${\phi }^{+}$ and entering flow ${\phi }^{-}$ are calculated as follows:

$${\phi }^{+}=\frac{1}{n-1}\sum _{b=1}^n\pi (a,b)=\frac{(0.152428017+0.55062249)}{3-1}=0.351525254$$

$${\phi }^{-}=\frac{1}{n-1}\sum _{b=1}^n\pi (b,a)=\frac{(0.44937751+0.107132361)}{3-1}=0.278254935$$

As we are using PROMETHEE-II, we need to calculate net flow $\Phi$. The best way to do it is to build another table with each alternative and its corresponding leaving and entering flows. Add the column for net flow ($\Phi ={\phi }^{+}-{\phi }^{-}$) and order the net flows from highest to lowest to rank all alternatives available. Table 23 shows the results.

Table 23. Ranking of alternatives.

	Leaving flow ${\mathit{\phi }}^{+}$	Entering flow ${\mathit{\phi }}^{-}$	Net Flow $\Phi$	Rank
AWS	0.351525254	0.278254935	0.073270318	2
Azure	0.594908565	0.076214009	0.518694557	1
GCP	0.053566181	0.645531056	−0.591964875	3

4. Discussion

The methodology proposed to find the best alternative within a decision matrix, using all criteria, and applied to an example, finds the best solution. However, as part of this research, we decided to execute two validations. The first one uses the proposed methodology with criteria subsets. The second consists of running the full criteria (14 elements) with three different methods: TOPSIS, and its use has been reported in literature for similar problems, MOORA and Dimensional Analysis (DA), using the same alternatives and values in decision matrix.

Our proposed methodology with criteria subsets shows a good consistency in the alternative selected, except when we used five criteria. When use seven or ten criteria, the result is exactly the same, as shown in Table 24 and Figure 8.

Table 24. Ranking with our proposed methodology with criteria subsets (1 is highest).

	5 Criteria	7 Criteria	10 Criteria	Full Criteria (14)
AWS	1	2	2	2
Azure	2	1	1	1
GCP	3	3	3	3

Figure 8. Comparison of results using different criteria subsets with the same methodology.

In addition, we found that there is a change of index values when adding criteria. Figure 9 depicts how alternative AWS lowers when adding criteria, and alternative GCP increases. It can be observed also how alternative Azure remains not only as the best alternative, but also consistent in the index value.

Figure 9. Comparison of resulting indexes in the proposed methodology.

Now, comparing TOPSIS, MOORA, and DA against our proposed methodology, the results are consistent, as all algorithms selected the same alternative with same number of criteria considered. Table 25 and Figure 10 show that all three other methods selected the same alternative as our methodology.

Table 25. Proposed methodology validation with three more algorithms using full criteria.

	Ours	TOPSIS	MOORA	AD
AWS	2	2	2	3
Azure	1	1	1	1
GCP	3	3	3	2

Figure 10. Comparing different methodologies against our proposed methodology.

Becasue TOPSIS has been used in similar problems, we decided to do an additional comparison. By running TOPSIS against the same criteria subsets, we can observe that the selected alternative is the same for all cases, as shown in Table 26.

Table 26. Ranking with our proposed methodology with criteria subsets (1 is highest).

	5 Criteria	7 Criteria	10 Criteria	Full Criteria (14)
AWS	1	2	2	2
Azure	2	1	1	1
GCP	3	3	3	3

As it can be observed, when the number of criteria varies, only in one case, the one with fewest criteria subset, the result changes while the rest remains constant. This suggests that there should be a criteria subset that could provide the best selection option. We analyzed another set of scenarios, in order to identify the minimum criteria subset. To achieve this, it is interesting to take a look at resulting indexes, to identify: 1) where is the major gap among alternatives ranked, and 2) what is the trend by expanding the number of criteria. Ordering the criteria weights, it can be found that some criteria provide a very low percentage in the mix (we assume for every scenario $\sum \left(w_i\right)=1$) (Table 27).

Table 27. Ordered weighted preferences.

TS	TCp	TMi	TDm	TD	EC	TAi
0.1598	0.1290	0.0991	0.0947	0.0918	0.0829	0.0661
SHr	ST	SCs	ETc	TAr	EFc	EM
0.0563	0.0460	0.0431	0.0420	0.0331	0.0306	0.0253

If all criteria weights were equally important (baseline), for each criterion, its deviation from that baseline is identified. Positive deviation means more importance, while a negative deviation means lower importance. By using this reasoning, we found six criteria candidates that could lead us to the minimum subset. Figure 11 shows the subset chosen {Security, Communication Protocols, Managed Integration, Device Management, and Display and Cost}.

Figure 11. Importance comparison based on distance to baseline.

To verify that this is a significant subset, this subset called “Top” is evaluated in PROMETHEE-II, and, to double check our selection, we add three more scenarios: top subset with equal weights (TopEq), bottom criteria (Bot), and bottom criteria with equal weights (BotEq). The results in Figure 12 show an extremely well performance selecting the alternative $S_2$ (Azure) with the largest separation from the other two alternatives, being AWS and GCP with negative numbers, in both Top and TopEq scenarios. In comparison with our full criteria set (weighted in the graph), Top and TopEq scenarios show the largest distance between alternative $S_2$ and $S_1$ (AWS). In addition, it can be observed that Bot and BotEq scenarios are very close each other, with less separation between $S_2$ and $S_1$. Finally, the criteria subset labeled Top provides the highest rank index, suggesting that the selected subset is feasible to be the minimum set required.

Figure 12. Comparison of results with different criteria subsets.

5. Conclusions

As technology in IIoT and the cloud advances, there will be new options available in the market for the organizations. In addition, there are aspects that are relevant, not only technical, but economical and social. The three alternatives evaluated for this paper are aligned to leaders identified by Gartner up to 2018; however, it doesn’t assure they will be the only ones in the near-, mid- or long-term.

The criteria proposed follows and adapt for today’s vision. People must have double deep abilities—which are technical and business. This is one of the reasons to add to technical criteria the angles of economics and social view. As per our literature review, economics and social views have not been considered. Our contribution to industry provides these two missing aspects.

Cost is one of the most difficult and confusing comparisons, if there is not a good scenario to run against each price schema. However, as it is shown in Table 11, cost is not the main driver to take a decision in IIoT. Security has the highest weight and this is understandable as an organization’s IIoT implementations and solutions will transmit sensitive data. Communication protocols are the second most important criterion, and the reasoning behind is the flexibility required for different sensors available in the market. Device management and display are very close in importance, which is logical as organizations need to deploy from dozens to thousands of devices for a solution, and having a dashboard to locate and get information about devices is important.

Of economic and social criteria, the most significant are cost and available resources, respectively; longevity in the market was the least important criterion. This can be read as organizations possibly being open to experiments and learning with newcomers.

It is the best to have different experts from different backgrounds or responsibility within the organization. The roles suggested in this methodology (Table 5) cover a large part of main organization areas. We decided to include not only the IT department, but operations, financial, human resources, and business unit leaders. This proves to be aligned with the criteria suggested. By inviting the ability to participate in different roles, the weighting criteria become more accurate; therefore, the selection process will be better. We do not suggest to have a single expert to provide an opinion on criteria weighting. As people may have different understanding or could be biased towards a specific criteria, having more than one expert is preferred, and our proposed set of roles provides the options to select the experts.

Use of Saaty scale and method to evaluate criteria importance was proven to be effective. However, we discover that the validation of opinions is even more important, in order to provide trustworthy weights for the selection criteria. In our experiment, consistency ratio was 0.06, which is acceptable and allows for continuing with the process. Organizations must use these kinds of validations when choosing what would be more important over other criteria.

As it was discovered in the literature review (Table 2), most work related to cloud and IIoT has focused on AHP and TOPSIS. However, selecting an IIoT platform cannot have a single alternative winner; it is better to have all alternatives ranked. Our experience states, in some cases, that the vendor selected cannot deliver or does not meet other organizations’ requirements such as terms, legal contracts, conditions, or timing. When this happens, it would be a waste of time to redo the whole MCDA process again. This is why PROMETHEE-II has been proven to be effective as it can rank from top to bottom the alternatives available. In our exercise, Azure was the first option, followed by AWS and GCP.

It is important to notice that PROMETHEE-II and our methodology will not say which platform or technology is better, from an absolute standpoint, but which platform or technology is better suited for the organization based on the weights and grades provided by experts within the organization.

The paper demonstrated that our proposed methodology is effective at finding the best alternative to select an IIoT platform vendor as it has been performed consistently with five, seven, and ten criteria subsets, as well as comparing results against other methods. In addition, it contributes to the field of IIoT, as it provides a novel method to solve the problem many organizations are or will face at any time. Combining Saaty weight method and PROMEHEE-II, decision makers have a good tool to perform the selection. However, if it is limited to the technical aspects, the result may be biased and miss important aspects of the market. For example, if the technology is very good, the platform is the most complete and least expensive, but if there are not engineers or developers available, or training classes cost a fortune, implementing this platform will be a difficult and expensive project, with hidden costs not detected since inception. This is the reason and justification to include economic and social aspects in the criteria, as our methodology proposes.

IIoT platform selection should not be left to IT departments or CIO or CTO. Doing that will miss the point of view of other important leaders that will use, maintain or benefit from the selected platform. The Chief Operation Officer, leaders from business units, interdisciplinary teams, and even human resources and finance should participate in the MCDA process, as they bring ideas and considerations that sometimes are ignored unintentionally. Our proposed methodology provides a suggested list of key persons that should participate, something that was not found in the literature, and is very valuable for the decision process.

As a side discovery, comparing price schemes among vendors is not an easy task. We saw it as very useful to have a common scenario to run against the price schemes. To build a common scenario, it is required to have a close to reality idea of usage, number of devices, message size, and frequency of communication. Trying to compare price schemes without this scenario could lead to incorrect information entered into the grading matrix of the PROMETHEE-II part (Table 16).

The process of doing calculations and operations is laborious, due to the nature of algorithms used in our proposed methodology. This inspires us to continue the future work enhancing the methodology, creating a software to facilitate the computation. Another key aspect is the importance grading from Saaty’s process. Filling the matrix with reciprocal values could lead to human error easily. This also highlights, as part of our future work, to develop a graphical user interface that experts can use in a friendly fashion to enter the importance between criteria and fully automate our methodology when multiple experts participate in the process.

Future research work will focus on the fact that, by 2047, the year with the greatest incidence of a paradigm change in Generation Z in Industry 4.0, each tender that will require detailing the side effect of environmental impact can be carried out by an intelligent system using multi-criteria analysis to determine the best option for an alternative in a set of parts supplying resolution possibilities, where decision-making is decisive for its adequate solution, as can be seen in the following Figure 13.

Figure 13. Conceptual diagram of an Intelligent Model that can adequately determine the best multi-criteria selection of a component supply model associated with Industry 4.0.

The decision-making in this century will allow for extending in the Z generation to societies with a specific competitive value such as Bouganville, Brunei, Chuuk, East Timor, Rapa Nui, Sarawak, and Tuva that will have more symbolic capital with a combination of low population and diverse natural resources. Where manual work or traditional manufacturing will generate valuable cultural artifacts such as a French poodle made with balloons, and of which there will be no mass production, something that will be an avant-garde model for the Z generation and their descendants.

Finally, our future work will explore the use and implementation of other techniques to find the minimum criteria required to select the optimum IIoT platform, applying machine learning and data mining techniques. In addition, we plan to expand data acquisition from different experts around the globe in the roles identified previously. This is planned to be achieved by publishing a tool accessed via a web browser to collect the importance of each criterion in pairwise comparison.