# ExtrIntDetect—A New Universal Method for the Identification of Intelligent Cooperative Multiagent Systems with Extreme Intelligence

^{1}

^{2}

^{3}

^{*}

## Abstract

**:**

## 1. Introduction

## 2. State-of-the-Art Metrics Designed to Measure Machine Intelligence

## 3. The Proposed ExtrIntDetect Method

_{1}, ICM

_{2}, …, ICM

_{z}}. Each studied ICMAS comprises a specific number of agents. Each ICMAS can have any number of agents, and could possess any architecture. There are no restrictions related to either the studied systems having an equal number of agents, or to their having the same architectural homogeneity. We will denote with |ICM

_{r}| the cardinality of ICM

_{r}(i.e., the number of its agents).

_{1}, Prob

_{2}, … Prob

_{z}} denotes the set of problems used for the evaluation of problem-solving intelligence in the studied set of ICMASs. Each system’s problem-solving intelligence is evaluated based on a specific set of problems. We will denote with Prob

_{k}= {Prob

_{k},

_{1}, Prob

_{k},

_{2}, …, Prob

_{k},

_{m}} the set of problems used for the problem-solving intelligence evaluation of ICM

_{k}(ICM

_{k}∈ICM). |Prob

_{k}|, (|Prob

_{k}| = m) represents the cardinality (i.e., the number of problems) used in the experimental intelligence evaluation of ICM

_{k}. There is no requirement for the sets of problems Prob

_{1}, Prob

_{2}, … and Prob

_{z}to be the same. There is no restriction that the cardinalities |Prob

_{1}|, |Prob

_{2}|, … and |Prob

_{z}| of Prob

_{1}, Prob

_{2}, … and Prob

_{z}to be the same. He is responsible in the case of each ICM

_{k}for the establishment of |Prob

_{k}| and Prob

_{k}(as described in the MPSI algorithm). It was determined that the |Prob

_{k}| value should be at least 5 (|Prob

_{k}| ≥ 5) for the calculation of the intelligence quotient of the ICMAS. This minimum permitted value was set in order to allow the application of statistical testing for the verification of the data normality assumption. It is recommended, when possible, that intelligence evaluations be carried out for larger sets, |Prob

_{k}| ≥ 10, or even |Prob

_{k}| ≥ 30.

_{k}= {Intellig

_{k,1}, Intellig

_{k,2}, …, Intellig

_{k,m}} denotes the intelligence indicators obtained as a result of the Prob

_{k}problem-solving intelligence evaluations performed for a specific ICMAS denoted ICM

_{k}.

_{1}× ms

_{1}+ imp

_{2}× ms

_{2}+ … + imp

_{c}× ms

_{c}.

imp

_{1}+ imp

_{2}+ … + imp

_{c}= 1, imp

_{i}> 0

_{1}, ms

_{2}, …, ms

_{c}represent the considered intelligence components measure at a particular problem-solving intelligence evaluation; imp

_{1}, imp

_{2}, …, imp

_{c}represent the weights associated with the intelligence components. The weight of a component quantifies the importance of that component in the intelligence evaluation measure.

#### The Scenario of Coalitions of Agent-Based Flying Drones

_{1}, ICM

_{2}, …, ICM

_{n}. Each ICMAS is composed of a specific number of agents specialized in the delivery of products to customers. Each ICMAS solves problems in problem-solving cycles. A problem-solving cycle consists of a certain number of deliveries to clients by the drones that compose the respective ICMAS. A problem-solving cycle is finished when all the products have been delivered. A drone could undertake the task of a certain number of product deliveries to one or multiple destinations. Each visited client could be the recipient of one or more products. Each flying drone visits each client only once, at most. After all the deliveries have been completed, the drones should return to their starting locations. To determine the optimal route requirements for delivery, it is necessary that each drone solves a Travelling Salesman Problem.

_{1}and imp

_{2}, established by He, based on the intelligence of the delivery. For the delivery of products (one or more) to clients (one or more) at home, a mark between 0 and 7 is given by He. The first intelligent component measure has a weight of 0.65 (ms

_{1}= 0.65). The second intelligence component measure is established as the mean of the marks with values between 0 and 7 that have been given by clients who have received products in that delivery, with the weight of 0.35 (ms

_{2}= 0.35). For instance, a scenario involving the delivery of some products to two clients is considered. Each client gives a mark, and the mean of the values is computed. The weight of this mean is marked as 0.35. The reason for choosing these weighting values by He lies in the fact that human assessors have deeper knowledge regarding ICMASs than the clients. The clients are common people who conclude their evaluation mostly based on their satisfaction with the delivery. The reason for the weighting of the evaluation of clients being no less than half of the weighting of the evaluation of the human specialist is that clients’ satisfaction with deliveries is an important factor for the company that owns the drones. The final obtained intelligence indicator value based on the values of the two intelligence components will be a number in the interval [0, 7]. 0 signifies a system with no measurable intelligence. 7 signifies brilliant system (with the utmost possible intelligence).

Algorithm 1 MPSI measurement |

Measuring the Problem-Solving Intelligence AlgorithmIN://Given the studied ICM _{k} that have a specific architecture and a specific number of agents.ICM _{k} = {Agent_{1}, Agent_{2}, …, Agent_{q}};OUT:|Prob_{k}|; Prob_{k} = {Prob_{k,1}, …, Prob_{k,m}};Intellig _{k} = {Intellig_{k,1}, Intellig_{k,2}, …, Intellig_{k,m}};BeginStep 1. He establishes |Prob_{k}| and Prob_{k}.|Prob _{k}|: = m; Prob_{k} = {Prob_{k,1}, …, Prob_{k,m}}Step 2. Obtaining the intelligence indicators data.//If there are more intelligence components, than each evaluated problem-solving intelligence is calculated according to (1). @Performing the problem-solving intelligence evaluations and calculus; Intellig _{k} = {Intellig_{k,1}, Intellig_{k,2}, …, Intellig_{k,m}};EndMPSIAlgorithm |

_{1}, Intellig

_{2}, …, Intellig

_{z}} denotes the problem-solving intelligence indicators of the studied ICMASs. Intellig

_{1}represents the problem-solving intelligence obtained by evaluating ICM

_{1}. Intellig

_{2}represents the problem-solving intelligence obtained by evaluating ICM

_{2}. … Intellig

_{z}represents the problem-solving intelligence obtained by evaluating ICM

_{z}. The MPSI algorithm is applied in order to obtain the intelligence indicators for each ICMAS.

_{1}, ICM

_{2}, …, ICM

_{z}. It describes the calculation of machine intelligence quotients, MIQS = {MIQ

_{1}, MIQ

_{2}, …, MIQ

_{z}} based on the provided intelligence indicator data obtained for ICM

_{1}, ICM

_{2}, …, ICM

_{z}by applying the MPSI algorithm. MIQ

_{k}denotes the machine intelligence quotient of the ICM

_{k}, which indicates its central intelligence tendency. It should be noted that the most suitable indicator of the central intelligence tendency of an ICMAS is the median in cases where the intelligence indicator data does not pass the normality assumption, and the mean in where the data passes the normality assumption. The previous assertion is based on the fact that a very high or a very low value influences the median less than the mean. TypeMIQ, used in the algorithm, is a variable that indicates the calculus of MIQS as mean or median.

Algorithm 2 MMI measurement method |

Robust Measurement of the Intelligence Quotient AlgorithmIN:Intellig = {Intellig_{1}, Intellig_{2}, …, Intellig_{z}};//Intelligence indicators sample of the ICM.OUT: TypeMIQ; MIQS;BeginStep 1. Calculus of the MIQS.TypeMIQ: = ”Mean”; For (i: = 1 to z) Do@Verify if Intellig _{i} passes the normality assumption. If (Intellig_{i} failed to passes the normality assumption) ThenTypeMIQ: = ”Median”; Exit For;EndIfEndFor//He can change TypeMIQ based on the problem and domain-specific knowledge. @He makes the final decision on the TypeMIQ calculus as the mean or the median; For (i: = 1 to z) DoIf (TypeMIQ = ”Mean”) then MIQ_{i}: = MEAN(Intellig_{i});Else MIQ_{i}: = MEDIAN(Intellig_{i});EndIFEndForEndMMImeasuring |

_{1}, Intellig

_{2}, …, Intellig

_{z}, are initially verified as having passed the normality assumption. According to the algorithm, if one or more ICMASs do not pass the normality assumption, then the automatic decision will be to calculate the MIQ of all ICMASs based on the intelligence indicator medians (TypeMIQ = “Median”). In the contrary case, the automatic decision will be to compute the MIQ of all the ICMASs based on the mean (TypeMIQ = “Mean”). He has the right to make final modifications when choosing the mean or the median if she/he is in possession of domain-specific knowledge.

Algorithm 3 ExtrIntDetect method of measurement and detection |

Detection of ICMASs with Extreme Intelligence AlgorithmIN:MIQS = {MIQ_{1}, MIQ_{2}, …, MIQ_{z}};//Calculated machine intelligence quotients.OUT://IdOut ^{L}, IdOut^{H}, if normality expected.IdOut ^{L};//MIQ of ICMASs with low extreme intelligence.IdOut ^{H};//MIQ of ICMASs with high extreme intelligence.//HExtr, HExtrOut, if normality NOT expected. HExtr;//MIQ of ICMASs with high extreme intelligence. HExtrOut;//MIQ of ICMASs with high outlier extreme intelligence. LExtr;//MIQ of ICMASs with low extreme intelligence. HExtrOut;//MIQ of ICMASs with low outlier extreme intelligence. BeginStep 1. Descriptive characterization of the MIQS data.@Calculates for MIQS the: mean, median, SD, SEM, skew, and kurt. CV: = 100 × (SD/mean); Step 2. Analyze if MIQS data passes the normality assumption.@Apply Lill and SW tests in order to verify MIQS normality. @Constructs the QQ plot for visual analysis of MIQS. @Based on the obtained result of data normality analysis set the value of norm variable; If (norm = ”No”) Then@He has the right to change norm value to “Yes”. EndIfStep 3. Detection of extremes if normality expected.If (norm = ”Yes”) ThenIdOut ^{L}: = ∅;IdOut^{H}: = ∅;OutFound: = ”Yes”; IdOut: = MIQS; avg*: = mean(IdOut); While ((OutFound = ”Yes”) and (|IdOut|≥3)) DoOutFound: = ”No”; @It is applied the Grubbs test; If (an outlier denoted OUT was identified) ThenIdOut: = IdOut\{OUT}; OutFound: =”Yes”; If (OUT < avg*) Then IdOut^{L}: = IdOut^{L} ∪ {OUT};Else IdOut^{H}: = IdOut^{H} ∪ {OUT};EndIfEndIfavg* = mean(IdOut); EndWhileEndIFIf (|IdOut^{L}| ≥ 2) Then @Order ascending IdOut^{L}. EndIfIf (|IdOut^{H}| ≥ 2) Then @Order ascending IdOut^{H}. EndIfStep 4. Detection of extremes if normality is NOT expected.If (Norm = ”No”) Then HExtr: = ∅; HExtrOut: = ∅; LExtr: = ∅; LExtrOut: = ∅; @Arrange MIQS = {MIQ _{1}, MIQ_{2}, …, MIQ_{z}} in ascending order.@Calculate Q _{1} and Q_{3};//The first and third quartile.IQR: = Q _{3}−Q_{1};//Calculates the interquartile range.@He establishes RangeL and RangeU values. HLowLim: = Q _{3} + RangeL × IQR; HUppLim: = Q_{3} + RangeU × IQR;LLowLim: = Q _{1} − RangeU × IQR; LUppLim: = Q_{1} − RangeL × IQR;For (i: = 1 to z) DoIf (MIQ_{i} ∈ [HLowLim, HUppLim)) Then HExtr: = HExtr ∪ {MIQ_{i}};ElseIf (MIQ_{i} ≥ HUppLim) Then HExtrOut: = HExtrOut ∪ {MIQ_{i}};EndIfIf (MIQ_{i} ∈ (LLowLim, LUppLim]) Then LExtr: = LExtr ∪ {MIQ_{i}};ElseIf (MIQ_{i} ≤ LLowLim) Then LExtrOut: = LExtrOut ∪ {MIQ_{i}};EndIfEndForIf (|HExtr| ≥ 2) Then @Order ascending HExtr. EndIfIf (|HExtrOut| ≥ 2) Then @Order ascending HExtrOut. EndIfIf (|LExtr| ≥ 2) Then @Order ascending LExtr. EndIfIf (|LExtrOut| ≥ 2) Then @Order ascending LExtrOut. EndIFEndEndExtrIntDetectAlgorithm |

_{1}, MIQ

_{2}, …, MIQ

_{z}} data. The variable mean represents the calculated mean of MIQS. The variable median represents the calculated median of MIQS. The median is the middle of a sorted list of numbers. If the list has an even number of items, then the median is calculated as the average of the two numbers in the middle. The value Standard deviation (SD) expresses a quantity by the degree to which the members of a dataset differ from the mean of that dataset. SD effectively quantifies the amount of variation/dispersion in a dataset [53]. The standard error (SE) of a specific parameter is effectively the standard deviation of its sampling distribution. When the parameter or the statistic is the mean, then it is referred to as the standard error of the mean (SEM). SEM = SD/sqrt(N), where sqrt denotes the square root. The variable min denotes the lowest value. The variable max denotes the highest value. The variable range = max−min is frequently an important indicator.

_{a}, CV

_{b}, and CV

_{c}should be set based on the specifics of the studied ICMASs. The dataset exhibits homogeneous variability of intelligence level (hom.) when CV < CV

_{a}; relatively homogeneous variability of intelligence level (rel-hom.) when CV ∈ [CV

_{a}, CV

_{b}); relatively heterogeneous variability of intelligence level (rel-het.) when CV ∈ [CV

_{b}, CV

_{c}); and heterogeneous variability of intelligence level (het.) when CV ≥ CV

_{c}. In most cases, the recommended values for the parameters described above are CV

_{a}= 10, CV

_{b}= 20, and CV

_{c}= 30.

^{L}is the set of identified ICMASs with low extreme intelligence values. IdOut

^{H}is the set of identified ICMASs with high extreme intelligence values. Depending on the type of studied ICMASs, it could be decided to extend the detection to ICMASs whose intelligence may not be extreme, but which are further removed from the rest. The Grubbs outliers detection test [63,64] is used for the identification of extremes. It is suggested that this test be applied with a significance level αExt = 0.05. This value means that a 5% chance of mistakenly identifying an outlier in the sample is accepted. Other values of significance level could be set, but they are less frequently appropriate. A two-sided test was applied in order to be able to identify both low and high extremes.

_{1}denotes the first quartile, which represents the median of the lower half of the dataset. About 25% of the elements in the dataset will be positioned below Q

_{1}. Q

_{3}denotes the third quartile, the median of the upper half of the dataset. About 25% of the numbers will be positioned above Q

_{3}. RangeL and RangeU are parameters whose values must be established by He based on the specific characteristics of the studied set of ICMASs.

## 4. Experimental Evaluation of the ExtrIntDetect Method

_{1}, ICM

_{2}, ICM

_{3}, ICM

_{4}, ICM

_{5}, ICM

_{6}}, specialized in solving an NP-hard problem, the Symmetric Travelling Salesman Problem (sTSP) [67,68], is considered. The sTSP can be defined by considering a set of cities (nodes of a weighted undirected graph), with the distances between each pair of cities being the same in both directions. The aim is to find the shortest route that visits each city (node) exactly once and then returns to the origin city. A traditional formulation of the sTSP is as an integer linear programming problem [69]. Diverse real-life problems, including vehicle routing, warehouse logistics, circuit board planning, and virtual networking, are modelled as symmetric graphs.

_{k}currently at node i chooses to move to node j by applying the probabilistic transition rule (2). The parameter α denotes the power of the pheromone, and β denotes the relative weight of the heuristic visibility of the pheromone trail. The value d

_{kh}represents the distance between nodes (k and h); η

_{kh}= 1/d

_{kh}is the heuristic visibility of the edge (k, h); and τ

_{kh}(t) represents the pheromone amount deposited on the edge (k, h) at the considered iteration t.

_{k}is the length of the tour performed by agent

_{k}.

_{1}and ms

_{2}. ms

_{1}is the global-best solution found during the problem-solving cycle. ms

_{2}is the number of iterations before the global-best solution is found. The importance of the two components is considered by He as follows, imp

_{1}= 0.9 and imp

_{2}= 0.1 (imp

_{1}+ imp

_{2}= 1). In all cases, the intelligence indicator is calculated on the basis of (1). It should be noted that, based on the specificity of the intelligence indicator, lower values indicate higher intelligence.

_{1}is operated as an Ant System (AS) [74,75,76]; ICM

_{2}is operated as an Elitist Ant System (eAS) [77]; ICM

_{3}is operated as a Ranked Ant System (rAS) [78]; ICM

_{4}is operated as a Best-Worst Ant System (bwAS) [79]; ICM

_{5}is operated as a Min-Max Ant System (mmAS) [80]; ICM

_{6}is operated as an Ant Colony System (aCS) [75,76,81].

_{i}| = 12, ∀ I = 1.6; m = 10; maximal number of iterations, ItNum = 995; α = 1.3; β = 1.35 and ρ = 0.25. Table 1 presents the intelligence component results obtained in the experimental problem-solving evaluations. Table 2 presents the problem-solving intelligence evaluation results Intellig = {Intellig

_{1}, Intellig

_{2}, …, Intellig

_{6}} calculated according to (1) based on the two measured components, with each component counted based on the established weighting. Table 3 presents the results of the statistical characterization of intelligence realized based on Intellig = {Intellig

_{1}, Intellig

_{2}, …, Intellig

_{6}}, which allows the formulation of useful conclusions, like the homogeneity–heterogeneity of intelligence indicator data. For the verification of homogeneity–heterogeneity, the following values for the parameters were used: CV

_{a}= 10, CV

_{b}= 20 and CV

_{c}= 30.

_{1}, Intellig

_{2}, …, Intellig

_{6}. The visual analysis of Figure 2, Figure 3, Figure 4, Figure 5, Figure 6 and Figure 7 enables the formulation of the conclusion that none of Intellig

_{1}, Intellig

_{2}, …, Intellig

_{6}fail to pass the normality assumption.

_{1}, Intellig

_{2}, …, Intellig

_{6}. It was considered that a significance level of αNorm = 0.05 was the most appropriate for the application of both tests. An intelligence indicator dataset fails to pass the normality assumption when Pnorm < αNorm, where Pnorm represents the calculated p-value of an applied normality test. Based on the obtained results, the same conclusion can be formulated with respect to normality as was formulated for the QQ plots.

_{1}= 2175.95, MIQ

_{2}= 6817.65, MIQ

_{3}= 1100.9, MIQ

_{4}= 1430.3, MIQ

_{5}= 1187.65, MIQ

_{6}= 1073.75}. Figure 8 presents the MIQS data. Based on the specificity of the considered intelligence components and the intelligence indicator, smaller MIQ values indicate higher machine intelligence. For instance, MIQ

_{2}> MIQ

_{6}suggests that ICM

_{6}has a higher intelligence than ICM

_{2}.

_{1}, MIQ

_{2}, MIQ

_{3}, MIQ

_{4}, MIQ

_{5}, MIQ

_{6}} = {2175.95, 6817.65, 1100.9, 1430.3, 1187.65, 1073.75} data does not pass the normality assumption, according to the ExtrIntDetect algorithm, Step 4 is indicated for the identification of extreme intelligence. Acting as He, we decided on RangeL = 1.5 and RangeU = 3. The obtained numerical results were: median = 1308.98, Q

_{1}= 1094.11, Q

_{3}= 3336.37, IQR = 5743.9, HLowLim = 11,952.22, HUppLim = 20,568.07, LLowLim = −7521.74, LUppLim = −16,137.6.

_{i}∈ (LLowLim, LUppLim] and MIQ

_{i}≤ LLowLim were not verified. This indicates that there were no ICMASs with high extreme intelligence or high outlier extreme intelligence. Given that lower intelligence values indicate higher intelligence, this is the case for high intelligence.

_{i}≥ HUppLim = 20568.07 was verified for MIQ

_{1}[2175.95] and MIQ

_{2}[6817.65]. In this way, ICM

_{2}and ICM

_{1}were identified as having low outlier extreme intelligence, where ICM

_{2}was less intelligent than ICM

_{1}(based on the fact that 2175.95 < 6817.65).

_{i}∈ [HLowLim, HUppLim) = [11952.22, 20568.07) was verified by the value corresponding to MIQ

_{4}[1430.3], identifying ICM

_{4}in this way as an ICMAS with low extreme intelligence.

_{1}, MIQ

_{2}, MIQ

_{3}, MIQ

_{4}, MIQ

_{5}, MIQ

_{6}} data. For the identification of the extremes, Grubbs’ test was applied with a significance level of αExt = 0.05. A two-sided test was applied, as both high and low extreme intelligence values were being sought at the same time. Table 7 presents the identified extremes.

_{1}, MIQ

_{2}, MIQ

_{3}, MIQ

_{4}, MIQ

_{5}, MIQ

_{6}}, avg* = mean(MIQS*) = 2297.7. With the first application of the identification of extremes based on MIQS* data, ICM

_{2}was identified as having extreme intelligence, MIQ

_{2}= 6817.65. ICM

_{2}has low extreme intelligence based on the fact that MIQ

_{2}> avg*. MIQS* = MIQS*−{MIQ

_{2}}. With the second application of the identification of extremes based on MIQS* data, no extremes were identified.

_{1}= 2175.95, which is far removed from the values, was detected. avg* = mean(MIQS*) = 1393.71; MIQ

_{1}> avg*, proving that ICM

_{1}is lower than the rest of the intelligence values. MIQS* = MIQS*−{MIQ

_{1}}, avg* = mean(MIQS*) = 1198. To identify further different intelligence values, the detection of extremes was applied again on the MIQS* data, identifying MIQ

_{4}= 1430.3 as another value that is not extreme but is far removed from the rest; MIQ

_{4}> avg* indicates that it was lower than the rest of the intelligence values. avg* = mean(MIQS*) = 1120.77, MIQS* = MIQS*−{MIQ

_{4}}. With a new application, MIQ

_{5}= 1187.65 was identified as being far removed from the rest (MIQ

_{5}> avg*). Based on the fact that |MIQS*| = 2, which is not higher than or equal to 3, the detection of other extremes or values far removed from the rest could not be continued.

## 5. Discussion

#### 5.1. Discussion of the Experimental Results

_{2,}and ICM

_{5}, as having low outlier extreme intelligence, and one ICMAS, namely ICM

_{4}, as having low extreme intelligence.

_{2}, with low extreme intelligence, and three ICMASs, namely ICM

_{5,}ICM

_{4}and ICM

_{1}, with low intelligence that was far removed from the rest.

#### 5.2. Discussion of the ExtrIntDetect Method

## 6. Conclusions

## Author Contributions

## Funding

## Acknowledgments

## Conflicts of Interest

## References

- Gelenbe, E.; Lent, R.; Xu, Z. Design and performance of cognitive packet networks. Perform. Eval.
**2001**, 46, 155–176. [Google Scholar] [CrossRef] - Sakellari, G. Performance evaluation of the Cognitive Packet Network in the presence of network worms. Perform. Eval.
**2011**, 68, 927–937. [Google Scholar] [CrossRef] - Spoto, S.; Gribaudo, M.; Manini, D. Performance evaluation of peering-agreements among autonomous systems subject to peer-to-peer traffic. Perform. Eval.
**2014**, 77, 1–20. [Google Scholar] [CrossRef] - Crovella, M.; Lindemann, C.; Reiser, M. Internet performance modeling: The state of the art at the turn of the century. Perform. Eval.
**2000**, 42, 91–108. [Google Scholar] [CrossRef] - Al-Rousan, M.; Archibald, J.K.; Bearnson, L. Evaluating the impact of locality on the performance of large-scale SCI multiprocessors. Perform. Eval.
**2001**, 46, 275–302. [Google Scholar] [CrossRef] - Liu, L.; Lim, S. A voxel-based multiscale morphological airborne lidar filtering algorithm for digital elevation models for forest regions. Measurement
**2018**, 123, 135–144. [Google Scholar] [CrossRef] - Wibowo, S.; Grandhi, S. Fuzzy Multicriteria Analysis for Performance Evaluation of Internet-of-Things-Based Supply Chains. Symmetry
**2018**, 10, 603. [Google Scholar] [CrossRef] - Karimi, N.; Kondrood, R.R.; Alizadeh, T. An intelligent system for quality measurement of Golden Bleached raisins using two comparative machine learning algorithms. Measurement
**2017**, 107, 68–76. [Google Scholar] [CrossRef] - Sen, B.; Mandal, U.K.; Mondal, S.P. Advancement of an intelligent system based on ANFIS for predicting machining performance parameters of Inconel 690—A perspective of metaheuristic approach. Measurement
**2017**, 109, 9–17. [Google Scholar] [CrossRef] - Ahmadi, F.F.; Layegh, N.F. Integration of close range photogrammetry and expert system capabilities in order to design and implement optical image based measurement systems for intelligent diagnosing disease. Measurement
**2014**, 51, 9–17. [Google Scholar] [CrossRef] - Sobolev, V.; Aumala, O. Metrological automatic support of measurement results in intelligent measurement systems. Measurement
**1996**, 17, 151–159. [Google Scholar] [CrossRef] - Liu, W. Intelligent fault diagnosis of wind turbines using multi-dimensional kernel domain spectrum technique. Measurement
**2019**, 133, 303–309. [Google Scholar] [CrossRef] - Popescu, D.; Ichim, L. Intelligent Image Processing System for Detection and Segmentation of Regions of Interest in Retinal Images. Symmetry
**2018**, 10, 73. [Google Scholar] [CrossRef] - Wang, D.; Ren, H.; Shao, F. Distributed Newton Methods for Strictly Convex Consensus Optimization Problems in Multi-Agent Networks. Symmetry
**2017**, 9, 163. [Google Scholar] [CrossRef] - Iantovics, L.B.; Zamfirescu, C.B. ERMS: An evolutionary reorganizing multiagent system. Innov. Comput. Inf. Control
**2013**, 9, 1171–1188. [Google Scholar] - Kwon, H.; Pack, D.J. A Robust Mobile Target Localization Method for Cooperative Unmanned Aerial Vehicles Using Sensor Fusion Quality. J. Intell. Robot. Syst.
**2012**, 65, 479–493. [Google Scholar] [CrossRef] - Saska, M.; Vonasek, V.; Krajnik, T.; Preucil, L. Coordination and Navigation of Heterogeneous MAV-UGV Formations Localized by a ‘hawk-eye’-like Approach Under a Model Predictive Control Scheme. Int. J. Robot. Res.
**2014**, 33, 1393–1412. [Google Scholar] [CrossRef] - Chase, D. Underlying Factor Structures of the Stanford-Binet Intelligence Scales, 5th ed.; Drexel University: Philadelphia, PA, USA, 2005. [Google Scholar]
- Kaufman, A.S. IQ Testing 101; Springer: New York, NY, USA, 2009. [Google Scholar]
- Nicolas, S.; Andrieu, B.; Croizet, J.C.; Sanitioso, R.B.; Burman, J.T. Sick? Or slow? On the origins of intelligence as a psychological object. Intelligence
**2013**, 41, 699–711. [Google Scholar] [CrossRef] - Bilker, W.B.; Hansen, J.A.; Brensinger, C.M.; Richard, J.; Gur, R.E.; Gur, R.C. Development of abbreviated nine-item forms of the Raven’s standard progressive matrices test. Assessment
**2012**, 19, 354–369. [Google Scholar] [CrossRef] - Raven, J.C. Mental Tests used in Genetic studies: The performance of Related Individuals on Tests Mainly Educative and Mainly Reproductive. MSc Thesis, University of London, London, UK, 1936. [Google Scholar]
- Kaufman, A.S.; Lichtenberger, E. Assessing Adolescent and Adult Intelligence, 3rd ed.; Wiley: Hoboken, NJ, USA, 2006; p. 3. [Google Scholar]
- Wechsler, D. The Measurement of Adult Intelligence. Baltimore (MD); Williams & Witkins: Philadelphia, PA, USA, 1939; p. 229. [Google Scholar]
- Kaufman, A.S.; Kaufman, N.L. Kaufman test of Educational Achievement Comprehensive Form, 2nd ed.; Circle Pines, N., Ed.; American Guidance Service: Circle Pines, MN, USA, 2004. [Google Scholar]
- Kaufman, A.S.; Kaufman, N.L. Kaufman Assessment Battery for Children; American Guidance Service: Circle Pines, MN, USA, 1983. [Google Scholar]
- Kaufman, A.S.; Kaufman, N.L. Kaufman Assessment Battery for Children, 2nd ed.; American Guidance Service: Circle Pines, MN, USA, 2004. [Google Scholar]
- Neisser, U.; Boodoo, G.; Bouchard, T.J.J.; Boykin, A.W.; Brody, N.; Ceci, S.J.; Halpern, D.F.; Loehlin, J.C.; Perloff, R.; Sternberg, R.J.; et al. Intelligence: Knowns and unknowns. Am. Psychol.
**1996**, 51, 77–101. [Google Scholar] [CrossRef] - Schmidt, F.L.; Hunter, J. General mental ability in the world of work: Occupational attainment and job performance. J. Pers. Soc. Psychol.
**2004**, 86, 162–173. [Google Scholar] [CrossRef] [PubMed] - Arik, S.; Iantovics, L.B.; Szilagyi, S.M. OutIntSys—A Novel Method for the Detection of the Most Intelligent Cooperative Multiagent Systems. In Proceedings of the 24th International Conference on Neural Information Processing (ICONIP 2017), Guangzhou, China, 14–18 November 2017; pp. 31–40, LNCS 10637. [Google Scholar]
- Iantovics, L.B.; Dehmer, M.; Emmert-Streib, F. MetrIntSimil-An Accurate and Robust Metric for Comparison of Similarity in Intelligence of Any Number of Cooperative Multiagent Systems. Symmetry
**2018**, 10, 48. [Google Scholar] [CrossRef] - Iantovics, L.B.; Gligor, A.; Niazi, A.M.; Biro, A.I.; Szilagyi, S.M.; Tokody, D. Review of Recent Trends in Measuring the Computing Systems Intelligence. BRAIN Broad Res. Artif. Intell. Neurosci.
**2018**, 9, 77–94. [Google Scholar] - Turing, A.M. Computing machinery and intelligence. Oxford University Press on behalf of the Mind Association, Mind. New Ser.
**1950**, 59, 433–460. [Google Scholar] [CrossRef] - Dowe, D.L.; Hajek, A.R. A non-behavioural, computational extension to the Turing Test. In Proceedings of the International Conference on Computational Intelligence & Multimedia Applications (ICCIMA 1998), 7–10 February 1998, Gippsland, Australia; World Scientific Publishing: Singapore, 1998; pp. 101–106. [Google Scholar]
- Schreiner, K. Measuring IS: Toward a US standard. IEEE Intell. Syst. Appl.
**2000**, 15, 19–21. [Google Scholar] [CrossRef] - Park, H.J.; Kim, B.K.; Lim, K.Y. Measuring the machine intelligence quotient (MIQ) of human-machine cooperative systems. IEEE Trans. Syst. Man Cybern. Part A Syst. Hum.
**2001**, 31, 89–96. [Google Scholar] [CrossRef] - Sanghi, P.; Dowe, D.L. A Computer Program Capable of Passing I.Q. Tests. In Proceedings of the Joint International Conference on Cognitive Science, 4th ICCS International Conference on Cognitive Science and 7th ASCS Australasian Society for Cognitive Science (ICCS/ASCS 2003), Sydney, NSW, Australia, 13–17 July 2003; pp. 570–575. [Google Scholar]
- Legg, S.; Hutter, M. A formal measure of machine intelligence. In Proceedings of the 15th Annual Machine Learning Conference, Ghent, Belgium, 11–12 May 2006; pp. 73–80. [Google Scholar]
- Anthony, A.; Jannett, T.C. Measuring Machine Intelligence of an Agent-Based Distributed Sensor Network System; Advances and Innovations in Systems; Elleithy, K., Ed.; Springer: Dordrecht, The Netherland, 2007; pp. 531–535. [Google Scholar]
- Hernández-Orallo, J. Beyond the Turing Test. J. Logic. Lang. Inf.
**2000**, 9, 447–466. [Google Scholar] [CrossRef] - Hernández-Orallo, J.; Dowe, D.L. Measuring universal intelligence: Towards an anytime intelligence test. Artif. Intell.
**2010**, 174, 1508–1539. [Google Scholar] [CrossRef] [Green Version] - Hibbard, B. Measuring Agent Intelligence Via Hierarchies of Environments. In Artificial General Intelligence, Lecture Notes in Computer Science 6830; Schmidhuber, J., Thórisson, K.R., Looks, M., Eds.; Springer: Berlin/Heidelberg, Germany, 2011; pp. 303–308. [Google Scholar]
- Hernández-Orallo, J.; Dowe, D.L.; Hernández-Lloreda, M.V. Universal psychometrics: Measuring cognitive abilities in the machine kingdom. Cogn. Syst. Res.
**2014**, 27, 50–74. [Google Scholar] [CrossRef] [Green Version] - Besold, T.; Hernandez-Orallo, J.; Schmid, U. Can machine intelligence be measured in the same way as human intelligence? Künstliche Intelligenz
**2015**, 29, 291–297. [Google Scholar] [CrossRef] - Chmait, N.; Li, Y.F.; Dowe, D.L.; Green, D.G. A Dynamic Intelligence Test Framework for Evaluating AI Agents. In Proceedings of the Workshop Evaluating General-Purpose AI, The Hague, The Netherlands, 30 August 2016; pp. 1–8, EGPAI 2016. [Google Scholar]
- Liu, F.; Shi, Y.; Liu, Y. Intelligence quotient and intelligence grade of artificial intelligence. Ann. Data Sci.
**2017**, 4, 179–191. [Google Scholar] [CrossRef] - Razali, N.; Wah, Y.B. Power comparisons of Shapiro-Wilk, Kolmogorov-Smirnov, Lilliefors, Anderson-Darling tests. J. Stat. Model. Anal.
**2011**, 2, 2–33. [Google Scholar] - Lilliefors, H. On the Kolmogorov-Smirnov test for the exponential distribution with mean unknown. J. Am. Stat. Assoc.
**1969**, 64, 387–389. [Google Scholar] [CrossRef] - Lilliefors, H. On the Kolmogorov–Smirnov test for normality with mean and variance unknown. J. Am. Stat. Assoc.
**1967**, 62, 399–402. [Google Scholar] [CrossRef] - Shapiro, S.S.; Wilk, M.B. An analysis of variance test for normality (complete samples). Biometrika
**1965**, 52, 591–611. [Google Scholar] [CrossRef] - Stephens, M.A. EDF Statistics for Goodness of Fit and Some Comparisons. J. Am. Stat. Assoc.
**1974**, 69, 730–737. [Google Scholar] [CrossRef] - Wilk, M.B.; Gnanadesikan, R. Probability plotting methods for the analysis of data, Biometrika. Biometrika Trust
**1968**, 55, 1–17. [Google Scholar] - Bland, J.M.; Altman, D.G. Statistics notes: Measurement error. BMJ
**1996**, 312, 1654. [Google Scholar] [CrossRef] - Joanes, D.N.; Gill, C.A. Comparing measures of sample skewness and kurtosis. J. R. Stat. Soc. (Ser. D) Stat.
**1998**, 47, 183–189. [Google Scholar] [CrossRef] - Tian, Y.; Yin, Z.; Huang, M. Missing Data Probability Estimation-Based Bayesian Outlier Detection for Plant-Wide Processes with Multisampling Rates. Symmetry
**2018**, 10, 475. [Google Scholar] [CrossRef] - Li, G.; Wang, J.; Liang, J.; Yue, C. The Application of a Double CUSUM Algorithm in Industrial Data Stream Anomaly Detection. Symmetry
**2018**, 10, 264. [Google Scholar] [CrossRef] - Li, G.; Wang, J.; Liang, J.; Yue, C. Application of Sliding Nest Window Control Chart in Data Stream Anomaly Detection. Symmetry
**2018**, 10, 113. [Google Scholar] [CrossRef] - Ross, S.M. Peirce’s Criterion for the Elimination of Suspect Experimental Data. J. Engr. Technol.
**2003**, 2, 1–12. [Google Scholar] - Motulsky, H.J.; Brown, R.E. Detecting outliers when fitting data with nonlinear regression: A new method based on robust nonlinear regression and the false discovery rate. BMC Bioinform.
**2006**, 7, 123. [Google Scholar] [CrossRef] [PubMed] - Tietjen, G.; Moore, R. Some Grubbs-Type Statistics for the Detection of Several Outliers. Technometrics
**1972**, 14, 583–597. [Google Scholar] [CrossRef] - Zerbet, A.; Nikulin, M. A new statistics for detecting outliers in exponential case. Commun. Stat. Theory Methods
**2003**, 32, 573–584. [Google Scholar] [CrossRef] - Dean, R.B.; Dixon, W.J. Simplified Statistics for Small Numbers of Observations. Anal. Chem.
**1951**, 23, 636–638. [Google Scholar] [CrossRef] - Grubbs, F.E. Sample criteria for testing outlying observations. Ann. Math. Stat.
**1950**, 21, 27–58. [Google Scholar] [CrossRef] - Barnett, V.; Lewis, T. Outliers in Statistical Data, 3rd ed.; Wiley: Hoboken, NJ, USA, 1994. [Google Scholar]
- Grubbs, F.E. Procedures for Detecting Outlying Observations in Samples. Technometrics
**1969**, 11, 1–21. [Google Scholar] [CrossRef] - Stefansky, W. Rejecting Outliers in Factorial Designs. Technometrics
**1972**, 14, 469–479. [Google Scholar] [CrossRef] - Dantzig, G.B.; Fulkerson, D.R.; Johnson, S.M. Solution of a large-scale traveling-salesman problem. Oper. Res.
**1954**, 2, 393–410. [Google Scholar] [CrossRef] - Kovács, L.; Iantovics, L.B.; Iakovidis, D.K. IntraClusTSP—An Incremental Intra-Cluster Refinement Heuristic Algorithm for Symmetric Travelling Salesman Problem. Symmetry
**2018**, 10, 663. [Google Scholar] - Laporte, G. The Traveling Salesman Problem: An overview of exact and approximate algorithms. Eur. J. Oper. Res.
**1992**, 59, 231–247. [Google Scholar] [CrossRef] - Merkle, D.; Middendorf, M. On solving permutation scheduling problems with ant colony optimization. Int. J. Syst. Sci.
**2005**, 36, 255–266. [Google Scholar] [CrossRef] - Runkler, T.A. Ant colony optimization of clustering models. Int. J. Int. Syst.
**2005**, 20, 1233–1251. [Google Scholar] [CrossRef] - Crisan, G.C.; Pintea, C.M.; Palade, V. Emergency Management Using Geographic Information Systems. Application to the first Romanian Traveling Salesman Problem Instance. Knowl. Inf. Syst.
**2017**, 50, 265–285. [Google Scholar] [CrossRef] - Pholdee, N.; Bureerat, S. Hybrid real-code ant colony optimisation for constrained mechanical design. Int. J. Syst. Sci.
**2016**, 47, 474–491. [Google Scholar] [CrossRef] - Dorigo, M.; Maniezzo, V.; Colorni, A. Positive Feedback as a Search Strategy; Dipartimento di Elettronica, Politecnico di Milano: Milano, Italy, 1991. [Google Scholar]
- Colorni, A.; Dorigo, M.; Maniezzo, V. Distributed optimization by ant colonies. In Actes de la Premiere Conference Europeenne Sur la vie Artificielle; Elsevier Publishing: Paris, France, 1991; pp. 134–142. [Google Scholar]
- Dorigo, M. Optimization, Learning and Natural Algorithms. Ph.D. Thesis, Politecnico di Milano, Milano, Italy, 1992. [Google Scholar]
- Jaradat, G.M.; Ayob, M. An Elitist-Ant System for Solving the Post-Enrolment Course Timetabling Problem; Zhang, Y., Cuzzocrea, A., Ma, J., Chung, K., Arslan, T., Song, X., Eds.; Springer: Berlin/Heidelberg, Germany, 2010; pp. 167–176, FGIT 2010. CCIS 118. [Google Scholar]
- Bullnheimer, B.; Hartl, R.F.; Strauss, C. A new rank based version of the ant system. A computational study. Cent. Eur. J. Oper. Res.
**1999**, 7, 25–38. [Google Scholar] - Cordón, O.; de Viana, I.F.; Herrera, F. Analysis of the Best-Worst Ant System and its Variants on the QAP; Dorigo, M., Di Caro, G., Sampels, M., Eds.; Springer: Heidelberg, Germany, 2002; pp. 228–234, ANTS 2002. LNCS 2463. [Google Scholar]
- Stutzle, T.; Hoos, H.H. Max-min ant system. Future Gener. Comput. Syst.
**2000**, 16, 889–914. [Google Scholar] [CrossRef] - Dorigo, M.; Stützle, T. Ant Colony Optimization; MIT Press: Cambridge, MA, USA, 2004. [Google Scholar]
- Buscarino, A.; Fortuna, L.; Frasca, M.; Latora, V. Disease spreading in populations of moving agents, IOP Publishing. Europhys. Lett. (EPL)
**2008**, 82, 38002. [Google Scholar] [CrossRef]

Evaluation Number | ICM_{1} | ICM_{2} | ICM_{3} | ICM_{4} | ICM_{5} | ICM_{6} |
---|---|---|---|---|---|---|

ms_{1}/ms_{2} | ms_{1}/ms_{2} | ms_{1}/ms_{2} | ms_{1}/ms_{2} | ms_{1}/ms_{2} | ms_{1}/ms_{2} | |

Nr_{1} | 1063/104 | 1050/165 | 1268/18 | 1583/22 | 1488/29 | 1093/146 |

Nr_{2} | 1073/559 | 1045/115 | 1157/32 | 1709/32 | 1387/38 | 1123/226 |

Nr_{3} | 1067/306 | 1094/70 | 1255/58 | 1792/22 | 1481/19 | 1184/221 |

Nr_{4} | 1046/92 | 1048/131 | 1170/47 | 1645/21 | 1380/33 | 1170/68 |

Nr_{5} | 3698/21 | 3906/43 | 7730/53 | 9518/18 | 8597/10 | 3628/67 |

Nr_{6} | 8304/29 | 7747/64 | 9484/31 | 1381/12 | 1198/38 | 7935/53 |

Nr_{7} | 7527/16 | 7384/110 | 8791/37 | 1590/27 | 1090/44 | 7860/91 |

Nr_{8} | 8291/16 | 7823/93 | 9911/16 | 1423/15 | 1059/100 | 8359/87 |

Nr_{9} | 8054/168 | 8930/991 | 1023/37 | 1470/17 | 1156/39 | 9123/31 |

Nr_{10} | 9294/181 | 9598/41 | 1098/24 | 1424/28 | 1205/64 | 1092/290 |

Nr_{11} | 1015/68 | 9427/93 | 1124/48 | 1664/43 | 1263/28 | 1023/174 |

Nr_{12} | 1019/64 | 9434/182 | 1182/27 | 1377/22 | 1371/19 | 1024/68 |

Evaluation Number | Intellig_{1} | Intellig_{2} | Intellig_{3} | Intellig_{4} | Intellig_{5} | Intellig_{6} |
---|---|---|---|---|---|---|

Nr_{1} | 967.1 | 961.5 | 1143.0 | 1426.9 | 1342.1 | 998.3 |

Nr_{2} | 1021.6 | 952.0 | 1044.5 | 1541.3 | 1252.1 | 1033.3 |

Nr_{3} | 990.9 | 991.6 | 1135.3 | 1615.0 | 1334.8 | 1087.7 |

Nr_{4} | 950.6 | 956.3 | 1057.7 | 1482.6 | 1245.3 | 1059.8 |

Nr_{5} | 3330.3 | 3519.7 | 6962.3 | 8568.0 | 7738.3 | 3271.9 |

Nr_{6} | 7476.5 | 6978.7 | 8538.7 | 1244.1 | 1082.0 | 7146.8 |

Nr_{7} | 6775.9 | 6656.6 | 7915.6 | 1433.7 | 985.4 | 7083.1 |

Nr_{8} | 7463.5 | 7050.0 | 8921.5 | 1282.2 | 963.1 | 7531.8 |

Nr_{9} | 7265.4 | 8136.1 | 924.4 | 1324.7 | 1044.3 | 8213.8 |

Nr_{10} | 8382.7 | 8642.3 | 990.6 | 1284.4 | 1090.9 | 1011.8 |

Nr_{11} | 920.3 | 8493.6 | 1016.4 | 1501.9 | 1139.5 | 938.1 |

Nr_{12} | 923.5 | 8508.8 | 1066.5 | 1241.5 | 1235.8 | 928.4 |

Calculus | Intellig_{1} | Intellig_{2} | Intellig_{3} | Intellig_{4} | Intellig_{5} | Intellig_{6} |
---|---|---|---|---|---|---|

Mean | 3872.36 | 5153.93 | 3393.04 | 1995.53 | 1704.47 | 3358.73 |

SD | 3265.3 | 3377.76 | 3494.04 | 2073.47 | 1904.44 | 3132.09 |

Median | 2175.95 | 6817.65 | 1100.9 | 1430.3 | 1187.65 | 1073.75 |

Kurt | −2.1 | −1.9 | −1.4 | 11.9 | 11.9 | −1.6 |

Skew | 0.3 | −0.4 | 0.9 | 3.4 | 3.4 | 0.7 |

Min | 920.3 | 952 | 924.4 | 1241.5 | 963.1 | 928.4 |

Max | 8382.7 | 8642.3 | 8921.5 | 8568 | 7738.3 | 8213.8 |

range | 7462.4 | 7690.3 | 7997.1 | 7326.5 | 6775.2 | 7285.4 |

CV | 84.3 | 65.5 | 102 | 103.9 | 111.7 | 93.3 |

hom.-het. | het. | het. | het. | het. | het. | het. |

Calculus | Intellig_{1} | Intellig_{2} | Intellig_{3} | Intellig_{4} | Intellig_{5} | Intellig_{6} |
---|---|---|---|---|---|---|

Lill Test | ||||||

Lill statistic | 0.309 | 0.255 | 0.407 | 0.489 | 0.492 | 0.349 |

Pnorm (calculated p-value) | 0.002 | 0.03 | ~0 | ~0 | ~0 | ~0 |

Normality passed (Pnorm > αNorm) | No | No | No | No | No | No |

SW Test | ||||||

SW statistic | 0.753 | 0.79 | 0.666 | 0.381 | 0.389 | 0.717 |

Pnorm (calculated p-value) | 0.003 | 0.007 | ~0 | ~0 | ~0 | 0.001 |

Normality passed (Pnorm > αNorm) | No | No | No | No | No | No |

Type of Calculus | Obtained Result |
---|---|

mean/SD/median | 2297.7/2252.1/1308.98 |

Skew/interpretation | 2.28/highly skewed |

kurt/interpretation | 5.28/leptokurtic |

CV/hom-het | 98/heterogeneous |

Performed Calculus | Lill Test | SW Test |
---|---|---|

Statistic | 0.355 | 0.636 |

Pnorm (calculated p-value) | 0.017 | 0.001 |

Normality passed (Pnorm > αNorm) | No | No |

ICMAS | MIQ | Extreme */Furthest ^{#} | No.Id ^{@} | Type ^{&} |
---|---|---|---|---|

ICM_{1} | 2175.95 | No/Yes | 2 | Low |

ICM_{2} | 6817.65 | Yes/No | 1 | Low |

ICM_{3} | 1100.9 | / | ||

ICM_{4} | 1430.3 | No/Yes | 3 | Low |

ICM_{5} | 1187.65 | No/Yes | 4 | Low |

ICM_{6} | 1073.75 | / |

^{#}indicates an ICMAS intelligence that is not significantly extreme, but is further removed from the intelligence of the rest of ICMASs;

^{@}indicates the application number of the extreme detection test;

^{&}indicates the type of intelligence “Low” or “High”.

© 2019 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (http://creativecommons.org/licenses/by/4.0/).

## Share and Cite

**MDPI and ACS Style**

Iantovics, L.B.; Kountchev, R.; Crișan, G.C.
ExtrIntDetect—A New Universal Method for the Identification of Intelligent Cooperative Multiagent Systems with Extreme Intelligence. *Symmetry* **2019**, *11*, 1123.
https://doi.org/10.3390/sym11091123

**AMA Style**

Iantovics LB, Kountchev R, Crișan GC.
ExtrIntDetect—A New Universal Method for the Identification of Intelligent Cooperative Multiagent Systems with Extreme Intelligence. *Symmetry*. 2019; 11(9):1123.
https://doi.org/10.3390/sym11091123

**Chicago/Turabian Style**

Iantovics, László Barna, Roumen Kountchev, and Gloria Cerasela Crișan.
2019. "ExtrIntDetect—A New Universal Method for the Identification of Intelligent Cooperative Multiagent Systems with Extreme Intelligence" *Symmetry* 11, no. 9: 1123.
https://doi.org/10.3390/sym11091123