A Fuzzy Inference System for Players Evaluation in Multi-Player Sports: The Football Study Case

Abstract

Decision support systems often involve taking into account many factors that influence the choice of existing options. Besides, given the expert’s uncertainty on how to express the relationships between the collected data, it is not easy to define how to choose optimal solutions. Such problems also arise in sport, where coaches or players have many variants to choose from when conducting training or selecting the composition of players for competitions. In this paper, an objective fuzzy inference system based on fuzzy logic to evaluate players in team sports is proposed on the example of football. Based on the Characteristic Objects Method (COMET), a multi-criteria model has been developed to evaluate players on the positions of forwards based on their match statistics. The study has shown that this method can be used effectively in assessing players based on their performance. The COMET method was chosen because of its unique properties. It is one of the few methods that allow identifying the model without giving weightings of decision criteria. Symmetrical and asymmetrical fuzzy triangular numbers were used in model identification. Using the calculated derivatives in the point, it turned out that the criteria weights change in the problem state space. This prevents the use of other multi-criteria decision analysis (MCDA) methods. However, we compare the obtained model with the Technique of Order Preference Similarity (TOPSIS) method in order to better show the advantage of the proposed approach. The results from the objectified COMET model were compared with subjective rankings such as Golden Ball and player value.

1. Introduction

1.1. Theoretical Underpinning

The prediction of the results in football matches is a difficult problem because of multiple variables [1,2]. Both supporters and coaches try to predict matches, but despite the many available solutions, it is still not successfully done with high accuracy. The correct prediction of a match result is associated with the satisfaction of football enthusiasts or coaches and huge money [3]. Clubs try to provide the best possible analysis of players to help the coach choose the winning line-up. The team’s good results may guarantee a large amount of money not only from the league but also from international championships. The fans also try to create such systems by themselves. It helps them identify the winner of a football game because betting might be a fast and profitable way to earn money if the predicted result is correct. Incorrect types can result in a significant loss, so when creating prediction systems for match results, it is worthy of analyzing specific data and using the various amount of methods for receiving more reliable solutions [4].

To better understand why so many people and clubs are trying to develop effective predicting football matches, it is enough to look at the amounts traded by the major European football clubs. According to a report prepared by Deloitte [5], in the 2016–2017 season, Manchester United was the football club with the highest revenues, earning 676 million euro and the revenues of the top 20 largest teams were as high as 7.9 billion euro. Most clubs are getting paid from television broadcasts, but results in European competitions are equally important. For winning the European League, Manchester United received 44.5 million euros from the organizers, so the club managed to overtake the second in this ranking, Real Madrid, by 1.7 million euros. Only the most influential teams from Europe take part in the European competitions, so to achieve success there and earn a lot of money you need not only good players but also proper analysis of opponents and evaluation of their skills. It shows that football clubs are a great way to earn money, but this is only possible if you are successful [6]. The better the club’s results, the more people will buy tickets to the matches, watch the games on TV, and buy club gadgets. Hence, it is necessary to use all available methods to win matches. That is why many clubs hire experts and researchers to analyze their rivals, evaluate them, and try to determine the best team composition, which will give them the victory in a duel with the opponent [7].

Another reason for creating match prediction systems may be betting. According to the European Gaming and Betting Association European Gaming and Betting Association (EGBA) [8], in 2012, profits from legal betting accounted for 14% ($58 billion) of all gambling activities, and benefits from illegal betting (especially in Asia) are expected to be many times higher [9]. In Poland, in 2017, the turnover of legal bookmakers amounted to 3.3 billion PLN, which constitutes about 40% of the mutual betting market in Poland [10]. On the other hand, the Gemius report [11] shows that in June 2018, Polish online bookmakers had almost 4.7 million users. It explains how many people use the services of bookmakers trying to get rich. Some people look at the form of their favorite teams, others hope for luck, but many use different systems and methods [12].

Popular methods include the Federation Internationale de Football Association (FIFA) ranking or the process using the Poisson distribution. The FIFA Ranking ranks national teams associated with the International Football Federation [13]. Criteria such as the importance of the game (friendly match, eliminations, world championships), the result of the game, and the difference in points between the teams are considered [14]. It can be used to determine the winner of a match based on its ranking. However, only national teams are listed there, and the algorithm for classifying teams consults a small number of criteria [15].

1.2. Methodical Background

The selection, skills assessment, and prediction of players’ performance in competitions are the issues on which all coaching teams work. The best sport clubs employ a large number of people responsible for player analysis, focusing on many aspects such as skills and physical conditions. However, many of them still have problems choosing the best line-up and correct evaluation. Therefore, this section will present different approaches from different sports disciplines [16,17].

The Poisson Schedule can be used to determine the winner of the match. First step in system application was to calculate the attack strength and defense for both teams. For this, the information about the number of scored and lost goals was used. Subsequent steps required calculation of the average amount of probably scored goals by each teams. Based on this result, the usage of Poisson distribution allows to determine the probability of scoring goals by clubs [18]. This approach take into consideration only scored and lost goals, while many other criteria are omitted.

The preceding analysis of football club revenues and bookmakers’ turnover, as well as sample systems, shows how much demand there is for reliable systems and methods of predicting match results. The presented systems have their advantages and disadvantages. Each of them emphasizes a different aspect and contains different criteria. The analysis and evaluation of the players are carried out differently and predicts the winner of the match. In this paper, it was decided to use multi-criteria decision analysis (MCDA) methods [19] to create a multi-criteria expert model based on the possibility of analyzing many criteria, including incompatible ones [20,21].

Some of these obstacles were solved using multi-criteria decision support methods. One of the proposed solutions is a two-part approach to the selection of players. Firstly, using the AHP method [22], each player’s attributes are ranked according to their importance in a given position. Afterward, a linear programming model is created using attribute weights to determine the best players to include in the team’s composition [23]. The AHP method was also used to solve the problem of granting football prizes. It was used for intuitive and accurate reasons and was tested to select the winner of one of the most prestigious football awards: the Golden Ball in 2014. Criteria such as an attack, defense, and fair play were included [24]. It is also possible to select players and set up a formation using the fuzzy inference system. In the first phase, the players were assessed, and the best players of the team were selected. In the second phase, alternative player combinations were evaluated using the fuzzy inference system, and the best player combinations for the respective formation were chosen. The task of selecting the best players was also executed using the Data Envelopment Analysis (DEA) method [25]. It was used to create a model and then to classify players, who were playing in the English league. The Technique of Order Preference Similarity (TOPSIS) method was used to determine the best group of players from among 24 young players. They were divided into four groups of six and then tested using anthropometric methods, fitness exercises, and football skills tests.

The solution to team classification was the usage of two methods: AHP and TOPSIS, as well as data from the highest German gameplay class. The reliability of the results was checked using Spearman’s Rank Correlation Ratio and Kendall’s Tau Kendall Ratio [26]. The AHP method was also used to predict the position of teams in the course of the competition in the Israeli league. Six attributes were selected: team level, coach skills, football fans’ engagement, team object, previous season’s achievements, and current disposition. Then for each of them, a pair comparison matrix and a final ranking of the teams were created [27].

Predicting the result of a match is an intriguing problem because of its difficulty caused by the need to analyze many factors such as team morale, players’ skills, or the results of previous games [1]. With the usage of a neural network, results have been obtained, which show that machine learning has great potential in predicting the effects of football matches. It has been done using match data from seven years of the Iranian league [28].

The other problem that can be solved with multi-criteria decision support methods is the selection of new players [29]. A support model for creating the right team may help by computing the impact of new players on the existing team. It has been tested under two different conditions: football, where team formation is not limited, and volleyball, where the relationships between the positions and the players are significant [30]. The problem of choosing new players was also solved using an ordered weighted average (OWA), which enabled the team selection of the best players for the team [31].

Methods of multi-criteria decision support were also used in cricket because it is a very demanding sport due to the necessity of a good play of players in many positions and a limited budget of teams. The NSGA-II algorithm was used to determine the optimal set of players. Among other things, the ability to collect balls, and the strength of the blows were considered. As a source of data was used performances of players from the cricket league in India [32]. The players’ evaluation was also achieved using linear binary programming to select the best team in the American cricket league [33].

Similarly, total number programming was used to create the best team out of 32 players from South Africa. During the development of the method the skills of throwing, catching, and collecting balls were examined [34]. The DEA method was also useful when trying to select the optimal cricket team. A new phrase was proposed to evaluate players’ performance and rank them in terms of the utility of their abilities in the team. Then the method was used to create the best team from the Indian cricket league. This approach has the prevalence of other methods because of the consideration of many factors related to the cricket players’ performance [35].

The genetic algorithm was also used to design a system to identify the right players. Various performance data were selected to create a balanced team [36]. The two-phase approach was made of the measurement of the performances of batsmen in the Indian cricket league. First, the TOPSIS method was used to assess the skills of the players. The calculation of the weights for the criteria using the AHP method was developed, and the variation analysis (ANOVA) was used to measure the impact of each criterion’s. In that way, modified weights of all criteria were obtained, and batsmen’s presence was evaluated [37]. The performance of the bowlers was similarly evaluated. TOPSIS and AHP methods were used, and among the criteria considered were strength, speed, and accuracy of throws [38]. The TOPSIS and AHP methods were reused with the Weighted Factors Analysis (WeFA) approach to pick the best players. The AHP method was practiced to determine the weights of the selected criteria and TOPSIS to determine the ranking of alternatives [39].

Multi-criteria decision support has been used in many other sports disciplines. In the case of evaluation players in badminton, the Delphic method was used to validate 17 indicators such as the player’s body structure, physical characteristics, and intelligence. After that, the AHP method was applied to determine the importance of potential players’ indicators for badminton coaches. In this way, a model for the selection of the best players was created [40]. In basketball, the most popular rating system for players is based on performance statistics, which are considered by many to be unbiased. The establishment of a TOPSIS-based system was provided that assures better performance in the evaluation of players and teams, optimization of players training for versatility, and more accurate sports performance predictions [41]. The selection of the right candidates for basketball players is also a demanding process due to qualitative and quantitative attributes. The TOPSIS method can help create a model for identifying future players from a group of young players aged 7 to 14. Measurements of physical fitness and technical skills were used to determine the weighting of the chosen criteria. Then the measured values were converted into fuzzy values using fuzzy sets. Finally, a ranking of players was generated and compared with the opinions of sports experts, which confirmed the reliability of the model [42].

The support system for selecting players based on their performances is made on the grounds of the data from the Spanish basketball league. The proposal of the method to summarize a large amount of data was given, which allows coaches to check the strengths and weaknesses of players and build strategies to increase team performance [43]. Additionally, in rugby, there is a lot of information about players and teams, which is still not fully exploited. Based on this data, the performance evaluation system was created which considers the uncertainties of both rating and the preferences of some aspects of the game over other [44].

The selection of beginning pitchers in baseball has a significant influence on the team’s result. The model using the AHP method to determine the weights of the selected criteria may be facilitative. Besides, the TOPSIS method was used to create the final ranking of the pitchers [45]. The AHP method is also used in volleyball to assign skill weights to players and positions on the pitch to select the right players for the team [46]. It is very complex to choose the most relevant players and their positions from all available to the coach by many combinations, even if the team consists of very few players. A mathematical model is proposed, which helps create the best team taking into account the strategy and harmony of the team, players’ skills, and the skills of the next opponent [47].

1.3. Aim of the Study

The previous section presented different approaches and attempts to solve the analysis of the game and evaluation of teams and players using decision support methods. In the process of determining the winner of the game, it is necessary to create a model analyzing the players’ play in all positions in the team. Once all the players have been assessed, they can be compared with the opposing team, and the winner can be predicted. In this work, it was decided to create a model for the assessment of strikers, because they are the main contributors to the goals scored; however, it should be noticed that it is a team game and each position is important. It is the preliminary work which can be used to develop a full assessment of the whole team, as a larger structured model rather than an ordinary sum. Achievements are the main reason for investing money in football, and it is thanks to victories that people decide to play in bookmaking betting. Creating a forecasting winning system would be very useful for both football clubs and fans. However, the work is limited to assessing the play of the attackers using a selection of 17 decision criteria. Based on which the characteristics of the model will be presented, the aim of which is to objectify the assessment.

In this paper, the COMET method is used because of the number of their benefits [48,49]. The COMET method works based on a fuzzy inference system, and this approach has used in team sport players assessment in [50,51,52]. This technique does not require the weighting values for decision criteria [53,54]. Instead, it requires the decision maker to fill in the matrix comparisons in pairs, where only 3 values are used [55]. This method can be used both in a monolithic and structural version [56], which takes into account the hierarchy of criteria as in the AHP method. This allows a significant reduction of queries needed for the expert. Using this method we obtain a full continuous model in a given field, so that it is possible to count the derivatives at a point, which allow to analyze the relevance of the criteria at a given point [57]. Finally, it should be stated that the model identified by the COMET method is fully resistant to the rank reversal phenomenon [58], because the final assessment does not depend on the chosen set of alternatives, but on the unchanging set of characteristic objects [48] and does not require the use of normalization techniques that affect the differences in final results in many popular MCDA methods [59,60,61,62].

The paper is organized as follows: in Section 2, some basic definitions are provided to facilitate the paper understanding. Fuzzy sets theory preliminary is presented in Section 2.1. A detailed description of the COMET method is presented in Section 2.2. Ranking similarity coefficients are presented in Section 2.3. Section 3 describes step by step the identification process of decision support system for players evaluation. In Section 4 we show the comparison of rankings obtained with the help of our model and two subjective ranks, i.e., Golden Ball and player value. Section 5 includes some conclusions and future research challenges.

2. Preliminaries

2.1. Fuzzy Sets Theory Preliminary

The fuzzy set theory has a very large number of practical implementations, where the mechanisms of traditional logic proved to be insufficient [63,64,65,66]. Many decision support methods use fuzzy data directly [67,68,69]. Moreover, many more advanced methods use fuzzy set generalizations such as hesitant fuzzy sets [70,71], intuitionistic fuzzy sets [72,73], interval-valued fuzzy numbers [74,75,76], type-2 fuzzy sets [77,78] and others [79,80,81]. This section presents the necessary definitions that are used in the COMET method [82].

Definition 1.

The fuzzy set and the membership function—the characteristic function ${\mu }_A$ of a crisp set $A\subseteq X$ assigns a value of either 0 or 1 to each member of X, and the crisp sets only allow a full membership $({\mu }_A\left(x\right)=1)$ or nonmembership at all $({\mu }_A\left(x\right)=0)$. This function can be generalized to a function ${\mu }_{\tilde{A}}$ so that the value assigned to the element of the universal set X falls within a specified range, i.e., ${\mu }_{\tilde{A}}:X\to [0,1]$. The assigned value indicates the degree of membership of the element in the set A. The function ${\mu }_{\tilde{A}}$ is called a membership function and the set $\tilde{A}=\left\{(x,{\mu }_{\tilde{A}}\left(x\right))\right\}$, where $x\in X$, defined by ${\mu }_{\tilde{A}}\left(x\right)$ for each $x\in X$ is called a fuzzy set.

Definition 2.

The triangular fuzzy number (TFN)—a fuzzy set A, defined on the universal set of real numbers $\mathbb{R}$, is said to be a TFN A(a,m,b) if its membership function has the following form (1), and Figure 1 gives an example of the TFN.

$${\mu }_{\tilde{A}}(x,a,m,b)=\left\{\begin{array}{cc}0\hfill & x\le a\hfill \\ \frac{x-a}{m-a}\hfill & a\le x\le m\hfill \\ 1\hfill & x=m\hfill \\ \frac{b-x}{b-m}\hfill & m\le x\le b\hfill \\ 0\hfill & x\ge b\hfill \end{array}\right.$$

(1)

The following properties are observed (2):

$$\begin{array}{c}{x}_1,x_2\in [a,m]\wedge x_2>x_1\Rightarrow {\mu }_{\tilde{A}\left(x_2\right)}>{\mu }_{\tilde{A}\left(x_1\right)}\hfill \\ x_1,x_2\in [m,b]\wedge x_2>x_1\Rightarrow {\mu }_{\tilde{A}\left(x_2\right)}<{\mu }_{\tilde{A}\left(x_1\right)}\hfill \end{array}$$

(2)

Figure 1. Visualization of the triangular fuzzy number $\tilde{A}$(a, m, b).

Definition 3.

The support of a TFN $\tilde{A}$ is the crisp subset of the set $\tilde{A}$ whose all elements have nonzero membership values in the set $\tilde{A}$:

$$S\left(\tilde{A}\right)=\left\{x:{\mu }_{\tilde{A}}\left(x\right)>0\right\}=(a,b)$$

(3)

Definition 4.

The core of a TFN $\tilde{A}$ is a singleton with the membership value equal to 1. The core of a TFN $\tilde{A}$ we formally write as

$$C\left(\tilde{A}\right)=\left\{x:{\mu }_{\tilde{A}}\left(x\right)=1\right\}=m$$

(4)

Definition 5.

The fuzzy rule can be based on the Modus Ponens tautology. The premise input is A and the consequent output is B can be true to a degree, instead of entirely false or entirely true. The reasoning process uses the IF–THEN, OR and AND logical conjunction.

Definition 6.

The rule base (linguistic model) is called a set of fuzzy rules consists of logical rules determining the causal relationships existing in the system between the input and output fuzzy variables.

Definition 7.

The T-norm operator (product) is a function T modelling the intersection operation AND of two or more fuzzy numbers. This operator is a generalization of the usual two-valued logical conjunction for fuzzy logics.

$${\mu }_{\tilde{A}}\left(x\right)AND{\mu }_{\tilde{B}}\left(y\right)={\mu }_{\tilde{A}}\left(x\right)·{\mu }_{\tilde{B}}\left(y\right)$$

(5)

Definition 8.

The S-norm operator (union), or T-conorm is an S function modelling the OR union operation of two or more fuzzy numbers. This operator is a generalization of the usual two-valued logical union for fuzzy logics.

$${\mu }_{\tilde{A}}\left(x\right)OR{\mu }_B\left(y\right)=\left({\mu }_{\tilde{A}}\left(x\right)+{\mu }_B\left(y\right)\right)\wedge 1$$

(6)

2.2. The COMET Method

The COMET is a newly developed method for identifying a multi-criteria expert decision-making model to solve decision-making problems. Work on the basic version of the method, allowing for individual expert decisions, was completed in [49]. The COMET method has unique properties that are rare in the field of multi-criteria decision-making methods. First of all, the resistance to the COMET rank reversal paradox should be mentioned [58]. This property results from the fact that the COMET method evaluates alternatives using a model identified based on characteristic objects, which are independent of the set of assessed decision alternatives [83]. It means that unlike many other methods of multi-criteria decision analysis, the assessed alternatives are not compared with each other, and the result of their assessment is concluded only based on the obtained model. Therefore, if we use the same decision-making model, the values of assessments for alternatives will not change, so the mentioned paradox will never occur [58].

The decision model defines the assessment pattern for all decision options in the given space of the problem state, which can be compared to measuring the length of an object using a predefined pattern and not comparisons between measured objects [84]. The identification of the decision model allows additionally to assess any set of alternatives in the given numerical space without re-engaging the expert in the assessment process, as the model is identified in the continuous space [85]. Competitive methods in such situations most often require repetition of the whole identification and calculation procedure from the beginning, because they identify only the assessment values for the currently considered set of alternatives, and not the whole space of the problem state [48].

The COMET method also allows for relatively easy identification of both linear and non-linear human decision-making functions, which allows increasing its applicability to solve both linear and non-linear problems [86]. Another issue is the use of global criterion weights, which determine the average significance of a given criterion for the final assessment. The higher the weighting, the more relevant the criterion is on average. Linear inclusion of weights in non-linear problems leads additionally to a decrease in the accuracy of obtained results. Apart from that, the problem is how such weights should be determined. Therefore, in the calculation procedure of the COMET method, there is no arbitrary determination of weights for individual criteria. Recently, there have also been some interesting developments related to the hesitant [87], intuitionistic [88] and interval valued fuzzy set [76] extensions. In this study we have limited ourselves to the basic version of the method as a preliminary study. The whole decision-making process by using the COMET method is presented in Figure 2. The formal notation of this method can be presented using the following five steps.

Figure 2. The procedure of the Characteristic Objects Method (COMET) to identify decision-making model.

Step 1.

Define the space of the problem—an expert determines dimensionality of the problem by selecting number r of criteria, $C_1,C_2,\dots ,C_r$. Subsequently, the set of fuzzy numbers for each criterion $C_i$ is selected, i.e., ${\tilde{C}}_{i1},{\tilde{C}}_{i2},\dots ,{\tilde{C}}_{i{c}_i}$. Each fuzzy number determines the value of the membership for a particular linguistic concept for specific crisp values. Therefore it is also useful for variables that are not continuous. In this way, the following result is obtained (7).

$$\begin{array}{c}{C}_1=\{{\tilde{C}}_{11},{\tilde{C}}_{12},\dots ,{\tilde{C}}_{1c_1}\}\hfill \\ C_2=\{{\tilde{C}}_{21},{\tilde{C}}_{22},\dots ,{\tilde{C}}_{2c_1}\}\hfill \\ \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \hfill \\ C_r=\{{\tilde{C}}_{r1},{\tilde{C}}_{r2},\dots ,{\tilde{C}}_{r{c}_r}\}\hfill \end{array}$$

(7)

where $c_1,c_2,\dots ,c_r$ are numbers of the fuzzy numbers for all criteria.

Step 2.

Generate the characteristic objects—characteristic objects are objects that define reference points in n-dimensional space. They can be either real or idealized objects that cannot exist. The characteristic objects ($CO$) are obtained by using the Cartesian product of fuzzy numbers cores for each criteria as follows (8):

$$CO=\{\{C\left({\tilde{C}}_{11}\right),C\left({\tilde{C}}_{12}\right),\dots ,C\left({\tilde{C}}_{1c_1}\right)\}\times \dots \times \{C\left({\tilde{C}}_{r1}\right),C\left({\tilde{C}}_{r2}\right),\dots ,C\left({\tilde{C}}_{r{c}_r}\right)\}\}$$

(8)

As the result, the ordered set of all $CO$ is obtained (9):

$$\begin{array}{c}C{O}_1=\{C\left({\tilde{C}}_{11}\right),C\left({\tilde{C}}_{21}\right),\dots ,C\left({\tilde{C}}_{r1}\right)\}\hfill \\ C{O}_2=\{C\left({\tilde{C}}_{11}\right),C\left({\tilde{C}}_{21}\right),\dots ,C\left({\tilde{C}}_{r2}\right)\}\hfill \\ \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \dots \hfill \\ C{O}_t=\{C\left({\tilde{C}}_{1c_1}\right),C\left({\tilde{C}}_{2c_2}\right),\dots ,C\left({\tilde{C}}_{r{c}_r}\right)\}\hfill \end{array}$$

(9)

where t is a number of $CO$ (10):

$$t=\prod _{i=1}^r{c}_i$$

(10)

Step 3.

Rank the characteristic objects—the expert determines the Matrix of Expert Judgement ($MEJ$). It is a result of pairwise comparison of the characteristic objects by the expert knowledge. The $MEJ$ structure is as follows (11):

$$MEJ=\left(\begin{array}{cccc}{\alpha }_{11}& {\alpha }_{12}& \dots & {\alpha }_{1t}\\ {\alpha }_{21}& {\alpha }_{22}& \dots & {\alpha }_{2t}\\ \dots & \dots & \dots & \dots \\ {\alpha }_{t1}& {\alpha }_{t2}& \dots & {\alpha }_{tt}\end{array}\right)$$

(11)

where ${\alpha }_{ij}$ is a result of comparing $C{O}_i$ and $C{O}_j$ by the expert. The more preferred characteristic object gets one point and the second object get zero points. If the preferences are balanced, the both objects get half point. It depends solely on the knowledge of the expert and can be presented as (12):

$${\alpha }_{ij}=\left\{\begin{array}{cc}0.0,\hfill & f_{exp}\left(C{O}_i\right)<f_{exp}\left(C{O}_j\right)\hfill \\ 0.5,\hfill & f_{exp}\left(C{O}_i\right)=f_{exp}\left(C{O}_j\right)\hfill \\ 1.0,\hfill & f_{exp}\left(C{O}_i\right)>f_{exp}\left(C{O}_j\right)\hfill \end{array}\right.$$

(12)

where $f_{exp}$ is an expert mental judgement function. Afterwards, the vertical vector of the Summed Judgements ($SJ$) is obtained as follows (13):

$$S{J}_i=\sum _{j=1}^t{\alpha }_{ij}$$

(13)

The number of query is equal $p=\frac{t(t-1)}{2}$ because for each element ${\alpha }_{ij}$ we can observe that ${\alpha }_{ji}=1-{\alpha }_{ij}$. The last step assigns to each characteristic object an approximate value of preference $P_i$ by using the following Matlab pseudo-code:

1: k = length(unique(SJ));

2: P = zeros(t, 1);

3: for i = 1:k

4:      ind = find(SJ == max(SJ));

5:      p(ind) = (k - i)/(k - 1);

6:      SJ(ind) = 0;

7: end

In the result, the vector P is obtained, where i-th row contains the approximate value of preference for $C{O}_i$.

Step 4.

The rule base—each characteristic object is converted into a fuzzy rule, where the degree of belonging to particular criteria is a premise for activating conclusions in the form of $P_i$. Each characteristic object and value of preference is converted to a fuzzy rule as follows detailed form (14). In this way, the complete fuzzy rule base is obtained, that approximates the expert mental judgement function $f_{exp}\left(C{O}_i\right)$.

$$\begin{array}{cccccccc}IF\hfill & C_1˜{\tilde{C}}_{1i}\hfill & AND\hfill & C_2˜{\tilde{C}}_{2i}\hfill & AND\hfill & \dots \hfill & THEN\hfill & P_i\hfill \end{array}$$

(14)

Step 5.

Inference and final ranking—The each one alternative $A_i$ is a set of crisp numbers $a_{ri}$ corresponding to criteria $C_1,C_2,\dots ,C_r$. It can be presented as follows (15):

$$A_i=\{a_{1i},a_{2i},\dots ,a_{ri}\}$$

(15)

Each alternative activates the specified number of fuzzy rules, where for each one is determined the fulfilment degree of the complex conjunctive premise. Fulfilment degrees of all activated rules are summed to one. The preference of alternative is computed as the sum of the product of all activated rules, as their fulfilment degrees, and their values of the preference. The final ranking of alternatives is obtained by sorting the preference of alternatives, where one is the best result, and zero is the worst. More details can be found in [89].

TOPSIS

In Technique of Order Preference Similarity (TOPSIS), we measure the distance of alternatives from the reference elements, which are respectively positive and negative ideal solution. This method was widely presented in [62,90,91]. The TOPSIS method is a simple MCDA technique used in many practical problems. Thanks to its simplicity of use, it is widely used in solving multi-criteria problems. Below we present its algorithm [60,91]. We assume that we have a decision matrix with m alternatives and n criteria is represented as $X={\left(x_{ij}\right)}_{m\times n}$.

Step 1.

Calculate the normalized decision matrix. The normalized values $r_{ij}$ calculated according to Equation (16) for profit criteria and (17) for cost criteria. We use this normalization method, because [62] shows that it performs better than classical vector normalization. Although, we can also use any other normalization method.

$$r_{ij}=\frac{x_{ij}-mi{n}_j\left(x_{ij}\right)}{ma{x}_j\left(x_{ij}\right)-mi{n}_j\left(x_{ij}\right)}$$

(16)

$$r_{ij}=\frac{ma{x}_j\left(x_{ij}\right)-x_{ij}}{ma{x}_j\left(x_{ij}\right)-mi{n}_j\left(x_{ij}\right)}$$

(17)

Step 2.

Calculate the weighted normalized decision matrix $v_{ij}$ according to Equation (18).

$$v_{ij}=w_i{r}_{ij}$$

(18)

Step 3.

Calculate Positive Ideal Solution (PIS) and Negative Ideal Solution (NIS) vectors. PIS is defined as maximum values for each criteria (19) and NIS as minimum values (20). We do not need to split criteria into profit and cost here, because in step 1 we use normalization which turns cost criteria into profit criteria.

$$v_j^{+}=\{v_1^{+},v_2^{+},\cdots ,v_n^{+}\}=\left\{ma{x}_j\left(v_{ij}\right)\right\}$$

(19)

$$v_j^{-}=\{v_1^{-},v_2^{-},\cdots ,v_n^{-}\}=\left\{mi{n}_j\left(v_{ij}\right)\right\}$$

(20)

Step 4.

Calculate distance from PIS and NIS for each alternative. As shown in Equations (21) and (22).

$$D_i^{+}=\sqrt{\sum _{j=1}^n{(v_{ij}-v_j^{+})}^2}$$

(21)

$$D_i^{-}=\sqrt{\sum _{j=1}^n{(v_{ij}-v_j^{-})}^2}$$

(22)

Step 5.

Calculate each alternative’s score according to Equation (23). This value is always between 0 and 1, and the alternatives which have values closer to 1 are better.

$$C_i=\frac{D_i^{-}}{D_i^{-}+D_i^{+}}$$

(23)

2.3. Ranking Similarity Coefficients

Ranking similarity coefficients allow to compare obtained results and determine how similar they are [60]. The most popular are Spearman rank correlation coefficient (24), weighted Spearman correlation coefficient (26), and rank similarity coefficient WS (27) [92].

2.3.1. Spearman’s Rank Correlation Coefficient

Rank values $x_i$ and $y_i$ are defined as (24). However, if we are dealing with rankings where the values of preferences are unique and do not repeat themselves, each variant has a different position in the ranking, the Formula (25) can be used [93].

$$r_s=\frac{cov(x_i,y_i)}{{\sigma }_{x_i}{\sigma }_{y_i}}$$

(24)

$$r_s=1-\frac{6·{\sum }_{i=1}^N(x_i-y_i)}{N(N^2-1)}$$

(25)

2.3.2. Weighted Spearman’s Rank Correlation Coefficient

For a sample of size N, rank values $x_i$ and $y_i$ are defined as (26). In this approach, the positions at the top of both rankings are more important. The weight of significance is calculated for each comparison. It is the element that determines the main difference to the Spearman’s rank correlation coefficient, which examines whether the differences appeared and not where they appeared [94].

$$r_w=1-\frac{6{\sum }_{i=1}^N{(x_i-y_i)}^2((N-x_i+1)+(N-y_i+1))}{N^4+N^3-N^2-N}$$

(26)

2.3.3. Rank Similarity Coefficient

For a samples of size N, the rank values $x_i$ and $y_i$ is defined as (27) [92]. It is an asymmetric measure. The weight of a given comparison is determined based on the significance of the position in the first ranking, which is used as a reference ranking during the calculation [95].

$$WS=1-\sum _{i=1}^N{2}^{-x_i}\frac{|x_i-y_i|}{max\left(\right|x_i-1|,|x_i-N\left|\right)}$$

(27)

3. Decision Support System

This section proposes an objectified approach to evaluating attackers using the COMET method. The usage of one of these method advantages—the possibility of applying a hierarchical structure—significantly accelerated the model’s construction. Independence of rank reversal paradox by comparing characteristic objects rather than the assessed alternatives prevented additional problems when adding further alternatives. To create a model, it was necessary to select the most important criteria determining the effectiveness of a player on the position of an attacker. In the second step, they had to be grouped into categories aggregating the related criteria. Therefore, the number of characteristic objects and the number of queries was possible to reduce.

Compared with strikers, it is necessary to create a model of assessment of a player, considering different characteristics and useful skills during the match. For this purpose, a group of 10 students was selected from 100 volunteers. The most knowledgeable people were engaged, who were themselves professionals in soccer. The filling of the $MEJ$ matrix took place on a voting basis. This was to reduce uncertainty in individual responses. To identify the model, the 17 most essential criteria for strikers were selected from the many potential criteria and divided into five categories. The Figure 3 presents proposed structure to assess strikers. The following determinations were used in the model:

Figure 3. The hierarchical structure of the attackers ranking assessment model.

• $P_1$—Metrics assessment;
• $P_2$—Ball possession assessment;
• $P_3$—Offensiveness assessment;
• $P_4$—Technique assessment;
• $P_5$—Penalty assessment.

The final model will be created by applying five smaller models built according to the hierarchy shown in Figure 3. Without it, the preceding criteria generate 129,140,163 characteristic objects and 8,338,590,785,263,203 queries. Thanks to the application of a hierarchical structure, the number of characteristic objects of the final model decreased to only 32 objects and the number of queries to 496. The sample evaluation for the match is determined for the players presented in Table 1. It should be noted that the contestant himself is not subject to assessment as many as 17 selected parameters. Thus, the score is the rating for a specific set of attributes for a specific match.

Table 1. Specification of the players for the evaluation example for the match.

Personal Details	${\mathit{A}}_{\mathit{i}}$	Match	Date [d.m.y]
Antoine Griezmann	$A_1$	Sevilla 2:5 Atletico Madrid	25.02.2018
Lionel Messi	$A_2$	Barcelona 2:0 Athletic Bilbao	18.03.2018
Loren Moron	$A_3$	Alaves 1:3 Real Betis	12.03.2018
Cristiano Ronaldo	$A_4$	Real Madrid 6:3 Girona	18.03.2018
Franco Vazquez	$A_5$	Sevilla 2:0 Athletic Bilbao	03.03.2018
Simone Zaza	$A_6$	Valencia 3:1 Alaves	17.03.2018

3.1. Metrics

The first of the category is the metric of height, weight, and age. The height may be decisive for winning head fights, but it also affects the player’s agility. Lower players can better keep the ball close and faster change the direction of the run [96]. Weight affects the strength, stamina, and speed of a player. A higher weight means not only slower movement but also greater strength in duels with an opponent. Age, on the other hand, determines the possibility of a player’s development but also shows his experience. Younger players can learn new techniques, while older players can use their experience [97]. These criteria are marked as follows:

• $C_1$—height of a player (in centimeters), where $C_1\in [160,210]$;
• $C_2$—weight of a player (in kilograms), where $C_2\in [50,100]$;
• $C_3$—age of a player (in years), where $C_3\in [18,40]$.

For each of the linguistic variables, one characteristic value was additionally defined as the average value of a given characteristic set. It means that it was average value form gathering data (based on whoscored.com; year 2017; only players in striker’s position). In this way, the linguistic variables were identified, and the corresponding triangular fuzzy numbers, which are shown in Figure 4.

Figure 4. The visualization of the linguistic variables and fuzzy triangular numbers for the metric assessment module ($P_1$).

Based on triangular numbers, 27 characteristic objects were defined, and then, after 351 pairwise comparisons, a matrix of MEJ expert assessments was created (all MEJ matrices are presented in Appendix B), taking the following form (A1). The number of pairwise comparisons is because only the upper triangular matrix should be filled in. Based on the identified MEJ matrix, the value of vector SJ and vector P is calculated, which are presented in detail in Table A1. An illustrative assessment of the metric is shown in Table 2.

Table 2. Overview of the players’ characteristics and rating $P_1$.

${\mathit{A}}_{\mathit{i}}$	${\mathit{C}}_1$	${\mathit{C}}_2$	${\mathit{C}}_3$	${\mathit{P}}_1$
$A_1$	175	71	27	0.7929
$A_2$	170	72	30	0.6087
$A_3$	188	66	24	0.8846
$A_4$	187	83	33	0.7232
$A_5$	186	82	29	0.8353
$A_6$	186	84	26	0.8794

From the calculations received, it seemed that the best strikers in the metric category were $A_3$ and $A_6$ players, who were highly rated due to their height and young age. The lowest scores were given to the $A_2$ player, who was the weakest of the analyzed and the $A_4$ player, whose age makes us think that his best years of playing football were just passing by.

3.2. Passing

The second category is passing. These are important for attackers because passing is often better than playing alone or dribbling due to too many opponents [98]. It includes the number of passes a player has made, which shows how often he is looking for other players in better positions. The second criteria in this category is the accuracy of the passes, which allows you to check if a player’s passes are reaching their target. The last one is the key passes, that is passes that are followed by a promiscuous situation at the opponent’s field goal. These criteria are described as follows:

• $C_4$—passes made by player, where $C_4\in [0,70]$;
• $C_5$—the accuracy of the player’s passes (in percentages), where $C_5\in [0,100]$;
• $C_6$—key passes made by player, where $C_6\in [0,5]$.

The average value of each attribute was determined, and the linguistic variables and triangular numbers presented in Figure 5 were created. The number of key passes was a discrete variable because they were integer values from 0 to 5. However, each of these values had three values of membership to the concept of small, medium and large numbers of key passes. This was the fuzzy logic element that helped to identify the decision model. Then 27 characteristic objects were generated, and 351 pairwise comparisons were made. Based on these comparisons, a MEJ matrix was created, taking the following form (A2). The result is a second linguistic model with 27 rules, and the results are presented in Table A2.

Figure 5. The visualization of the linguistic variables and fuzzy triangular numbers for the assessment of passes module ($P_2$).

An illustrative assessment of passes is shown in Table 3. The best strikers in the passes category are $A_4$ and $A_5$ players, as they have made many accurate passes and several key passes. The worst score was given to the $A_3$ player. It is due to the low number of accurate passes and the lack of key passes.

Table 3. Overview of players’ results and assessment of passes $P_2$.

${\mathit{A}}_{\mathit{i}}$	${\mathit{C}}_4$	${\mathit{C}}_5$	${\mathit{C}}_6$	${\mathit{P}}_2$
$A_1$	56	71.4	2	0.7409
$A_2$	39	76.9	1	0.6209
$A_3$	21	61.9	0	0.3550
$A_4$	37	81.1	3	0.7754
$A_5$	43	74.4	3	0.7683
$A_6$	29	72.4	1	0.5372

3.3. Offensive

The third category is offensive. It contained crucial criteria for each attacker [99]. These were the goals scored, the assists for the goals of the other team players, the number of shots per goal, and the number of accurate shots. With these criteria, we can see how the striker was doing in important moments for his position. The accuracy of the shots shows the player’s efficiency, whether the shots he made threatened the goal of the opposing team or were only a loss of the ball. The following criteria were used in the model:

• $C_7$—goals scored by a player, where $C_7\in [0,5]$;
• $C_8$—the assists made by a player, where $C_8\in [0,3]$;
• $C_9$—shots made by a player, where $C_9\in [0,8]$;
• $C_{10}$—shots on target made by a player, where $C_{10}\in [0,8]$.

The linguistic variables were determined and shown in Figure 6.

Figure 6. The visualization of linguistic variables and fuzzy triangular numbers for the offensive assessment model ($P_3$).

Based on triangular fuzzy numbers, characteristic objects were created. A total of 3240 pairwise comparisons were made. Based on these comparisons, the MEJ matrix was created, taking the following form (A4). The calculations to determine the third model are shown in Table A3.

A comparative assessment of the offensive is shown in Table 4. The best score was achieved by a $A_4$ player with four goals and a large number of shots on target. The lowest scores were given to $A_2$ and $A_3$ players, who scored fewer goals, had no assists, and scored an average number of shots on target.

Table 4. Overview of player performance and offensive assessment $P_3$.

${\mathit{A}}_{\mathit{i}}$	${\mathit{C}}_7$	${\mathit{C}}_8$	${\mathit{C}}_9$	${\mathit{C}}_{10}$	${\mathit{P}}_3$
$A_1$	3	1	4	3	0.6576
$A_2$	1	0	8	5	0.5096
$A_3$	2	0	4	3	0.4967
$A_4$	4	1	8	7	0.8621
$A_5$	1	1	4	3	0.5474
$A_6$	1	1	7	3	0.5936

3.4. Technique

The next category is the player’s technique. It contained criteria related to technical training, such as bad ball receiving or loss of the ball caused by the opponent’s attack, which largely determines whether the team is able to attack the rivals’ goal. There were also contacts with the ball which showed if a player had a chance to score a goal and dribbling [100]. The criteria were marked as follows:

• $C_{11}$—bad ball receiving by a player, where $C_{11}\in [0,4]$;
• $C_{12}$—the loss of the ball by a player, where $C_{12}\in [0,5]$;
• $C_{13}$—player’s contact with the ball, where $C_{13}\in [0,90]$;
• $C_{14}$—dribbles made by a player, where $C_{14}\in [0,7]$.

Values specific to each criteria were defined. Next, linguistic variables and triangular fuzzy numbers were created, which are presented in Figure 7.

Figure 7. The visualization of linguistic variables and fuzzy triangular numbers for the technique assessment model ($P_4$).

Afterwards, 81 characteristic objects were created, and 3240 pairwise comparisons were made. Based on these comparisons, the MEJ matrix was created, taking the following form (A4). As a result of the calculations, the SJ vector and the P preference vector were created, presented in Table A4.

An illustrative assessment of the technique is shown in Table 5. Players $A_1$ and $A_2$ achieved the highest scores in the technique category because they had little ball loss and were often with the ball nearby. The worst result was $A_3$ player. He had a lot of ball losses, so his rivals could attack more often.

Table 5. Overview of competitors’ results and technique assessment $P_4$.

${\mathit{A}}_{\mathit{i}}$	${\mathit{C}}_{11}$	${\mathit{C}}_{12}$	${\mathit{C}}_{13}$	${\mathit{C}}_{14}$	${\mathit{P}}_4$
$A_1$	0	1	78	3	0.8298
$A_2$	1	0	69	5	0.8603
$A_3$	1	2	41	1	0.3701
$A_4$	0	2	55	1	0.4754
$A_5$	2	5	74	5	0.5235
$A_6$	2	2	57	2	0.4942

3.5. Offences

The last category is offences. It contained criteria related to the misbehavior of a player on the pitch. These were player fouls, fouls on a player, and offside positions. They determined the penalties received by the players such as yellow or red cards. Through their fouls, players could harm their team if they had to leave the field early or help if they were fouled by an opponent [101]. The criteria were determined as follows:

• $C_{15}$—player fouls, where $C_{15}\in [0,5]$;
• $C_{16}$—fouls on a player, where $C_{16}\in [0,5]$;
• $C_{17}$—offside positions, where $C_{17}\in [0,5]$.

Based on them, triangular fuzzy numbers were presented in Figure 8.

Figure 8. The visualization of linguistic variables and fuzzy triangular numbers for the offense assessment model ($P_5$).

Based on the criteria related to offenses, 27 characteristic objects were generated, and 351 pairwise comparisons were made, the results presented in the form of an MEJ matrix (A5):

The calculations to determine the fifth model are shown in Table A5.

An illustrative assessment of offenses is shown in Table 6. The calculation shows that the best striker in the category of offences was an $A_2$ player. He was often fouled by his opponents, while he did not foul nor was caught in an offside position. The worst score was $A_4$ as he was in the offside position and fouled the opponent, and did not win a foul on himself.

Table 6. Players’ results and offense rating $P_5$.

${\mathit{A}}_{\mathit{i}}$	${\mathit{C}}_{15}$	${\mathit{C}}_{16}$	${\mathit{C}}_{17}$	${\mathit{P}}_5$
$A_1$	1	1	0	0.6818
$A_2$	0	4	0	0.9697
$A_3$	3	3	0	0.6060
$A_4$	1	0	2	0.4318
$A_5$	2	5	0	0.8408
$A_6$	1	1	0	0.6818

3.6. Final Model

After applying a hierarchical structure, the final model with 32 characteristic objects was created. After 496 pairwise comparisons, the MEJ matrix was created and presented as (A6). The P preference vector is shown in Table A6. An illustrative final rating is shown in Table 7.

Table 7. Players’ results and final rating P.

${\mathit{A}}_{\mathit{i}}$	${\mathit{P}}_1$	${\mathit{P}}_2$	${\mathit{P}}_3$	${\mathit{P}}_4$	${\mathit{P}}_5$	P
$A_1$	0.7929	0.7410	0.6577	0.8298	0.6818	0.7757
$A_2$	0.6087	0.6209	0.5096	0.8603	0.9697	0.7127
$A_3$	0.8846	0.3551	0.4968	0.3702	0.6060	0.5268
$A_4$	0.7233	0.7754	0.8622	0.4754	0.4318	0.7283
$A_5$	0.8353	0.7683	0.5474	0.5236	0.8409	0.7003
$A_6$	0.8795	0.5372	0.5936	0.4942	0.6818	0.6451

The highest score was achieved by the $A_1$ alternative. He did not have the most top scores from all categories, but high and average scores from all categories gave him the best position. In particular, he had excellent ratings in the metrics ($P_1$) and technique ($P_4$) categories. The lowest score was achieved by the player $A_3$. Despite the high mark for the metrics ($P_1$), the other weak scores were reflected in the lowest final score.

In order to show the advantage of the COMET method over the methods used, we will analyze also the example consisting of six players using the TOPSIS method with equal weigths [102]. Table 8 presents the results of the TOPSIS calculation using the algorithm presented in TOPSIS. Based on the detailed results, it can be seen that depending on whether the calculation is based on a six-element set or one of the six five-element sets, the results differ. This is due to the fact that in the other methods, the evaluation is created based on tested alternatives (as in the TOPSIS method case). This fact also explains why these methods are susceptible to the rank reversal phenomenon, as the value of each player’s preferences depends on which players are compared. Thus, the result of preferences is different each time (in Table 6, we exclude one alternative in turn). Particularly interesting is the fact of comparing the results for the full set and the five-letter set, where the player $A_3$ is excluded. The ranking reversal phenomenon occurs at the beginning because, in the full set, the $A_2$ player was better than the $A_5$ player. In the set, with the excluded $A_3$ player the relationship is reversed, i.e., $A_2$ player was worse than the $A_5$ player. Besides, it should be noted that the elimination of $A_4$ from the full set of players has made it impossible to normalize the last criterion because all alternatives have the same value. Therefore, it is not possible to judge with all the selected criteria. This explains why the COMET method was used to identify this model.

Table 8. Preference values and rankings obtained using the Technique of Order Preference Similarity (TOPSIS) method and equal weights for a full set of players and six sets consisting of five players.

	Preference for Different Sets							Ranking for Different Sets
${\mathit{A}}_{\mathit{i}}$	Full Set	${\mathit{A}}_{\mathbf{1}}$	${\mathit{A}}_{\mathbf{2}}$	${\mathit{A}}_{\mathbf{3}}$	${\mathit{A}}_{\mathbf{4}}$	${\mathit{A}}_{\mathbf{5}}$	${\mathit{A}}_{\mathbf{6}}$	Full Set	${\mathit{A}}_{\mathbf{1}}$	${\mathit{A}}_{\mathbf{2}}$	${\mathit{A}}_{\mathbf{3}}$	${\mathit{A}}_{\mathbf{4}}$	${\mathit{A}}_{\mathbf{5}}$	${\mathit{A}}_{\mathbf{6}}$
$A_1$	0.5684	—	0.5722	0.5298	—	0.5593	0.5692	2	—	2	2	—	2	2
$A_2$	0.5520	0.5661	—	0.5129	—	0.5585	0.5528	3	2	—	4	—	3	3
$A_3$	0.3784	0.3784	0.3851	—	—	0.3662	0.3784	6	5	5	—	—	5	5
$A_4$	0.5861	0.5972	0.5996	0.5597	—	0.5624	0.5875	1	1	1	1	—	1	1
$A_5$	0.5425	0.5568	0.5472	0.5172	—	—	0.5440	4	3	3	3	—	—	4
$A_6$	0.5069	0.5144	0.5234	0.4629	—	0.4857	—	5	4	4	5	—	4	—

Because the identified model was continuous, we could calculate the derivatives at the point. For each criterion, we calculated the quotient of the differential ratio of the preference increment value to the attribute increment. Detailed results are presented in Table 9. Analyzing column by column, we can see that the relevance of each attribute was different in each of the considered alternative cases. For example, for the $C_1$ criterion, three derivative values were positive, and another three were negative. A large variety of values in the columns shows that it was difficult to find such weights to use them as a universal value in other methods.

Table 9. Value of the point derivative for individual alternatives to individual criteria.

${\mathit{A}}_{\mathit{i}}$	${\mathit{C}}_1$	${\mathit{C}}_2$	${\mathit{C}}_3$	${\mathit{C}}_4$	${\mathit{C}}_5$	${\mathit{C}}_6$	${\mathit{C}}_7$	${\mathit{C}}_8$	${\mathit{C}}_9$	${\mathit{C}}_{10}$	${\mathit{C}}_{11}$	${\mathit{C}}_{12}$	${\mathit{C}}_{13}$	${\mathit{C}}_{14}$	${\mathit{C}}_{15}$	${\mathit{C}}_{16}$	${\mathit{C}}_{17}$
$A_1$	0.46	0.13	−0.41	0.10	0.11	1.27	1.77	0.93	0.69	−3.08	−0.53	−1.10	0.17	0.58	−1.37	1.10	−0.55
$A_2$	0.48	0.15	−0.42	0.12	0.14	1.60	2.82	4.08	0.00	−0.10	−0.65	−1.59	0.13	0.67	−1.05	0.42	−0.63
$A_3$	−0.06	0.16	0.53	0.36	0.04	1.30	2.68	4.01	0.94	−2.75	−0.35	−0.65	0.13	0.50	−1.54	0.82	−0.82
$A_4$	−0.01	−0.35	−0.47	0.08	0.09	1.51	2.46	1.22	0.00	0.47	−0.35	−0.81	0.22	1.28	−1.17	1.39	−1.46
$A_5$	−0.01	−0.31	−0.47	0.08	0.09	1.60	1.90	0.62	0.53	−2.74	−0.96	0.00	0.14	0.45	−1.49	0.00	−0.89
$A_6$	0.02	−0.26	−0.50	0.13	0.14	1.98	2.32	1.38	0.56	−0.37	−0.55	−0.96	0.23	0.90	−1.48	1.19	−0.59

Additionally, the stability of the solution was verified in terms of the obtained ranking. Table 10 gives values of intervals for which the obtained ranking wildidl not change. It also shows which aggregated criteria for which players were more important in terms of changing the final ranking, and which were less important. For better readability, Table 11. also shows the length of the adjacent intervals. The solution obtained was the most stable for a $A_3$ player. This is because he was significantly different from other players. The most sensitive player was $A_2$, where the width of the interval is 0.087. The obtained solution can be considered as stable for one player.

Table 10. Robustness of the obtained results.

${\mathit{A}}_1$	${\mathit{P}}_1$	${\mathit{P}}_2$	${\mathit{P}}_3$	${\mathit{P}}_4$	${\mathit{P}}_5$
$A_1$	[0.53389, 1.00000]	[0.51597, 1.00000]	[0.50968, 1.00000]	[0.62783, 1.00000]	[0.28880, 1.00000]
$A_2$	[0.54271, 0.69071]	[0.56490, 0.69190]	[0.47060, 0.55760]	[0.81232, 0.92032]	[0.87970, 1.00000]
$A_3$	[0.00000, 1.00000]	[0.00000, 0.86006]	[0.00000, 0.80880]	[0.00000, 0.82917]	[0.00000, 1.00000]
$A_4$	[0.64227, 0.96827]	[0.70342, 0.99442]	[0.81918, 0.99218]	[0.40742, 0.68143]	[0.31085, 0.79985]
$A_5$	[0.56436, 0.89636]	[0.52433, 0.82333]	[0.38743, 0.58343]	[0.29757, 0.57457]	[0.41887, 0.93588]
$A_6$	[0.28445, 1.00000]	[0.01522, 0.78122]	[0.26562, 0.74662]	[0.01123, 0.71923]	[0.00000, 1.00000]

Table 11. The range of stability intervals of the obtained solutions.

${\mathit{A}}_{\mathit{i}}$	${\mathit{P}}_1$	${\mathit{P}}_2$	${\mathit{P}}_3$	${\mathit{P}}_4$	${\mathit{P}}_5$
$A_1$	0.466	0.484	0.490	0.372	0.711
$A_2$	0.148	0.127	0.087	0.108	0.120
$A_3$	1.000	0.860	0.808	0.829	1.000
$A_4$	0.326	0.291	0.173	0.274	0.489
$A_5$	0.332	0.299	0.196	0.277	0.517
$A_6$	0.716	0.766	0.481	0.708	1.000

4. Illustrative Examples

We compare the presented model with subjective rankings. The calculation for two different cases is conducted to achieve assessment values for attackers’ performance. The first case contains the general ranking of attackers from different clubs. The comparison includes the assessment rating received from the model and the estimated value of player’s worth on the transfer market. The second case includes the process of assessing the attackers nominated to Golden Ball 2017 plebiscite. Only the players with the highest positions were taken into consideration. Appendix A presents all raw data use in this section.

4.1. Overall Ranking of Attackers

To create an overall ranking of attackers, the ratings for meetings have been calculated for five attackers who, in the period from 10 August 2017 to 31 October 2017 played at least six matches in which they spent at least 75 min on the field. An average score was calculated from the marks received and compared with the estimated value of a player. It makes it possible to check whether the amounts offered by the football clubs for strikers correspond to the skills presented by them. Table 12 shows the rated players, the average rating for the matches played, and the estimated value of the player. The similarity of these rankings is rather small, i.e., $WS=0.63$ and $r_w=0.25$. The valuation of players is based less on the season, but more on the whole career of the player. There is an aspect of psychological evaluation, where behind rising stars or old wolves, the price will always be higher than the current results indicate.

Table 12. Attackers comparison, their average model assessment mark, estimated values.

Player	Average Mark	Value [In Bln Euro]
$S_1$—Lionel Messi	0.6602	180
$S_2$—Leroy Sane	0.6545	75
$S_3$—Mohamed Salah	0.5354	80
$S_4$—Kylian Mbappe	0.5196	120
$S_5$—Antoine Griezmann	0.5094	100

Lionel Messi’s match individual ratings are summarized in Table 13, the individual rating chart in Figure 9 and the final rating chart in Figure 10.

Table 13. Assessment comparison of player Lionel Messi $S_1$.

Match Date	${\mathit{P}}_1$	${\mathit{P}}_2$	${\mathit{P}}_3$	${\mathit{P}}_4$	${\mathit{P}}_5$	P
2017-08-20	0.5866	0.7724	0.1539	0.8786	0.9091	0.6243
2017-08-26	0.5866	0.9001	0.5961	0.7679	0.9091	0.7663
2017-09-09	0.5866	0.9026	0.6763	0.8036	0.9091	0.8017
2017-09-16	0.5866	0.6764	0.2019	0.7188	0.8637	0.5622
2017-09-19	0.5866	0.5594	0.7820	0.5759	0.8182	0.6996
2017-09-23	0.5866	0.7332	0.1570	0.4792	0.6742	0.4626
2017-10-01	0.5866	0.7132	0.7148	0.7893	0.9091	0.7709
2017-10-14	0.5866	0.7948	0.1885	0.7592	0.9394	0.6091
2017-10-21	0.5866	0.8108	0.4616	0.5440	0.8637	0.6360
2017-10-28	0.5866	0.9264	0.4885	0.6112	0.7197	0.6697

Figure 9. The assessment of individual models of player Lionel Messi ($S_1$).

Figure 10. Final rating chart of player Lionel Messi ($S_1$).

The biggest variation of particle results is for the $P3$ submodel. The worst match took place on 23 September and the best on 9 September. The difference in the rating of this player in these two games was over 0.339. Most of the meetings were rated above 0.6, and only the meetings of 16 and 23 September had such low marks.

Leroy Sane’s match individual ratings are summarized in Table 14, the individual rating chart in Figure 11 and the final rating chart in Figure 12. It is evident in this case that once again, the lowest final preference ratings have been recorded for meetings with the worst rating in terms of the attack model (21 August and 30 September). While for $S_1$ the number of matches with a rating below 0.6 was 2 out of 10, for a $S_2$ player this preference was also below 0.6 twice but for six matches. A very high rating is characteristic of the metric, due to the potential of the player.

Table 14. Assessment comparison of player Leroy Sane $S_2$.

Match Date	${\mathit{P}}_1$	${\mathit{P}}_2$	${\mathit{P}}_3$	${\mathit{P}}_4$	${\mathit{P}}_5$	P
2017-08-21	0.9446	0.4374	0.1154	0.6429	0.7727	0.5190
2017-08-23	0.9446	0.9285	0.6442	0.6836	0.6818	0.8065
2017-09-30	0.9446	0.6449	0.1410	0.3965	0.4773	0.4663
2017-10-14	0.9446	0.7318	0.6269	0.4821	0.8333	0.7306
2017-10-21	0.9446	0.7099	0.4776	0.5205	0.8182	0.6818
2017-10-28	0.9446	0.8000	0.5962	0.5355	0.6364	0.7228

Figure 11. The assessment of individual models of player Leroy Sane ($S_2$).

Figure 12. Final rating chart of player Leroy Sane ($S_2$).

Mohamed Salah’s match individual ratings are summarized in Table 15, the individual rating chart in Figure 13 and the final rating chart in Figure 14. In the case of an $S_3$ player, the smallest dispersion of marks is visible, but only two of them exceed 0.6 marks. In the analyzed period, he was indeed a player weaker than the first two. However, his transfer value was slightly higher in this period than that of a $S_2$ player. This shows that the player’s rating is not entirely connected with his game in the short term, but rather with his entire career and possible trend.

Table 15. Assessment comparison of player Mohamed Salah $S_3$.

Match Date	${\mathit{P}}_1$	${\mathit{P}}_2$	${\mathit{P}}_3$	${\mathit{P}}_4$	${\mathit{P}}_5$	P
2017-08-12	0.8154	0.3442	0.4731	0.2619	0.6364	0.4775
2017-08-27	0.8154	0.4878	0.4962	0.3914	0.4318	0.5259
2017-09-16	0.8154	0.7868	0.4385	0.6358	0.4318	0.6363
2017-09-23	0.8154	0.5256	0.4673	0.4464	0.8939	0.5997
2017-10-01	0.8154	0.6180	0.1410	0.4715	0.8637	0.6062
2017-10-14	0.8154	0.5677	0.1923	0.4698	0.4318	0.4519
2017-10-22	0.8154	0.6333	0.4039	0.3896	0.8637	0.5823
2017-10-28	0.8154	0.5686	0.2596	0.3758	0.8182	0.5037

Figure 13. The assessment of individual models of player Mohamed Salah ($S_3$).

Figure 14. Final rating chart of player Mohamed Salah ($S_3$).

Kylian Mbappe’s match individual ratings are summarized in Table 16, the individual rating chart in Figure 15 and the final rating chart in Figure 16. In the case of the $S_4$ player, the metric is rated relatively high. Its weakest point is the attack, as evidenced by the $P_3$ model rating. At seven matches he exceeded the 0.6 marks in only two cases, it was on 8 and 30 September when he played best in the attack. Interestingly, despite such low final results, it is the player occupying the second position in the table of transfer values of the considered players.

Table 16. Assessment comparison of player Kylian Mbappe $S_4$.

Match Date	${\mathit{P}}_1$	${\mathit{P}}_2$	${\mathit{P}}_3$	${\mathit{P}}_4$	${\mathit{P}}_5$	P
2017-09-08	0.8398	0.8886	0.4808	0.5267	0.6818	0.6856
2017-09-17	0.8398	0.6545	0.1474	0.4417	0.9091	0.5202
2017-09-23	0.8398	0.5375	0.1827	0.3757	0.6818	0.4546
2017-09-30	0.8398	0.6447	0.4295	0.4435	0.9091	0.6199
2017-10-14	0.8398	0.3876	0.2776	0.5064	0.8636	0.5157
2017-10-22	0.8398	0.4343	0.1154	0.2664	0.3182	0.3258
2017-10-27	0.8398	0.6458	0.1923	0.4200	0.8182	0.5154

Figure 15. The assessment of individual models of player Kylian Mbappe ($S_4$).

Figure 16. Final rating chart of player Kylian Mbappe ($S_4$).

Antoine Griezmann’s match individual ratings are summarized in Table 17, the individual rating chart in Figure 17 and the final rating chart in Figure 18. During the analyzed period the $S_5$ player had the average of the lowest scores of all players. Only once did he receive a score above 0.6 in the match of 20 September. This is the third player in terms of price in the analyzed set. He got the worst grade on 19 August, when he was also rated the worst for playing in attack.

Table 17. Assessment comparison of player Antoine Griezmann $S_5$.

Match Date	${\mathit{P}}_1$	${\mathit{P}}_2$	${\mathit{P}}_3$	${\mathit{P}}_4$	${\mathit{P}}_5$	P
2017-08-19	0.8324	0.3375	0.1410	0.3714	0.5455	0.3691
2017-09-16	0.8324	0.5377	0.2884	0.3952	0.5341	0.4780
2017-09-20	0.8324	0.8473	0.2327	0.5585	0.7273	0.6041
2017-10-14	0.8324	0.4889	0.2115	0.6286	0.6364	0.5161
2017-10-22	0.8324	0.4792	0.2019	0.4584	0.9091	0.5012
2017-10-28	0.8324	0.8940	0.2693	0.4232	0.7197	0.5885

Figure 17. The assessment of individual models of player Antoine Griezmann ($S_5$).

Figure 18. Final rating chart of player Antoine Griezmann ($S_5$).

The highest score among the players considered is received by the player $S_1\left(0.6602\right)$ with an estimated value of 180 mln euro. He is the most expensive player, so the highest overall score should not come as a surprise. However, player $S_2$ valued at 75 mln euro, scored 0.6545, which is very similar to the $S_1$ striker. It may be due to a significant age difference between the players being compared. It is worth noting the relatively low rating of the $S_4$ player, which is valued at 120 mln euro. Following the approach that a high value of a player means a high final score, there is a contradiction here, because the model assesses the player at an average level. However, he is a player that is so promising that his value is fully justified. It also shows that the value of the transfer is based on the hope that the player will play better. So it is a model that can sometimes differ from the actual game results. The final ratings of the players mentioned above are again shown in Figure 19.

Figure 19. Final assessments of all players.

4.2. The Golden Ball 2017

The Golden Ball is an annual poll in which sports journalists vote for the players, who, in their opinion, presented themselves best individually during the year. We decided to consider only five highest-rated players and compare their assessment marks with the position taken in the poll. Table 18 presents those players, their average score for the whole year, and the position in Golden Ball ranking. The similarity of rankings is again at a low level and is 0.48 and 0.17 for $r_w$ and $WS$ respectively.

Table 18. Ranking of attackers, average scores and positions in Golden Ball 2017. Ranking based on www.whoscored.com.

Player	Average Mark	Position	Position among Attackers
$Z_1$—Cristiano Ronaldo	0.5384	1	1
$Z_2$—Lionel Messi	0.6433	2	2
$Z_3$—Neymar	0.7072	3	3
$Z_4$—Kylian Mbappe	0.5652	7	4
$Z_5$—Robert Lewandowski	0.5114	9	5

Cristiano Ronaldo’s matches statistics for 2017 are presented in Table A12. Ratings from individual matches are shown in Table A7, the graph of individual ratings and the graph of final ratings are shown in Figure 20 and Figure 21 respectively.

Figure 20. Diagram of subsequent models assessment for player Cristiano Ronaldo ($Z_1$).

Figure 21. Diagram of final assessments values for Cristiano Ronaldo ($Z_1$).

Lionel Messi’s matches statistics for 2017 are presented in Table A13. Ratings from individual matches are shown in Table A9, the graph of individual ratings and the graph of final ratings are shown in Figure 22 and Figure 23 respectively.

Figure 22. Diagram of subsequent models assessment for player Lionel Messi ($Z_2$).

Figure 23. Diagram of final assessments values for Lionel Messi ($Z_2$).

Neymar’s matches statistics for 2017 are presented in Table A14. Ratings from individual matches are shown in Table A8. Figure 24. shows the graph of individual ratings and the Figure 25 shows the final assessment.

Figure 24. Diagram of subsequent models assessment for player Neymar ($Z_3$).

Figure 25. Diagram of final assessments values for Neymar ($Z_3$).

Kylian Mbappe’s matches statistics for 2017 are presented in Table A15. Ratings from individual matches are shown in Table A10, the graph of individual ratings and the graph of final ratings are shown in Figure 26 and Figure 27 respectively.

Figure 26. Diagram of subsequent models assessment for player Kylian Mbappe ($Z_4$).

Figure 27. Diagram of final assessments values for Kylian Mbappe ($Z_4$).

Robert Lewandowski’s matches statistics for 2017 are presented in Table A16. Ratings from individual matches are shown in Table A11. The graph of individual ratings is shown in Figure 28 and the graph of final ratings is shown in Figure 29.

Figure 28. Diagram of subsequent models assessment for player Robert Lewandowski ($Z_5$).

Figure 29. Diagram of final assessments values for Robert Lewandowski ($Z_5$).

The obtained ranking show that the best player is $Z_3$, but he only took third place in the poll. The first place was taken by a $Z_1$ player, but his rating indicates that he was not the best player in 2017. Such a high position in the ranking may be due to the victory of his team in Champions League—the most prestigious European cup. For several years now, different opinions of [103,104] about the plebiscite have been heard, among other things, that the victory is determined by the trophies won by the team represented by the player, not by his individual achievements. Another reason for such a high position of a $Z_1$ and $Z_2$ player is his outstanding achievements over the past years. Each of them has already won this trophy five times, but their careers are slowly coming to an end and their skills are no longer as high as a few years ago. Their positions have not been achieved on the basis of their individual achievements but because of many years of playing at the highest level. It is only since the third position in ranking that we can see that the results of the players match the position in the ranking. These are players at different stages of their career. Some more experienced, others less experienced. Some have already scored some trophies, others are just starting to score and it is clear here that none of them are favored because of achievements other than individual skills. This shows that the Golden Ball is very subjective and does not allow for an up-to-date assessment of a player’s skills. The final grades of the players are shown again in Figure 30.

Figure 30. Diagram of final assessment for attackers $Z_1$, $Z_2$, $Z_3$, $Z_4$, $Z_5$.

5. Conclusions and Future Research Directions

The purpose of this research was to create a multi-criteria expert model for evaluating performances in football matches. The main motivation is the lack of objectivised ways of assessing player quality in individual matches. Moreover, such a system could significantly contribute to improving the analysis of players’ performances by the clubs’ coaching staff. Creating a trustworthy model required choosing the appropriate method, defining subproblems and criteria for their assessment, and then calculating the results for players and comparing them. For this purpose, the COMET method was chosen, whose main advantage is that it is completely free of the phenomenon of ranking reversal. Characteristic objects were defined. Based on expert knowledge, a pairwise comparison of characteristic objects was made to obtain a rule base from which the assessment values for alternatives were calculated.

The research was conducted to assess the performance of the players playing as the attackers. The results showed that the model works best when analyzing a long time, such as a calendar year or an entire football season. Such analysis performed in a shorter period may give false results when a player temporarily achieves a better performance, which will overestimate his rating resulting from the model calculation. A more extended period from which the analyzed statistics come from gives a more reliable final result. The model would find its application in situations where football clubs would be interested in increasing its line-up with a well forward-looking striker, or even in the case of selecting a striker from among those in the club, for key meetings of the season.

In the future, the proposed model could be extended to include the possibility of rating players playing in other positions, such as goalkeeper, defender, or midfielder. This functionality would help to create a holistic model for evaluating the team’s performance over a particular time and would allow for the possibility to compare the performance of players on specific positions between teams in a given match. Besides, the model can be also identified by using the COMET method using hesitant or intuitionistic fuzzy set generalization.