Application of Bayesian Approach to Reduce the Uncertainty in Expert Judgments by Using a Posteriori Mean Function

Abstract

Much applied research uses expert judgment as a primary or additional data source, thus the problem solved in this publication is relevant. Despite the expert’s experience and competence, the evaluation is subjective and has uncertainty in it. There are various reasons for this uncertainty, including the expert’s incomplete competence, the expert’s character and personal qualities, the expert’s attachment to the opinion of other experts, and the field of the task to be solved. This paper presents a new way to use the Bayesian method to reduce the uncertainty of an expert judgment by correcting the expert’s evaluation by the a posteriori mean function. The Bayesian method corrects the expert’s evaluation, taking into account the expert’s competence and accumulated long-term experience. Since the paper uses a continuous case of the Bayesian formula, perceived as a continuous approximation of experts’ evaluations, this is not only the novelty of this work, but also a new result in the theory of the Bayesian method and its application. The paper investigates various combinations of the probability density functions of a priori information and expert error. The results are illustrated by the example of the evaluation of distance learning courses.

1. Introduction

In any field of activity, a person faces choices. Different decisions are made in different ways; sometimes people think them over too long, structuring the rationalization process, or, vice versa, they reach conclusions quickly and intuitively, without a clear logic or justification and without wondering about the consequences [1]. According to Hartley and French, a judgment should be a well thought-out opinion that considers the degree of scientific or other knowledge about previous experiences [2]. In decision-making, particular attention is given to the uncertainty of the initial data. The uncertainty of the data can be estimated in different ways, depending on its source [3]. Methods have been developed for this purpose on the basis of specific mathematical theories such as fuzzy set theory and mathematical statistics [4].

In solving decision-making tasks, we meet with problems of incomplete information [5]. In situations in which the object under study cannot be measured with a device and the empirical data are unavailable, incomplete, uninformative, or contradictory [6], the data can be supplemented with an evaluation by a qualified specialist, an expert in the relevant field who has knowledge and skills. An expert evaluation is a moment indicator that can change over time and under the influence of acquired knowledge. Experts are selected based on their professional competence, taking into account characteristics such as work experience, academic degree, experience in the research activity, and ability to solve specific tasks in the relevant field.

Unfortunately, the selection of a competent expert does not solve the problem of uncertain data because of human factors and incomplete knowledge of the true meaning of the object being evaluated. The existence of human errors does not imply or assume that people are incompetent in terms of perception or cognition [7]. An expert’s experience in one particular field does not necessarily mean that the expert is also good at assessing uncertainty in another or a related field [2]. Similarly, attachment to the opinion of other experts, a wish to influence the final result, excessive self-confidence or an unwillingness to agree with the opinion of others, the framing effect, disregard for the a priori probability [8], a lack of concentration, and a lack of interest can lead to an inaccurate evaluation. The data obtained based on expert judgments therefore have a stochastic nature. Changing the composition of the experts, reducing or increasing the number of experts, and repeating the evaluations will lead to ambivalent results [9]. If it is to be studied quantitatively, uncertainty must be represented by a mathematical concept such as probability [10].

The main difference between expert judgment and empirical data is that expert opinion is personal and has a so-called ‘subjective probability’ [2]. There is one opinion that it is impossible to get accurate results on the basis of subjective probability. Brownstein and others refute this view by arguing that science itself has a subjective component, aspects of which can be reported probabilistically and must be interpreted according to the theory of subjective probability [11,12]. Although judgments are subjective, expert evaluation results from a synthesis of relevant prior knowledge and experience based on observable evidence and careful reasoning [11]. In the Bayesian theory, subjective probabilities are based on a well-known accurate and comprehensible system of axioms, and in this way, there is no doubt from a mathematical point of view [13]. Cooke and Goossens believe that, within the subjective interpretation of probability, uncertainty is the degree of a person’s conviction, and can be measured by observing the person’s behaviour during the choice [10].

When considering expert opinions, special attention is paid to the psychology of experts, taking into account their competence to the same extent as the quantitative information, or the evaluation, that they provide [8]. One view is that if observers are allowed to influence each other, the sample size will decrease, and, because of this, the accuracy of the group evaluation will decrease. If useful information is to be obtained from multiple sources, then one should ensure that those sources are independent of each other [14]. However, if we combine all the individual judgments, the result will be unexpectedly good [8].

Experts express their judgments in different ways. In particular, the information owned by the expert is extracted, and a probabilistic representation of this knowledge is constructed [15], using pairwise comparisons, linguistic variables, and fuzzy and natural numbers [3,9].

The trial roulette method is used to elicit the distribution [16], or experts are asked questions about probabilities or quantiles that the analyst converts into parameters of probability distributions [17]. According to Hartley and French, it can be difficult in practice for experts to think in terms of distributions. Therefore, it is often wise to simplify the problem without directly identifying the distribution of the associated parameters [2]. In this paper, experts evaluate quality on the usual 10-point scale. The next section presents a review of the scientific literature on the Bayesian approach and expert judgment in the Web of Science database was performed since this study is carried out within the Bayesian method in decision-making.

2. Literature Review

The Bayesian approach is widely used in the theory and practice of various fields of science. According to Arimone et al., almost all probabilistic approaches are derived from Bayes’ theorem [18]. In Bayesian statistics, the preliminary distribution containing the probable values for each model parameter is updated with data, resulting in an a posteriori distribution. Bayesian methods based on models of the objective statistical function are optimal from the point of view of the average. Regardless of the stochasticity of the data, the average value remains constant [13,19]. The updated result can increase trust in the initial opinions of the experts or adapt these views. As a rule, reliable information is given in a narrow distribution received from an expert [2].

Podofillini et al. employed a Bayesian model to aggregate expert judgments to identify human failures in the field of radiation therapy. A qualitative scale was used for the expert judgments; the judgments were then interpreted as information about the order of magnitude of the error probability, and were aggregated according to the Bayesian scheme [20]. Bigün presented a study on the analysis of the risks of major aviation accidents in Europe using expert evaluation and a Bayesian approach. Models combining expert judgments to predict future risks were investigated [21]. In the work of Leden et al., expert judgments were taken into account using the Bayesian error model in a study of the evaluation of the effect on safety of a new bicycle crossing design [22]. Zavadskas et al. applied a Bayesian approach to modelling the uncertainty associated with the probability of failure, showing that the uncertainty can be reduced when new data are obtained. The method of technological risk management proposed by these authors applies to many industrial and non-industrial facilities that are subject to accidents [23].

A future direction in psychological research is the inclusion of the preliminary knowledge of experts in the statistical analysis [15]. Ramli and colleagues developed a new fuzzy Bayesian Network framework for modelling the psychological reaction and human behaviour in a fire. Experts used fuzzy linguistic terms to determine a priori and conditional probabilities in the Bayesian Network [24]. Zhou et al. offered a method using the Bayesian Network auxiliary model to study the parameters of this network. Together with the data set, expert judgments are used, and result in a more accurate machine learning result with a more minor data set [25]. Sigurdsson et al. presented an overview of Bayesian belief nets for managing expert judgments, and described their use in modelling system reliability [26]. Varis et al. presented a probabilistic Bayesian matrix approach for obtaining expert judgments using belief networks from artificial intelligence. The authors studied the impact of surface water on climate change [27]. Rosqvist also followed the Bayesian approach for the aggregation of expert evaluations and the use of expert judgment for new or regularly modified systems [28].

Wisse et al. used moment methods to combine expert judgments, showing how expert evaluations of moments can be combined in a non-Bayesian way to build an a priori estimate [29]. Smets compared the transferable belief model and the Bayesian approach in her study of expert judgment and the reliability problem. The author claimed that it is impossible to prove which solution is correct since each approach matches different regulatory requirements [30]. Hartley proposed the Bayesian Framework for Structural Expert Judgment (SEJ) alternative to traditional non-Bayesian methods. Posterior distributions are created using the clustering, calibration, and aggregation stages [2].

Mockus argued that, when comparing heuristic algorithms that reflect real-life conditions, the Bayesian approach improves efficiency. Researchers often work hard to determine the best parameters for a proposed heuristic. In Mockus’s work, the Bayesian approach is used to adjust heuristic parameters automatically to find optimal combinations of heuristics [31]. Capa Santos et al. used a Bayesian approach to model operational risk, including macroeconomic effects and expert evaluations [32]. Mazzuchi and van Dorp provided a Bayesian model of expert evaluation to determine lifetime distributions for optimizing maintenance [33]. Jiang et al. proposed a method based on Bayesian theory to identify the relationship between product reliability indicators and quality characteristics in production [34].

Koh et al. applied the Bayesian framework to research into the intensity of exposure to lead in the air in industry. The lead measurement data were logarithmically converted to log-transformed geometric means (LGM) and log-transformed geometric standard deviations (LGSD) values. The experts’ prior distributions were updated with the corresponding LGMS and LGSD values using the Bayesian approach [35]. Åström et al. used the Bayesian approach to include expert judgments in regional evaluations of the sensitivity and specificity of the analysis of the source of drinking water [36] and Parent and Bernier used for studying the risks of extreme hydrological events [37]. Washington and Oh applied Bayes’ theory in a study of the safety of railway crossings in Korea [38], and Ramachandran et al. did the same in the field of occupational health [39]. These authors claimed that expert judgment improved their knowledge about dangers in the workplace, and suggested combining expert assessments with sparse data [39]. Ramachandran et al. also noticed a very high degree of agreement among the experts, although the experts represented different areas of interest. They concluded that there is presumably a vast amount of specialized knowledge that experts use to make similar judgments [39]. Vinogradova et al., to clarify the values of the weights of multi-criteria methods, used the idea of a revaluation of the Bayes hypotheses, combining various evaluations of the weights obtained by various methods into an aggregate, integral judgment [9].

One of the critical points in the decision-making process is the identification of uncertainties, such as the consistency between parameters and the combination of judgments. Werner et al. paid great attention to the dependence between variables [40]. Wilson investigated the dependence of several expert evaluations, both within and between experts [17]. Werner et al. and French et al. discussed mathematical methods for the aggregation of evaluations, such as the pool of opinions, Cooke’s Classical Model, and Bayesian aggregation [6,40].

Considerable attention has been paid to verifying the stability of the methods themselves. Vinogradova has also researched the stability of decision-making methods, including Analytic Hierarchy Process (AHP), Fuzzy Analytic Hierarchy Process (FAHP) [3], Simple Additive Weighting (SAW), Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS), Multi-objective Optimization (MOORA), and Preference ranking organization method for enrichment evaluation (PROMETHEE) [41,42], and the influence of data uncertainty on the final results. Ziemba uses stochastic analysis to study the uncertainty of the parameters of alternatives in the PROSA-C (PROMETHEE for Sustainability Assessment—Criteria) method [43]. Lam et al. proposed an entropy–fuzzy VIKOR (VIseKriterijumska Optimizacija I Kompromisno Resenje) model in which objective weights are used, avoiding subjectivity in determining the weights of criteria [44].

A new concept is used to model uncertainty in decision-making problems—linear Diophantine fuzzy information. Diophantine equations have, as a rule, many solutions, therefore, are called indefinite. The problems of linear Diophantine fuzzy information were dealt with by Riaz et al., proposing new aggregation operators for modeling uncertainty that remove the strict restrictions of existing operators [45]. Particular attention should be paid to the works of Narayanamoorthy et al. [46,47,48,49,50] on decision making, using the methodology of MCDM in hesitant fuzzy. Narayanamoorthy uses Normal Wiggly Hesitant Pythagorean Fuzzy Set with DEMATEL (Decision Making Trial and Evaluation Laboratory Model) and COPRAS methods [46], the scenarios of hesitant fuzzy number with MOORA, TOPSIS, VIKOR [47], and SWARA (Step-wise Weight Assessment Ratio Analysis) [48] methods, proposed normal wiggly hesitant fuzzy set (NWHFS), as an extension of hesitant fuzzy set [49], also the new methodology HF-CRITIC (Hesitant Fuzzy Criteria Importance Through Inter-criteria Correlation) and HF-MAUT (Hesitant Fuzzy Multi Attribute Utility Theory) to get hesitant fuzzy information in order to select the best alternative [50]. Zavadskas et al. presented a new multi-criteria decision-making approach that reduces errors and instabilities caused by novice evaluators using the heuristic evaluation methodology HEBIN (Heuristic Evaluation Based on Interval Numbers) under MULTIMOORA-IVNS (Multi-Objective Optimization by Ratio Analysis under Interval-Valued Neutrosophic Sets [51]. When optimizing under uncertainty, two different approaches are widely used: Min-Max and Bayesian [31].

Literary analysis confirms the relevance of the task solved in this publication. Scientists have proposed many methods, methodologies, and theories working with data, trying to reduce their uncertainties, depending on the different source of their occurrence. There is no single correct solution or best method, which explains the emergence of new solution alternatives. This research proposes a new view of data uncertainty, using the accumulated experience of expert evaluations and the expert’s competence, based on the optimality of the Bayesian method in terms of the mean. The a posteriori mean function corrects the expert’s estimate. A continuous set of possible values of some finite interval of the evaluation scale is used, describing the accumulated experience and competence of the expert by some distributions—such a Bayesian approach has not been proposed in the literature, which gives grounds for conducting this research. This research is possible using mathematical packages that approximate each value of the estimation interval on a straight line. The optimality of the proposed approach is interpreted as the averaged value of the a posteriori function, depending on the a priori information and the expert’s competence.

The publication consists of seven sections. Section 1 is an introduction to the issues of the paper and Section 2 is a literary analysis. Section 3 describes the proposed theory of applying the Bayesian method in expert judgment, while Section 4 is devoted to the study and testing of the proposed theory. In Section 5, cases of using different density functions to describe the accumulated experience of expert estimates and expert error are considered. The cases are tested by changing the parameters used. In Section 5, the problem of the evaluation of the quality of distance courses is solved, in which the proposed theory is applied. In Section 6, the results are discussed, while Section 7 presents the conclusions.

3. Application of the Bayesian Approach

In this paper, the Bayesian method is used to adjust the expert’s evaluation using the value of the a posteriori probability mean function $f_m\left(X\right)$, which depends on the experience of the a priori estimation that has been collected and the qualification of the expert making the decision. The expert evaluates the grade of the quality as $X$. The a posteriori probability mean function (from now on, the mean function) adjusts the expert’s evaluation:

$$f_m\left(X\right)={{\displaystyle \int }}_a^b\theta ·f\left(\theta \vee X\right)d\theta ,$$

(1)

where [a, b] is the interval of expert evaluations. In this research, a = 1, b = 10.

In this case, the function $f_m\left(X\right)$ is the adjusted evaluation of the expert, and represents the best approximation to the expert’s judgment when $X$ is the expert’s grade. The adjustment (correction) of the expert’s evaluation is the difference between the adjusted value and the grade X.

The Bayesian method uses all the accumulated experience (i.e., all the past information about the evaluation and qualifications of a particular expert) to determine the a posteriori probability density function. This publication uses the continuous case of the Bayesian formula, which can be written in the following way [5]:

$$f\left(\theta \vee X\right)=\frac{f\left(X\vee \theta \right)·f\left(\theta \right)}{f\left(X\right)},$$

(2)

where the density function $f\left(\theta \vee X\right)$ is the a posteriori probability distribution of the parameter $\theta$ using empirical information about the random variable $X.$ The parameter $\theta$ is the true quality, otherwise known as the state of nature.

The conditional density function $f\left(X\vee \theta \right)$ in the Bayesian formula is the conditional probability distribution of the new evaluation $X$ when the true state of nature is $\theta$. This function defines the error that is made by the expert when accepting the grade $X$ instead of the true quality $\theta$. The conditional density function of the expert error $f\left(X\vee \theta \right)$ depends on the qualification of the expert.

The a posteriori probability density function $f\left(\theta \vee X\right)$ updates the a priori information about $\theta$ according to the sample data $X$. The a priori information accesses the a posteriori probability density function $f\left(\theta \vee X\right)$ through the a priori probability density function $f\left(\theta \right)$, with all the sample information entered via the function $f\left(X\vee \theta \right)$.

The a priori probability density function $f\left(\theta \right)$ is the accumulated evaluation experience, where $\theta$ is the true quality. The primary quality information is based on a subjective expert judgment or derived from previous observations and evaluations. The accumulation of expert judgment provides valuable knowledge for decision-makers [52]. Washington and Oh thought that, in most cases, the inclusion of a priori information in the evaluation process increases the reliability of the result [38].

The a priori probability distribution is adjusted according to the empirical information; it is not necessary to specify $f\left(\theta \right)$, and sometimes, it is enough to identify the a priori distribution type, which includes $f\left(\theta \right)$. More broadly, a priori information can be information based either on science or on accumulated objective data or self-assessment, research, or another type of subjective information. The use of a priori information depends on the purpose of the analysis. A comparison of a priori and a posteriori information identifies how the sample information changed the initial assumptions.

The function $f\left(X\right)$ is a density function of the judgments $X$ for all possible values of the parameter $\theta$, estimating their subjective probabilities [5]:

$$f\left(X\right)={{\displaystyle \int }}_{-\infty }^{\infty }f\left(X\vee \theta \right)·f\left(\theta \right)d\theta .$$

(3)

For a better understanding, the proposed Bayesian approach is illustrated in Figure 1.

The following section details the prior $f\left(\theta \right)$ and expert error $f\left(X\vee \theta \right)$ density functions used in the further research. The research analyses the possibility of using various combinations of probability density functions to evaluate the quality of distance learning courses.

3.1. A Priori Probability Density Functions

In this research, the a priori information is given by a uniform, triangular or Gaussian probability density function. If experience has not been accumulated or is unknown, then a uniform distribution is used as the a priori probability function. In other cases in this paper, when experience has been collected, the a priori information is set by the triangle or Gaussian distributions that are most suitable for describing expert judgments.

The a priori triangular distribution is used so that, in further research, the results obtained can be approximated and compared with other works in the field of expert judgments that use fuzzy numbers. Triangular fuzzy numbers specify the range of possible judgments, indicating the minimum and maximum values and the most likely evaluation that belongs to the specified interval. One of the most commonly used Gaussian distributions is characterized by random variables that sum up a set of independent factors. The Gaussian distribution is limiting for many other distributions.

3.1.1. The Uniform Distribution for Defining the a Priori Information

When the density of all random variables in the range is constant, it is given by a continuous uniform distribution. The uniform distribution probability density function is used when there is no collected experience. Alternatively, it may be supposed that the scores of the quality of the evaluated objects are uniformly distributed over the whole range; this means that it is assumed that $\theta$ is equally distributed over the possible evaluation scale. The probability density function of a uniform distribution $f\left(\theta \right)$ is determined by the following formula:

$$f\left(\theta \right)=\{\begin{array}{c}\frac{1}{b-a}, \mathrm{when} x ϵ\left[a,b\right] \\ 0, \mathrm{when} x\notin \left[a,b\right]\end{array}$$

(4)

The quality is evaluated on a ten-point scale, i.e., a = 1, b = 10.

3.1.2. The Triangular Distribution for Defining the a Priori Information

A triangular distribution is used when the most probable mean value in a given range is known. The probability density function $f\left(\theta \right)$ plot consists of two segments, one of which increases when $\theta$ changes from the minimum to the mean, and the other of which decreases from the mean to the maximum value $\theta$.

When the collected information is known in the case of expert evaluations, it is not difficult to write down three values: the minimum, the maximum, and the most likely ($\mu$) values. In this paper, we calculated the value of the mean of all the accumulated evaluations, but one can also use the mode.

Since a priori experience may not be enough, the usual interval from the evaluation scale [a, b] is used to determine the minimum and maximum values of the triangle:

$$f\left(\theta \right)=\{\begin{array}{c}\frac{2\left(\theta -a\right)}{\left(\mu -a\right)\left(b-a\right)}, \mathrm{when} a\le \theta \le \mu \\ \frac{2\left(b-\theta \right)}{\left(b-\mu \right)\left(b-a\right)}, \mathrm{when} \mu \le \theta \le b\\ 0, \mathrm{when} \theta \notin \left[a,b\right]\end{array}.$$

(5)

3.1.3. The Gaussian Distribution for Defining the a Priori Information

The probability density function of the normal or the Gaussian distribution is symmetric relative to the mean value, and gives a good characterization of many measurement results; it is therefore one of the most frequently used continuous distributions. The parameters when the a priori probability density function has a Gaussian distribution, i.e., an average value µ and standard deviation σ, are set based on the information collected by the institution for the previous years:

$$f\left(\theta \right)=\frac{1}{\mathsf{\sigma }\sqrt{2\pi }}{e}^{-\frac{{\left(\theta -\mu \right)}^2}{2{\mathsf{\sigma }}^2}}.$$

(6)

The standard deviation describes the change in a set of values about the mean and is calculated using the formula:

$$\sigma =\sqrt{\frac{1}{n-1}{\displaystyle \sum }_{i=1}^n{\left(p_i-\overline{p}\right)}^2}$$

(7)

where $p_i$ is the sample element, $\overline{p}$ the sample mean, and n the sample size.

3.2. The Conditional Density Function

The conditional density function describing the expert’s error depends on the expert’s competence, and shows how much the expert’s evaluation deviates from the true score.

The function $f\left(X\vee \theta \right)$ is defined by the parameters $X \mathrm{and} \theta$, where $X$ is the expert’s evaluation when the true quality is θ. The expert’s error $k$, given by the distribution $f\left(X\vee \theta \right)$, depends on the expert’s competence and is the deviation of the expert’s score of $X$ from the true quality θ. The error $k$ of an experienced expert is set at no more than 1, and $k=1$ is used in this paper. The more competent the expert, the smaller the error will be; the error of the most experienced expert is $k=0.8$. Correspondingly, the error of the least experienced expert is $k=1.2$.

3.2.1. The Triangular Distribution for Defining the Expert’s Error

The density function of the expert error $f\left(X\vee \theta \right)$ is given by a triangular distribution, which is symmetric relative to the state of nature θ. The error $k=\left|X-\theta \right|$ is determined as the deviation of the evaluation from the true goodness $\theta .$

The conditional probability density function given by the triangular distribution describing the expert’s error is written as follows:

$$f\left(X\vee \theta \right)=\{\begin{array}{c}\frac{X-\theta +k}{k^2}, \mathrm{when} \theta -k\le X\le \theta \\ \frac{-X+\theta +k}{k^2}, \mathrm{when} \theta \le X\le \theta +k\\ 0, \mathrm{when} X\notin \left[\theta -k,\theta +k\right]\end{array}.$$

(8)

3.2.2. Gaussian Distribution Used for Defining the Expert’s Error

The conditional probability density function given by the Gaussian distribution for the expert’s error is defined as:

$$f\left(X\vee \theta \right)=\frac{1}{k\sqrt{2\pi }}{e}^{-\frac{{\left(X-\theta \right)}^2}{2k^2}}.$$

(9)

The function is symmetric relative to the state of nature $\theta$.

The probability density functions described above are used in the following sections to analyse the possibility of combining them in the expert judgment. The influence of changes in the parameters of the functions describing the a priori information and the expert error on the clarification of the evaluation $X$, depending on the expert’s competence, is analysed.

The complexity of the problem solved in this article is that $\theta$ and $X$ are considered to be continuous random variables, while experts usually use integers $X$ in the range from 1 to 10 in their evaluations. For a continuous approximation to be acceptable, it is necessary to make a correct selection of the ranges of changes in the $\theta$ and $X$ values.

4. Case Study Using the Bayesian Approach for Expert Judgment

When the Bayesian approach is used in expert judgment, the expert evaluates the quality of the object in the usual way on a 10-point scale, giving a score of $X$. The conditional density function of the expert error, $f\left(X\vee \theta \right),$ depends on the competence of the evaluating expert. The complete evaluation of the estimated object is generalized by the a posteriori mean function, which is calculated using Formula (1).

This paper discusses the use of five different cases of combinations of the a priori and conditional density functions described in Section 2. During the analysis, the parameters describing the a priori information function and the expert error function change, and this tends to change the values of the mean functions. In the first and second cases, combinations of the a priori probability uniform distribution with the conditional triangular and Gaussian distributions are studied. In the third case, a triangular distribution is used for the a priori information and the expert error functions. An a priori probability Gaussian distribution is considered in the fourth and fifth options, with triangular and Gaussian distributions.

4.1. Case 1: Uniform Distribution with a Triangular Distribution

Since the true quality $\theta$ is unknown, $\theta$ may lie at any point in the range [a, b], thus the expert’s error triangle slides across the range of the a priori distribution. One of the separate cases is shown in Figure 2 when $\theta$ is found at Point 7.

Since $f\left(X\right)$ is an integral of the product of two functions, the density of the function $f\left(X\vee \theta \right)=0$ when $X\notin \left[X-k,X+k\right]$, and Formula (3) is rewritten as follows:

$$f\left(X\right)={{\displaystyle \int }}_{X-k}^{X}f\left(X\vee \theta \right)·f\left(\theta \right)d\theta +{{\displaystyle \int }}_X^{X+k}f\left(X\vee \theta \right)·f\left(\theta \right)d\theta .$$

(10)

The function of the estimates $X$ for all possible values of $\theta$ for the a priori uniform and conditional triangle distributions is as follows:

$$f\left(X\right)={{\displaystyle \int }}_{X-k}^X\frac{-X+\theta +k}{k^2}·\frac{1}{\left(b-a\right)}d\theta +{{\displaystyle \int }}_X^{X+k}\frac{X-\theta +k}{k^2}·\frac{1}{\left(b-a\right)}d\theta .$$

(11)

The a posteriori probability function equals:

$$\begin{array}{l}f\left(\theta \vee X\right)=\{\begin{array}{l}\frac{\frac{\left(-X+\theta +k\right)}{k^2}·\frac{1}{\left(b-a\right)}}{{{\displaystyle \int }}_{X-k}^X\frac{-X+\theta +k}{k^2}·\frac{1}{\left(b-a\right)}d\theta +{{\displaystyle \int }}_X^{X+k}\frac{X-\theta +k}{k^2}·\frac{1}{\left(b-a\right)}d\theta },\mathrm{when}X-k\le \theta \le X\\ \frac{\frac{\left(X-\theta +k\right)}{k^2}·\frac{1}{\left(b-a\right)}}{{{\displaystyle \int }}_{X-k}^X\frac{-X+\theta +k}{k^2}·\frac{1}{\left(b-a\right)}d\theta +{{\displaystyle \int }}_X^{X+k}\frac{X-\theta +k}{k^2}·\frac{1}{\left(b-a\right)}d\theta },\mathrm{when}X\le \theta \le X+k\\ 0,\mathrm{when}\theta \notin \left[X-k,X+k\right]\end{array}\\ =\{\begin{array}{l}\frac{\frac{\left(-X+\theta +k\right)}{k^2}·\frac{1}{\left(b-a\right)}}{\frac{1}{\left(b-a\right)}\left(\left(-\frac{\theta ·\left(2·X-\theta -2·k\right)}{2·k^2}\right){|}_{X-k}^X+\left(\frac{\theta ·\left(2·X-\theta -2·k\right)}{2·k^2}\right){|}_X^{X+k}\right)},\mathrm{when}X-k\le \theta \le X\\ \frac{\frac{\left(X-\theta +k\right)}{k^2}·\frac{1}{\left(b-a\right)}}{\frac{1}{\left(b-a\right)}\left(\left(-\frac{\theta ·\left(2·X-\theta -2·k\right)}{2·k^2}\right){|}_{X-k}^X+\left(\frac{\theta ·\left(2·X-\theta -2·k\right)}{2·k^2}\right){|}_X^{X+k}\right)},\mathrm{when}X\le \theta \le X+k\\ 0,\mathrm{when}\theta \notin \left[X-k,X+k\right]\end{array}\\ =\{\begin{array}{l}\frac{\left(-X+\theta +k\right)}{k^2}, \mathrm{when} X-k\le \theta \le X\\ \frac{\left(X-\theta +k\right)}{k^2}, \mathrm{when} X\le \theta \le X+k\\ 0, \mathrm{when} \theta \notin \left[X-k,X+k\right]\end{array}.\end{array}$$

(12)

The mean function of (12) is calculated according to Formula (1):

$$\begin{array}{l}{f}_m\left(X\right)={{\displaystyle \int }}_{X-k}^X\left(\theta ·\frac{\left(-X+\theta +k\right)}{k^2}\right)d\theta +{{\displaystyle \int }}_X^{X+k}\left(\theta ·\frac{\left(X-\theta +k\right)}{k^2}\right)d\theta \\ =\left(\frac{-3X{\theta }^2+2{\theta }^3+3k{\theta }^2}{6·k^2}\right){|}_{X-k}^X+\left(\frac{3X{\theta }^2-2{\theta }^3+3k{\theta }^2}{6·k^2}\right){|}_X^{X+k}\\ =\frac{3X-k}{6}+\frac{3X+k}{6}=X\end{array}$$

(13)

When the a priori probability density function is described by a uniform distribution and $f\left(X\vee \theta \right)$ by a triangular distribution, the mean function $f_m\left(X\right)$ equals $X$, regardless of the expert’s error $k$. In this case, the correction of the expert’s evaluation is equal to 0. Therefore, it is convenient to compare the evaluation results based on the accumulated a priori information with a linear function, which implies the absence of initial information.

4.2. Case 2: Uniform Distribution with a Gaussian Distribution

The conditional density function $f\left(X\vee \theta \right)$ of the Gaussian distribution shifts over the entire a priori uniform distribution interval. The function of the estimate $X$ for all possible values of the parameter $\theta$ is as follows:

$$f\left(X\right)={{\displaystyle \int }}_{X-k}^{X+k}\frac{1}{k\sqrt{2\pi }}{e}^{-\frac{{\left(X-\theta \right)}^2}{2k^2}}·\frac{1}{\left(b-a\right)}d\theta .$$

(14)

In this case, the a posteriori function is:

$$\begin{gathered} f\left(\theta \vee X\right)=\{\begin{array}{c}\frac{\frac{1}{k\sqrt{2\pi }}{e}^{-\frac{{\left(X-\theta \right)}^2}{2k^2}}·\frac{1}{\left(b-a\right)}}{{{\displaystyle \int }}_{X-k}^{X+k}\frac{1}{k\sqrt{2\pi }}{e}^{-\frac{{\left(X-\theta \right)}^2}{2k^2}}·\frac{1}{\left(b-a\right)}d\theta }, \mathrm{when} \theta ϵ\left[a,b\right]\\ 0, \mathrm{when} \theta \notin \left[a,b\right]\end{array} \\ =\{\begin{array}{c}\frac{e^{-\frac{{\left(X-\theta \right)}^2}{2k^2}}}{{{\displaystyle \int }}_{X-k}^{X+k}{e}^{-\frac{{\left(X-\theta \right)}^2}{2k^2}}d\theta }, \mathrm{when} \theta ϵ\left[a,b\right]\\ 0, \mathrm{when} \theta \notin \left[a,b\right]\end{array}. \end{gathered}$$

(15)

According to Formula (1), the mean function of the a posteriori function (15) is calculated as follows:

$$\begin{array}{ll}{f}_m\left(X\right)={{\displaystyle \int }}_{X-k}^{X+k}\theta ·& f\left(\theta \vee X\right)d\theta ={{\displaystyle \int }}_{X-k}^{X+k}\theta ·\frac{e^{-\frac{{\left(X-\theta \right)}^2}{2k^2}}}{{{\displaystyle \int }}_{X-k}^{X+k}{e}^{-\frac{{\left(X-\theta \right)}^2}{2k^2}}d\theta }d\theta \\ & =\frac{1}{{{\displaystyle \int }}_{X-k}^{X+k}{e}^{-\frac{{\left(X-\theta \right)}^2}{2k^2}}d\theta }·{{\displaystyle \int }}_{X-k}^{X+k}\theta ·e^{-\frac{{\left(X-\theta \right)}^2}{2k^2}}d\theta \\ & =\frac{1}{{{\displaystyle \int }}_{X-k}^{X+k}{e}^{-\frac{{\left(X-\theta \right)}^2}{2k^2}}d\theta }·{{\displaystyle \int }}_{X-k}^{X+k}\left(-X+\theta +X\right)·e^{-\frac{{\left(X-\theta \right)}^2}{2k^2}}d\theta \\ & =\frac{-{{\displaystyle \int }}_{X-k}^{X+k}\left(X-\theta \right)·e^{-\frac{{\left(X-\theta \right)}^2}{2k^2}}d\theta }{{{\displaystyle \int }}_{X-k}^{X+k}{e}^{-\frac{{\left(X-\theta \right)}^2}{2k^2}}d\theta }+\frac{X·{{\displaystyle \int }}_{X-k}^{X+k}·e^{-\frac{{\left(X-\theta \right)}^2}{2k^2}}d\theta }{{{\displaystyle \int }}_{X-k}^{X+k}{e}^{-\frac{{\left(X-\theta \right)}^2}{2k^2}}d\theta }\\ & =\frac{k^2{{\displaystyle \int }}_{X-k}^{X+k}{e}^{-\frac{{\left(X-\theta \right)}^2}{2k^2}}d(e^{-\frac{{\left(X-\theta \right)}^2}{2k^2}})}{{{\displaystyle \int }}_{X-k}^{X+k}{e}^{-\frac{{\left(X-\theta \right)}^2}{2k^2}}d\theta }+X=\frac{k^2{e^{-\frac{{\left(X-\theta \right)}^2}{2k^2}}|}_{X-k}^{X+k}}{{{\displaystyle \int }}_{X-k}^{X+k}{e}^{-\frac{{\left(X-\theta \right)}^2}{2k^2}}d\theta }+X=X.\end{array}$$

(16)

Combining a uniform and a Gaussian distribution gives an analogous result $f_m\left(X\right)=X$. It is convenient to compare the corrected values of the expert’s evaluation with the graphic $f_m\left(X\right)=X$ showing the zero correction of the expert’s evaluation.

Thus, for a priori probability having a uniform distribution with conditional density functions of the triangular or Gaussian distributions, the mean functions are $f_m\left(X\right)=X$, regardless of the parameter $k$ of the conditional density function. In the absence of accumulated experience in the evaluation, the expert scores $X$ are not adjusted and remain the same.

4.3. Case 3: Triangle Distribution That Determines a Priori Information and Expert’s Error

The a priori information is described by the triangle distribution given in Formula (5). The expert’s error is described by the conditional density function of the triangle (Formula (8)) with the mean at the point $\theta$. The triangle is moving within the a priori function.

Formula (10) calculates the function of the evaluations $X$ for all possible values of the parameter $\theta :$

$$f\left(X\right)={{\displaystyle \int }}_{X-k}^{X }\frac{\left(-X+\theta +k\right)}{k^2}·\frac{2\left(\theta -a\right)}{\left(\mu -a\right)\left(b-a\right)}d\theta +{{\displaystyle \int }}_X^{X+k}\frac{\left(X-\theta +k\right)}{k^2}·\frac{2\left(b-\theta \right)}{\left(b-\mu \right)\left(b-a\right)}d\theta .$$

(17)

Using the Bayesian Formula (2), the a posteriori function is calculated as follows:

$$f\left(\theta \vee X\right)=\{\begin{array}{l}\frac{\frac{\left(-X+\theta +k\right)}{k^2}·\frac{2\left(\theta -a\right)}{\left(\mu -a\right)\left(b-a\right)}}{{{\displaystyle \int }}_{X-k}^{X }\frac{\left(-X+\theta +k\right)}{k^2}·\frac{2\left(\theta -a\right)}{\left(\mu -a\right)\left(b-a\right)}d\theta +{{\displaystyle \int }}_X^{X+k}\frac{\left(X-\theta +k\right)}{k^2}·\frac{2\left(b-\theta \right)}{\left(b-\mu \right)\left(b-a\right)}d\theta }, \mathrm{when} X-k\le \theta \le X\\ \frac{\frac{\left(X-\theta +k\right)}{k^2}·\frac{2\left(b-\theta \right)}{\left(b-c\right)\left(b-a\right)}}{{{\displaystyle \int }}_{X-k}^{X }\frac{\left(-X+\theta +k\right)}{k^2}·\frac{2\left(\theta -a\right)}{\left(\mu -a\right)\left(b-a\right)}d\theta +{{\displaystyle \int }}_X^{X+k}\frac{\left(X-\theta +k\right)}{k^2}·\frac{2\left(b-\theta \right)}{\left(b-\mu \right)\left(b-a\right)}d\theta }, \mathrm{when} X\le \theta \le X+k\\ 0, \mathrm{when} \theta \notin \left[X-k,X+k\right]\end{array}$$

(18)

Table 1. Dependence of the function $f_m\left(X\right)$ value on the parameter $\mu$ and the grade $X$ in case 3.

${\mathit{f}}_{\mathit{m}}\left(\mathit{X}\right)$	$\mathit{\mu }=5$	$\mathit{\mu }=6.5$	$\mathit{\mu }=7$	$\mathit{\mu }=8.5$	$\mathit{\mu }=9$	$\mathit{\mu }=9.5$
$X=1$	1.3687	1.3477	1.3431	1.3333	1.3309	1.3288
$X=2$	2.2696	2.2959	2.3021	2.3162	2.3199	2.3232
$X=3$	3.1762	3.2392	3.2556	3.2952	3.3061	3.3159
$X=4$	4.0880	4.1770	4.2024	4.2688	4.2882	4.3064
$X=5$	5.0045	5.1082	5.1410	5.2346	5.2642	5.2932
$X=6$	5.9254	6.0320	6.0694	6.1884	6.2304	6.2736
$X=7$	6.8504	6.9469	6.9848	7.1228	7.1790	7.2418
$X=8$	7.7792	7.8513	7.8833	8.0222	8.0917	8.1810
$X=9$	8.7114	8.7432	8.7593	8.8485	8.9103	9.0175
$X=10$	9.6468	9.6199	9.6042	9.4762	9.3056	8.1111

and

Table 2. Dependence of the value of the function $f_m\left(X\right)$ on the expert error $k$ and the grade $X$ in case 3, when $\mu =7$.

k	$\mathit{X}=1$	$\mathit{X}=2$	$\mathit{X}=3$	$\mathit{X}=4$	$\mathit{X}=5$	$\mathit{X}=6$	$\mathit{X}=7$	$\mathit{X}=8$	$\mathit{X}=9$	$\mathit{X}=10$
0.8	1.2729	2.2403	3.2035	4.1615	5.1131	6.0568	6.9905	7.9111	8.8145	9.6943
1	1.3431	2.3021	3.2556	4.2024	5.1410	6.0694	6.9848	7.8833	8.7593	9.6042
1.2	1.4143	2.3646	3.3081	4.2435	5.1688	6.0814	6.9777	7.8531	8.7	9.5077

provide the corrected expert evaluations $f_m\left(X\right)$ for different scores of $X$ by changing the parameter of the mean $\mu$ of the a priori probability triangle function (Table 1) and the expert error variable $k$ (Table 2). When studying the influence of $\mu$ on the function $f_m\left(X\right)$, the expert error parameter $k$ remains unchanged and is equal to $k=1$.

Analysing the results of Table 1, we notice a trend in the way in which the mean value $\mu$ of the a priori probability density function affects the refinement of the evaluation of $X$ by the function $f_m\left(X\right)$. When $X<\mu$, the value of the evaluation of $X$ is adjusted upwards, and, accordingly, when $X>\mu$, the evaluation of $X$ decreases. In the case of $X=\mu$, the change in the value of $X$ is the most insignificant.

The closer the value of $X$ is to the value of $\mu$, the smaller the correction, and vice versa. Note that for $X=1$ and $X=10$, the results of refining the evaluations are inaccurate and illogical. Figure 3 shows the graphic $f_m\left(X\right)$ for $\mu$= 9.5, $k=1$ on the segment $X$ 𝝐 [0;10]. At the end of the interval, the graphic is bent, which can be explained by the use of a triangular distribution for the a priori density function; since $f\left(\theta \right)=0$, when $\theta \notin \left[a,b\right]$, when integrating, the conditional triangular distribution goes beyond the interval by a distance $k$.

This is especially evident when the value of $\mu$ is high. Therefore, for a high value of the mean $\mu$ of the function $f\left(\theta \right),$ this option is not recommended because of inaccuracies at the ends of the function $f_m\left(X\right)$.

When studying the dependence of the parameter $k$ on the function $f_m\left(X\right)$, at $\mu =7$, this pattern is observed (Table 2). The smaller the value of $k$, the more minor the correction of the evaluation of $X$. If the expert making the decision is of high competence, then his judgment is more trustworthy, and the a priori evaluation has less influence on the correction of his score.

Despite the size of the change in the evaluation depending on the value of $k$, the tendency to correct the score remains the same. The evaluation decreases if the mean value of the function $f\left(\theta \right)$ is less than the evaluation $X$ and increases in the opposite case.

4.4. Case 4: Gaussian Distribution with a Triangle Distribution

The a priori information is defined by a Gaussian distribution, and the expert error by a conditional triangle distribution. For this case, the function $f\left(X\right)$ is as follows:

$$f\left(X\right)={{\displaystyle \int }}_{X-k}^{X }\frac{\left(-X+\theta +k\right)}{k^2}·\frac{1}{\mathsf{\sigma }\sqrt{2\pi }}{e}^{-\frac{{\left(\theta -\mu \right)}^2}{2{\mathsf{\sigma }}^2}}d\theta +{{\displaystyle \int }}_X^{X+k}\frac{\left(X-\theta +k\right)}{k^2}·\frac{1}{\mathsf{\sigma }\sqrt{2\pi }}{e}^{-\frac{{\left(\theta -\mu \right)}^2}{2{\mathsf{\sigma }}^2}}d\theta .$$

(19)

Accordingly, the a posteriori function is:

$$f\left(\theta \vee X\right)=\{\begin{array}{l}\frac{\frac{\left(-X+\theta +k\right)}{k^2}·\frac{1}{\sigma \sqrt{2\pi }}{e}^{-\frac{{\left(\theta -\mu \right)}^2}{2{\sigma }^2}}}{{{\displaystyle \int }}_{X-k}^{X }\frac{\left(-X+\theta +k\right)}{k^2 }·\frac{1}{\sigma \sqrt{2\pi }}{e}^{-\frac{{\left(\theta -\mu \right)}^2}{2{\sigma }^2}}d\theta +{{\displaystyle \int }}_X^{X+k}\frac{\left(X-\theta +k\right)}{k^2}·\frac{1}{\sigma \sqrt{2\pi }}{e}^{-\frac{{\left(\theta -\mu \right)}^2}{2{\sigma }^2}}d\theta }, \mathrm{when} X-k\le \theta \le X\\ \frac{\frac{\left(X-\theta +k\right)}{k^2}·\frac{1}{\sigma \sqrt{2\pi }}{e}^{-\frac{{\left(\theta -\mu \right)}^2}{2{\sigma }^2}}}{{{\displaystyle \int }}_{X-k}^{X }\frac{\left(-X+\theta +k\right)}{k^2}·\frac{1}{\sigma \sqrt{2\pi }}{e}^{-\frac{{\left(\theta -\mu \right)}^2}{2{\sigma }^2}}d\theta +{{\displaystyle \int }}_X^{X+k}\frac{\left(X-\theta +k\right)}{k^2}·\frac{1}{\sigma \sqrt{2\pi }}{e}^{-\frac{{\left(\theta -\mu \right)}^2}{2{\sigma }^2}}d\theta }, \mathrm{when} X\le \theta \le X+k\\ 0, \mathrm{when} \theta \notin \left[X-k,X+k\right]\end{array}$$

(20)

We will examine the dependence of the mean function $f_m\left(X\right)$ on the parameters of the mean $\mu$, the standard deviation $\sigma$, and the expert error $k$ of the Gaussian distribution function $f\left(\theta \right)$.

Let us study the case for different $\mu$ when $\sigma =1$ and $k=1$ (

Table 3. Dependence of the function $f_m\left(X\right)$ value on the mean of $f\left(\theta \right)$ and the grade $X$ in case 4.

${\mathit{f}}_{\mathit{m}}\left(\mathit{X}\right)$	$\mathit{\mu }=5$	$\mathit{\mu }=6.5$	$\mathit{\mu }=7$	$\mathit{\mu }=8.5$	$\mathit{\mu }=9$	$\mathit{\mu }=9.5$
$X=1$	1.5024	1.6184	1.6447	1.7132	-	-
$X=2$	2.4033	2.5394	2.5854	2.6695	2.7105	2.7105
$X=3$	3.2837	3.4868	3.5024	3.6146	3.6711	3.3705
$X=4$	4.1469	4.3552	4.4033	4.5519	4.5568	4.6184
$X=5$	5	5.2171	5.2837	5.4554	5.5024	5.5443
$X=6$	5.8531	6.0741	6.1469	6.3459	6.4033	6.4554
$X=7$	6.7163	6.9259	7	7.2171	7.2837	7.3459
$X=8$	7.5967	7.7829	7.8531	8.0741	8.1469	8.2171
$X=9$	8.4976	8.6541	8.7163	8.9259	9	9.0741
$X=10$	9.4145	9.5446	9.5967	9.7829	9.8531	9.9259

). Analysing the results of Table 3, we can notice the same trend as we saw in case 3. When $X<\mu$, the value of the evaluation of $X$ is adjusted up, and, in the same way, when $X>\mu$, the evaluation of $X$ decreases. Unlike the previous case, if $X=\mu ,$ then the values of $X$ remain unchanged and are not adjusted.

The problem of correcting the evaluation appears at $X=1$ when the mean value of the $f\left(\theta \right)$ function is high ($\mu >9$). A single case of the graph of $f_m\left(X\right)$, when $\mu =9$ and the parameters are $\mathsf{\sigma }=1,k=1$, is shown in Figure 4.

The explanation for this is the thin Gaussian tail of the a priori distribution and the conditional triangular distribution that goes beyond the interval by a distance $k$. Despite this problem, this option can be used, since the evaluation of $X=1$ is unlikely when evaluating quality, and it is possible to narrow the evaluation scale to $\left[2; 10\right]$.

When $\mu =7$, this problem does not arise, with the same parameter of the a priori function $f\left(\theta \right)$ of $\mathsf{\sigma }=1$ (

Table 4. Dependence of the value of the function $f_m\left(X\right)$ on the expert error $k$ and the grade $X$ in case 4, when $\mathsf{\sigma }=1,$ $\mu =7$.

k	$\mathit{X}=1$	$\mathit{X}=2$	$\mathit{X}=3$	$\mathit{X}=4$	$\mathit{X}=5$	$\mathit{X}=6$	$\mathit{X}=7$	$\mathit{X}=8$	$\mathit{X}=9$	$\mathit{X}=10$
0.8	1.4632	2.2412	3.3498	4.2761	5.1916	6.6098	7	7.9018	8.8084	9.7239
1	1.6447	2.5854	3.5024	4.4033	5.2837	6.1469	7	7.8531	8.7163	9.5967
1.2	1.8281	2.7573	3.6633	4.5401	5.1842	6.2001	7	7.7992	8.6153	9.4599

). Changing the parameter $k$, which reflects the expert’s competence, the correction of the evaluation increases with the value of $k$. In the case of $\mu =X$, the evaluation correction is zero for any value of $k$.

To study the influence of the $\sigma$ parameter on the function $f_m\left(X\right)$, the remaining parameters $\mu =5$ and $k=1$ are left unchanged (

Table 5. Dependence of the value of the function $f_m\left(X\right)$ on $\mathsf{\sigma },$ the $f\left(\theta \right)$ standard deviation parameter, and the grade $X$ in case 4, when $\mu =5$, $k=1$.

$\mathit{\sigma }$	$\mathit{X}=1$	$\mathit{X}=2$	$\mathit{X}=3$	$\mathit{X}=4$	$\mathit{X}=5$	$\mathit{X}=6$	$\mathit{X}=7$	$\mathit{X}=8$	$\mathit{X}=9$	$\mathit{X}=10$
0.5	1.8451	2.7838	3.6618	4.4050	5	5.5949	6.3382	7.2162	8.1549	9.1197
1	1.5024	2.4033	3.2837	4.1469	5	5.8531	6.7163	7.5967	8.4976	9.4145
1.5	1.2693	2.2058	3.1391	4.0702	5	5.9298	6.8609	7.7942	8.7307	9.6711

). For a higher value of $\mathsf{\sigma }$, the a priori function $f\left(\theta \right)$ has a flatter form, and the correction of the evaluation of $X$ is insignificant. When the parameter $\mathsf{\sigma }$ decreases, the correction of the function $f_m\left(X\right)$ increases. In the case of $\mu =X$, the evaluation correction is zero for any value of the parameter $\mathsf{\sigma }$.

In all the cases of changes to the parameters studied here, the tendency to correct the evaluation remains the same. The evaluation of $X$ is corrected according to the regression principle: it decreases if $\mu <X$ and increases if $\mu >X$. When $\mu =X$, the evaluation of $X$ does not change.

4.5. Case 5: Gaussian Distribution That Determines a Priori Information and Expert’s Error

The a priori information is distributed according to a Gaussian distribution (Formula (6)). The conditional density function of the Gaussian distribution is given in Formula (9). The probability function of the observations $X$ for all possible values of the parameter $\theta$ is as follows:

$$f\left(X\right)={{\displaystyle \int }}_{X-k}^X\frac{1}{k\sqrt{2\pi }}{e}^{-\frac{{\left(X-\theta \right)}^2}{2k^2}}·\frac{1}{\sigma \sqrt{2\pi }}{e}^{-\frac{{\left(\theta -\mu \right)}^2}{2{\sigma }^2}}d\theta +{{\displaystyle \int }}_X^{X+k}\frac{1}{k\sqrt{2\pi }}{e}^{-\frac{{\left(X-\theta \right)}^2}{2k^2}}·\frac{1}{\sigma \sqrt{2\pi }}{e}^{-\frac{{\left(\theta -\mu \right)}^2}{2{\sigma }^2}}d\theta$$

(21)

Accordingly, the a posteriori function is:

$$f\left(\theta \vee X\right)=\{\begin{array}{l}\frac{\frac{1}{k\sqrt{2\pi }}{e}^{-\frac{{\left(X-\theta \right)}^2}{2k^2}}·\frac{1}{\sigma \sqrt{2\pi }}{e}^{-\frac{{\left(\theta -\mu \right)}^2}{2{\sigma }^2}}}{{{\displaystyle \int }}_{X-k}^X\frac{1}{k\sqrt{2\pi }}{e}^{-\frac{{\left(X-\theta \right)}^2}{2k^2}}·\frac{1}{\sigma \sqrt{2\pi }}{e}^{-\frac{{\left(\theta -\mu \right)}^2}{2{\sigma }^2}}d\theta +{{\displaystyle \int }}_X^{X+k}\frac{1}{k\sqrt{2\pi }}{e}^{-\frac{{\left(X-\theta \right)}^2}{2k^2}}·\frac{1}{\sigma \sqrt{2\pi }}{e}^{-\frac{{\left(\theta -\mu \right)}^2}{2{\sigma }^2}}d\theta }, \mathrm{when} X-k\le \theta \le X\\ \frac{\frac{1}{k\sqrt{2\pi }}{e}^{-\frac{{\left(X-\theta \right)}^2}{2k^2}}·\frac{1}{\sigma \sqrt{2\pi }}{e}^{-\frac{{\left(\theta -\mu \right)}^2}{2{\sigma }^2}}}{{{\displaystyle \int }}_{X-k}^X\frac{1}{k\sqrt{2\pi }}{e}^{-\frac{{\left(X-\theta \right)}^2}{2k^2}}·\frac{1}{\sigma \sqrt{2\pi }}{e}^{-\frac{{\left(\theta -\mu \right)}^2}{2{\sigma }^2}}d\theta +{{\displaystyle \int }}_X^{X+k}\frac{1}{k\sqrt{2\pi }}{e}^{-\frac{{\left(X-\theta \right)}^2}{2k^2}}·\frac{1}{\sigma \sqrt{2\pi }}{e}^{-\frac{{\left(\theta -\mu \right)}^2}{2{\sigma }^2}}d\theta },\mathrm{when} X\le \theta \le X+k.\\ 0, \mathrm{when} \theta \notin \left[a,b\right]\end{array}$$

(22)

For the case when the probability density functions $f\left(\theta \right)$ and $f\left(X\vee \theta \right)$ are Gaussian distributions, we analyse the effect of the parameter $\mu$ of the a priori function $f\left(\theta \right)$ on the function $f_m\left(X\right)$ for different values of $X$ (

Table 6. Dependence of the function $f_m\left(X\right)$ value on the mean of $f\left(\theta \right)$ and the grade $X$ in case 5.

${\mathit{f}}_{\mathit{m}}\left(\mathit{X}\right)$	$\mathit{\mu }=5$	$\mathit{\mu }=6.5$	$\mathit{\mu }=7$	$\mathit{\mu }=8.5$	$\mathit{\mu }=9$	$\mathit{\mu }=9.5$
$X=1$	1.6808	1.770	1.7909	1.8367	1.8481	1.8581
$X=2$	2.5851	2.7162	2.7456	2.8085	2.8237	2.8367
$X=3$	3.4435	3.6377	3.6808	3.7702	3.7909	3.8085
$X=4$	4.2444	4.5211	4.5851	4.7162	4.7456	4.7702
$X=5$	5	5.3513	5.4435	5.6377	5.6808	5.7162
$X=6$	5.7555	6.1256	6.2444	6.5211	6.5851	6.6377
$X=7$	6.5565	6.8744	7	7.3513	7.4435	7.5211
$X=8$	7.4149	7.6487	7.7555	8.1256	8.2444	8.3513
$X=9$	8.3192	8.4789	8.5565	8.8744	9	9.1256
$X=10$	9.2544	9.3623	9.4149	9.6487	9.7555	9.8744

). The parameter $\sigma$ of the function $f\left(\theta \right)$ is set to $1$ and the expert error $k=1$. The values of the function $f_m\left(X\right)$ increase as the parameter $\mu$ increases. The function $f_m\left(X\right)$ has no breaks at the beginning or the end, unlike the previous cases 3 and 4.

If the value of the expert’s evaluation is $X<\mu$, then the value of $X$ is corrected upwards; if $X=\mu$, then the value of $f_m\left(X\right)=X$; if $X>\mu$, then the value of $f_m\left(X\right)$ is less than the value of $X$ due to the low mean evaluation of the accumulated experience given by the $a priori$ probability distribution.

A study of the influence of the expert error on the function $f_m\left(X\right)$ ($\mu =8.5$, $\mathsf{\sigma }=1$) shows that with a smaller value of $k$, the value of $f_m\left(X\right)$ is smaller, and increasing the value of $k$ means that the value of $f_m\left(X\right)$ increases (

Table 7. Dependence of the function $f_m\left(X\right)$ value on the expert error $k$ and the grade $X$ in case 5, when $\mu =8.5$, $\mathsf{\sigma }=1$.

k	$\mathit{X}=1$	$\mathit{X}=2$	$\mathit{X}=3$	$\mathit{X}=4$	$\mathit{X}=5$	$\mathit{X}=6$	$\mathit{X}=7$	$\mathit{X}=8$	$\mathit{X}=9$	$\mathit{X}=10$
0.8	1.6395	2.6129	3.5777	4.5293	5.4619	6.3680	7.2414	8.0847	8.9153	9.7586
1	1.8367	2.8085	3.7702	4.7162	5.6377	6.5211	7.3513	8.1256	8.8744	9.6487
1.2	2.0336	3.0038	3.9628	4.9038	5.8154	6.6785	7.4672	8.1699	8.8301	9.5328

).

By changing the parameter $\mathsf{\sigma }$ for the a priori function $f\left(\theta \right)$, for $\mu =8.5$, $k=1$ (

Table 8. Dependence of the value of the function $f_m\left(X\right)$ on the standard deviation parameter $\sigma$ of the function $f\left(\theta \right)$ and the grade $X$ in case 5, when $\mu =8.5$ , $k=1$ .

$\mathit{\sigma }$	$\mathit{X}=1$	$\mathit{X}=2$	$\mathit{X}=3$	$\mathit{X}=4$	$\mathit{X}=5$	$\mathit{X}=6$	$\mathit{X}=7$	$\mathit{X}=8$	$\mathit{X}=9$	$\mathit{X}=10$
0.5	1.9606	2.9534	3.9431	4.9271	5.8996	6.8444	7.7069	8.3217	8.6783	9.2931
1	1.8367	2.8085	3.7702	4.7162	5.6377	6.5211	7.3513	8.1256	8.8744	9.6487
1.5	1.6449	2.5971	3.5391	4.6912	5.3857	6.2886	7.1790	8.0607	8.9393	9.8209

) and $\mu =5$, k = 1 (

Table 9. Dependence of the value of the function $f_m\left(X\right)$ on the standard deviation parameter $\mathsf{\sigma }$ of the function $f\left(\theta \right)$ and the grade X in case 5, when $\mu =5$, $k=1$.

$\mathit{\sigma }$	$\mathit{X}=1$	$\mathit{X}=2$	$\mathit{X}=3$	$\mathit{X}=4$	$\mathit{X}=5$	$\mathit{X}=6$	$\mathit{X}=7$	$\mathit{X}=8$	$\mathit{X}=9$	$\mathit{X}=10$
0.5	1.9154	2.8774	3.7927	4.56	5	5.4399	6.2073	7.1226	8.0846	9.0639
1	1.6808	2.5851	3.4435	4.2444	5	5.7555	6.5565	7.4149	8.3192	9.2544
1.5	1.4292	2.3388	3.2352	4.1206	5	5.8794	6.6748	7.6612	8.5708	9.4943

), the results are comparable to the fourth case, which also uses the a priori Gaussian function (Table 5). For a smaller value of $\mathsf{\sigma }$, the correction of the evaluation of $X$ is more significant; correspondingly, for a larger value of $\mathsf{\sigma }$, the values of $f_m\left(X\right)$ are lower.

The parameters $\mu =8.5$, $\mathsf{\sigma }=$0.5 of the a priori function $f\left(\theta \right)$ show that the scores were high for a long period of the evaluation and differed little from the mean value (Table 8). With a small spread of a priori scores, $\mu$ has a more significant impact on the correction of the $X$ evaluations. If the mean value of the a priori function $f\left(\theta \right)$ is high, then at $\mathsf{\sigma }=$0.5 the correction of the evaluation of $X$ is more significant than at $\mathsf{\sigma }=$1.5.

Comparing the results of Table 8 and Table 9, the values of the function $f_m\left(X\right)$ are higher with a larger value of $\mu$.

When the spread of the accumulated estimates is higher, the influence on the result of the mean value of the function $f\left(\theta \right)$ is less. Regardless of the size of the parameter $\mathsf{\sigma }$, for $\mu =X$, the function $f_m\left(X\right)=X$. In this case, when $\mu =5,$ $f_m\left(X\right)=5$ (Table 9). It can be seen from Figure 5 that the trend of the correction of the score X for different values of $\mathsf{\sigma }$ is the same.

Comparing the different combinations of a priori and conditional functions, case 5 appears to be the best for use in a quality evaluation. In cases 3 and 4, there are inaccuracies at the ends of the function $f_m\left(X\right)$.

5. Experimental Application of the Bayesian Approach for Evaluation of Distance Learning Courses

In this section, the above theory is applied to the evaluation of the quality of distance learning courses in one of the higher educational institutions of Lithuania. Distance learning courses are evaluated at least every semester, i.e., twice a year. For several years, the experience was collected. The courses are evaluated by a group of experts, with each expert making a decision independently, and only then the average mark for each course is calculated.

In this example, a group of six experts with different experiences and skills in distance learning education participate in evaluating the quality of the distance learning courses. The first two experts have been working in distance learning for over 12 years; therefore, for these more experienced specialists, the expert error is k = 0.8. The next three specialists have sufficient experience in teaching or writing curriculums, but they are not specialized in distance learning education, thus the error of these experts is k = 1. Moreover, a beginner specialist, specializing in the video recording of lectures and other videos and the animation design of the course, participates in evaluating the distance learning courses, with k = 1.2.

After determining the competence and the value of the expert error k for each commission member, the parameters of the a priori function $f\left(\theta \right)$ are calculated. In practice, a specific statistical distribution is selected after confirming the statistical hypothesis about the type of distribution over a significantly large number of measurements. In this analytical work, the variants mentioned above of the combinations of distributions are used. For the a priori probability triangular function, the mean value of the evaluations $\mu$ is calculated. For the a priori Gaussian function, two parameters are calculated: the mean value $\mu$ and the standard deviation $\sigma$ of the evaluations.

For an example of the application of the theory, three courses of different quality were selected. The first course, from the field of computer science and applied mathematics was ‘Machine learning and neural networks’, while the others, from the field of construction, were ‘Structural engineering’ and ‘Building management’.

To determine the parameters for the $f\left(\theta \right)$ functions, we use the experience of the previous two years. In the subject area of mathematics and computer science, 39 courses were evaluated in these two years, with a mean value $\mu$ of 8.638 and a standard deviation $\mathsf{\sigma }$ of 1.463. In the construction field, 36 courses were evaluated, with a mean value $\mu$ of 8.864, and a standard deviation $\mathsf{\sigma }$ of 0.995.

Table 10. Machine learning and neural networks’ course evaluations; $\mu =8.638$ , $\mathsf{\sigma }=1.463$.

Expert Error	Course Evaluation X	$\mathbf{Updated}\mathbf{Evaluation}{\mathit{f}}_{\mathit{m}}\left(\mathit{X}\right)$
		Case 3	Case 4	Case 5
k = 0.8	9	8.9047	8.9826	8.9698
k = 0.8	8	8.0385	8.0307	8.0532
k = 1	9	8.8628	8.9733	8.9539
k = 1	7	7.1377	7.1198	7.2036
k = 1	8	8.0398	8.047	8.0811
k = 1.2	10	9.2597	9.8598	9.7626
Mean:	8.5	8.3739	8.5022	8.5040

presents the experts’ evaluations X and their corrected values $f_m\left(X\right)$ according to their competence.

The previously described combinations of distributions show how they can be used in expert evaluation. The corrections in cases 4 and 5 are similar to each other, and both options use the a priori Gaussian function. When using the triangular function $f\left(\theta \right)$, the corrected evaluations are lower than in the other two cases, with a high average value of $\mu =8.638$. In case 3, the value $\mathsf{\sigma }$ is not taken into account. The value $\mathsf{\sigma }=1.463$ indicates a large spread of a priori estimates relative to the mean value.

For subsequent courses, the parameters of the $f\left(\theta \right)$ functions are the same, since both courses are from the field of construction (

Table 11. ‘Structural engineering’ course evaluations; $\mu =8.864$ , $\mathsf{\sigma }=0.995.$

Expert Error	Course Evaluation X	$\mathbf{Updated}\mathbf{Evaluation}{\mathit{f}}_{\mathit{m}}\left(\mathit{X}\right)$
		Case 3	Case 4	Case 5
k = 0.8	5	5.2063	5.3429	5.4916
k = 0.8	4	4.2273	4.4068	4.5509
k = 1	6	6.2189	6.3911	6.5719
k = 1	5	5.2562	5.4933	5.6727
k = 1	9	8.8903	8.9796	8.9652
k = 1.2	9	8.8424	8.9721	8.9529
Mean:	6.3333	6.4402	6.5976	6.7009

and

Table 12. True and upgraded ‘Building management’ course grades; $\mu =8.864$ , $\mathsf{\sigma }=0.995.$

Expert Error	Course Evaluation X	$\mathbf{Updated}\mathbf{Evaluation}{\mathit{f}}_{\mathit{m}}\left(\mathit{X}\right)$
		Case 3	Case 4	Case 5
k = 0.8	10	9.5598	9.8877	9.8111
k = 0.8	9	8.9283	8.9864	8.9766
k = 1	9	8.8903	8.9796	8.9652
k = 1	10	9.3730	9.8323	9.7230
k = 1	8	8.0711	8.1284	8.2149
k = 1.2	10	9.1388	9.7710	9.6292
Mean:	9.3333	8.9936	9.2642	9.22

). The mean value of the function $f\left(\theta \right)$ is high ($\mu =8.864$), and the variation of the evaluations relative to the mean value is not large. Therefore, the influence of accumulated experience on the evaluations of the experts is more significant.

The ‘Structural engineering’ course was evaluated ambiguously by the experts. The most highly qualified specialists gave a low rating to the course (scores of 4 and 5). Since $\mu <X$, these evaluations are adjusted upwards. Since the expert error is not large, $k=0.8$, the evaluation corrections are less significant than for the other expert groups. The least experienced expert, working in the multimedia field, rated the course $X=9$. Despite this expert’s error being $k=1.2$, his evaluation is corrected only slightly because of the high mean value of the function $f\left(\theta \right)$. In cases 4 and 5, the mean of the corrected evaluations is rounded up to a score of 7, but the mean of the X evaluations is rounded up to a score of 6.

By contrast, the ‘Building management’ course was judged highly. Most of the X scores exceeded the mean value $\mu$, thus the evaluations were adjusted downwards. In case 5, the same value of X = 10 was corrected for k = 1.2 to 9.6292, for k = 1 to 9.7230, and for k = 0.8 to 9.8111. Since the mean value of the function $f\left(\theta \right)$ is high, it did not significantly affect the overall result. The mean value of the evaluations X and the corrected values for all the options considered is equal to the score of 9 (Table 12).

The graphs of all the functions $f_m\left(X\right)$ described in this paper, clarifying the evaluations of the courses from the construction field for k = 1, are shown in Figure 6.

It is convenient to start considering the graphs from the bottom, from the line numbered 4 (black). These are cases 1 and 2, where the function $f\left(\theta \right)$ is given by a uniform distribution, which means the absence of accumulated, a priori, information. This function is $f_m\left(X\right)=X$; i.e., the estimate is not adjusted. It is convenient to compare the other cases with the line numbered 4 by fixing the influence of a priori information on changes in the function $f_m\left(X\right)$. Deviations from this line illustrate the influence of the a priori information on decision-making.

The next line of the function, number 3 (red), is for case 3, when the a priori information is described by a triangular function, and a triangle also gives the conditional function. This function $f_m\left(X\right)$ corrects the evaluation of X least of all for $\mu >X$ and most of all for $\mu <X$. Since $\mu$ is big enough, the line for the function bends; this is the weak point of this combination.

The following lines, numbers 2 and 1, differ little from each other. Both cases use the a priori Gaussian function $f\left(\theta \right)$. The line for function number 2 uses a triangular conditional function (case 4), and that numbered 1 uses a Gaussian function (case 5). Nonetheless, the most significant correction of the evaluated X for $\mu >X$ is in case 5 (line number 1).

6. Discussion

The Bayesian approach proposed in the paper has shown its capabilities for evaluating data uncertainty, particularly for distance learning courses. In this paper, the analytical part is also applicable to similar problems of quality evaluation, such as a re-evaluation of an exam score taking into account the student’s work during the entire semester.

This research demonstrates the possibility of using different combinations of distributions (their density functions) in the most suitable way for describing a priori information and expert error. In practice, the a priori probability distribution can be selected after confirming the statistical hypothesis about the type of distribution, according to a significant number of accumulated evaluations. The research presented in this paper is analytical, thus more general variants of distributions, such as Gauss and triangle distributions, are chosen.

It is proved that, using a uniform distribution ($f\left(\theta \right)$ density function) with conditional triangle and Gaussian distributions, the mean function $f_m\left(X\right)=X$. This result is interpreted as a lack of accumulated a priori experience, thus the evaluation of $X$ is not corrected. The function $f_m\left(X\right)=X$ is convenient to use for comparison with other options, to give a better determination of the influence of a priori experience on the final result of correcting the experts’ evaluation.

The research results for case 3 (a priori triangle with a conditional triangle distribution), case 4 (a priori Gaussian with a conditional triangle distribution), and case 5 (a priori Gaussian with a conditional Gaussian distribution) are similar, and show that the function of the averages $f_m\left(X\right)$ logically changes depending on the accumulated a priori experience. When $\mu >X$, the values of the function $f_m\left(X\right)$ increase, and, correspondingly, when $\mu <X$, they decrease.

The opinion of a highly qualified expert (expert error $k=0.5$) corresponds to greater trust, thus the correction of the evaluation is minor. With an increase in the error $k$ (expert error $k=1, k=1.5$), the size of the correction of the evaluation $X$ also increases.

When the a priori information is given by a Gaussian function, the result is also affected by the spread of the evaluations. A small spread of a priori evaluations has a more significant impact on the corrections of the X scores. Thus, if the mean value $\mu$ of the function $f\left(\theta \right)$ is high, then at $\mathsf{\sigma }=$0.5 the correction of the evaluation X is greater than it is at $\mathsf{\sigma }=$ 1.5. If $\mathsf{\sigma }$ is large, then the result of the correction does not differ much from the expert evaluation X itself.

After studying the third case of the combination of the triangular function $f\left(\theta \right)$ with a triangular conditional distribution, it becomes clear that with a high mean value of the function $f\left(\theta \right)$ and the expert error k = 1, the values of $f_m\left(X\right)$ function are not logical at the end of the interval. Inaccuracies at the end of the function appear because the function of the expert error at the ends of the range goes beyond the a priori distribution of the triangle. Therefore, when the mean value $\mu$ of the function $f\left(\theta \right)$ is high or low (close to the ends of the interval), this option is not recommended because of inaccuracies at the ends of the $f_m\left(X\right)$ function. The correction of the X scores, in this case, is the most minor, compared to the other combinations.

Combining the Gaussian function $f\left(\theta \right)$ with a triangular conditional distribution (case 4) in the case of a high a priori mean value $\mu$ is not suitable for correcting low evaluations, since the graph of the function $f_m\left(X\right)$ is not homogeneous at the very beginning of the interval. When the mean value of the a priori Gaussian function is very large (or very small), and the expert error is given by a narrow triangle ($k=1$), illogical results arise at the beginning (respectively, at the end) of the function related to the continuous approximation of random integers.

The fifth case, when both functions are Gaussian functions, appears to be the most stable. Even if the a priori function $f\left(\theta \right)$ mean value $\mu$ is very high (or very low), the function $f_m\left(X\right)$ results are logical in the entire interval. This combination shows the most relevant result of all the studied cases and it is recommended that this is applied in practice.

There is a plan for future work to compare the results of this paper with the application of the theory of fuzzy sets. This is possible and it will be interesting to use other combinations of probability distributions.

Even though the adjustment of the values of the expert evaluations is minor and varies within the range of ±1 point, this score can have a significant impact on decisions. In the work of Washington and Oh, applying a Bayesian approach to improving the safety of railway crossings, the authors concluded that the use of a priori information slightly changed the result, and argued that it is good when the expert’s evaluation and the a priori information do not contrast strongly with each other [38]. In this example of evaluating the quality of distance learning courses, a score of 9–10 increases the teacher’s salary at the above-mentioned higher education institution. Another example, the re-evaluation of an exam score based on a priori experience, can be decisive in obtaining a positive exam mark.

7. Conclusions

To reduce the uncertainty of an expert evaluation, the refinement of the experts’ scores is proposed through the use of an a posteriori mean function, taking into account all the collected a priori information and the experts’ competence. This paper illustrates a new way of using the well-known Bayes formula. The continuous case of the Bayesian formula is perceived as a continuous approximation of expert evaluations. The complexity of the task lies in the fact that the parameters of the a priori and conditional density functions for θ (state of nature, true goodness) and X (expert score) are considered as continuous random variables, but experts use integers X in making a decision. In this paper, a 10-point evaluation system is used for expert judgments. The difficulty in obtaining the results of this research lies in the correct choice of the intervals of change for the parameters θ and X for continuous approximation.

It is shown that, when using an a priori probability density function of a continuous uniform distribution, which means the absence of accumulated information, and conditional density functions of triangular and Gaussian distributions, the experts’ evaluation is not corrected. It can be seen that an a priori probability density function of a triangular distribution with a conditional density function also given by a triangle does not give entirely accurate or logical results for the function at the ends of the argument interval. When using a combination of the a priori density function of a Gaussian distribution with a narrow conditional density function of a triangular distribution, illogical results for the function values arise at the ends of the evaluation interval X when the mean value of the Gaussian distribution is very large or very small. When both functions are set by a Gaussian distribution, the result is the most stable and suitable for use in expert judgments.

The results of the research for different combinations are similar and show the following trend. The a posteriori mean function corrects the expert’s evaluation depending on the mean value of the a priori distribution, reducing the evaluation if the mean value is less than the expert’s evaluation, and otherwise increasing it. With a small spread of a priori evaluations, the mean value of the a priori density function has a weightier impact on the correction of the expert’s scores. Trust in the opinion of a highly qualified expert is significant and, accordingly, the correction of such an expert’s evaluation is small. By increasing the expert’s error parameter k, which means the expert is not sufficiently competent, the correction of the expert’s evaluation also increases.

The proposed method of correcting expert evaluations was used to assess the quality of distance learning courses. Even though the correction of the expert assessments appeared to be minor, this adjustment can have an impact on the final result in making a decision.

The goal of this study is to illustrate the approach of Bayesian approach in experts’ judgments for reducing uncertainty in data. This is the first part of a general study devoted to comparing probability and degrees of truth. There is a plan for future work to compare the results of this paper with the application of the theory of fuzzy sets. Therefore, the paper considers in detail different combinations of distributions and the influence of changes in their parameters on the result.