VASMA Weighting: Survey-Based Criteria Weighting Methodology that Combines ENTROPY and WASPAS-SVNS to Reflect the Psychometric Features of the VAS Scales

Abstract

Data symmetry and asymmetry might cause difficulties in various areas including criteria weighting approaches. Preference elicitation is an integral part of the multicriteria decision-making process. Weighting approaches differ in terms of accuracy, ease of use, complexity, and theoretical foundations. When the opinions of the wider audience are needed, electronic surveys with the matrix questions consisting of the visual analogue scales (VAS) might be employed as the easily understandable data collection tool. The novel criteria weighting technique VASMA weighting (VAS Matrix for the criteria weighting) is presented in this paper. It respects the psychometric features of the VAS scales and analyzes the uncertainties caused by the survey-based preference elicitation. VASMA weighting integrates WASPAS-SVNS for the determination of the subjective weights and Shannon entropy for the calculation of the objective weights. Numerical example analyzing the importance of the criteria that affect parents’ decisions regarding the choice of the kindergarten institution was performed as the practical application. Comparison of the VASMA weighting and the direct rating (DR) methodologies was done. It revealed that VASMA weighting is able to overcome the main disadvantages of the DR technique—the high biases of the collected data and the low variation of the criteria weights.

1. Introduction

Criteria weighting is an integral part of the multicriteria decision making (MCDM) models, that are widely applied in economics [1], service quality [2], talent identification process [3], robotics [4], healthcare [5], social studies [6], and other areas. Differences in the preference elicitations methodologies, transparency of the evaluation process, diversity of the opinions, and the competence of the decision-makers (DM) are the important factors affecting the final values of the criteria weights [7]. People participating in the decision-making processes tend to have not only a different understanding of the problem addressed but also to the factors associated with it. Moreover, the increasing interest in public participation activities enlightened differences between expert evaluations and public opinion [8]. Community involvement in the decision-making processes is particularly important when social, educational, environmental, and economic issues are addressed [9]. Non-symmetry in the reflection of the public opinion might cause not only inaccuracies in the preference elicitation results but also the repulsive reactions to the decisions based on them. In these circumstances, the increased interest in the criteria weighting approaches that respect opinions of the wider audience was recently observed.

1.1. Survey-Based Data Collection

Face-to-face interviews and workshops are the most common practices to collect opinions of the target groups. When it is impossible or too expensive to interview respondents through the direct meeting, online surveys might be especially helpful [10,11]. However, survey-based preference elicitation should be carefully organized, since criteria weighting results might be meaningfully affected by the survey structure, construction of the questions, or even by the visual means of the measurement scales. When contrasting unrelated questions are assessed, different survey items should be presented on separate pages [12,13]. If the respondents are asked to give ratings concerning the different aspects of a single latent variable, all the survey items should be intentionally presented on the same (web) page [14]. This particular way to present multiple, related items is called semantic differentials [15].

Semantic differentials are typically expressed as the matrix questions, where preferences are presented on the matrix side, and the response scale is presented on the top of it. Due to the ability to place multiple estimates on a single page, these structures are commonly used to collect public opinions on the quality, satisfaction, and the importance of the analyzed items [14]. Moreover, since humans are much better at making comparative judgments than the absolute ones [16,17], matrix questions might be valuable to increase the accuracy of the direct weighting techniques.

1.2. Matrix Questions and the Response Scales

Matrix questions are usually constructed when several questions about a similar idea should be assessed using the chosen measurement scale. Likert-type scales are typically expressed as the set of radio buttons, representing five or more discrete categories dedicated to revealing respondents’ current state, feelings, or traits [18]. Since Likert-type scales are easily understandable, they are frequently met in the online surveys. However, the ambiguous number of response categories is the important disadvantage of these scales. Moreover, intervals between values cannot be presumed to be equal, and the biases induced by the ordinal data points might cause adversative effects on the calculations of the statistical measures like mean, covariance, correlations, or the reliability coefficients [19].

Issues inherent from the Likert-type measurement increased scientists’ interests in the alternative scales [17]. Research on the historical origin of the semantic differentials revealed that they were initially made from the continuous scales, also known as the visual analogue scales (VAS). A VAS is typically presented as a horizontal line, anchored with two verbal descriptors at the extremes. A respondent indicates his opinion by placing a marker at the most appropriate point. Since VAS uses a line continuum to measure latent traits and to obtain data measurements, they are able to present weighting results without the constraints raised by the limited number of the response categories [20,21]. Fine-grained responses aid in reducing measurement error for both the value-based and the rank-based valuations. Since VAS scales produce interval-level measurement data, they are also better suited for statistical and mathematical algorithms [22].

VAS matrix is a set of the VAS scales placed in a single question. Since twofold data like the importance value and the ranking information can be gathered from a single VAS matrix, it might be successfully exploited for the preference elicitation tasks [23]. Besides, the high degree of details in the VAS scales is exceptionally beneficial when small differences can be detected between the evaluated subjects [24]. For instance, if 13 criteria ought to be assessed on the 7-point Likert scales, criteria of the different importance might fall into the same category making them indistinguishable from one another (Figure 1).

As can be seen, VAS scales are highly sensitive to the respondent’s opinion. Due to this sensitivity, VAS scales are widely applied in medical studies and other areas where small differences might be significant.

1.3. Uncertainty of the Collected Data

VAS scales are easy to understand, administer, and score when implemented in online surveys [25]. Survey-based weighting processes are typically accompanied by the biases of the evaluators and the uncertainty of the experimental conditions. End-aversion bias and the positive skew are also the companions of the VAS scales [26]. End-aversion bias refers to the respondents’ reluctance to use extreme categories such as “extremely important” or “absolutely unimportant”. It does not affect the mean values of the respondent group, but it reduces the variance of the recorded scores [27]. Positive skew refers to the data distribution situation when the responses are not evenly distributed over the range of the scale but show a positive skew towards the favorable end [28].

Both the end-aversion bias and the positive skew suppose that data points belonging to the different ranges of the VAS scales should be treated unequally. Cautious attitude toward the psychometric features of the response scales and the uncertainty of the collected data is required to ensure the accuracy of the criteria weighting results. A new preference elicitation technique that uses the VAS Matrix for the survey-based data collection and employs the appropriate data processing approach to reduce the uncertainties of the collected data is going to be presented in this paper.

2. Criteria Weighting Approaches

Determination of the criteria weights is an important step of the decision-making processes related to the current state of the economic, social, or environmental aspects [5,29]. Since there is no unique classification of the criteria weighting methods, preference elicitation can be divided into statistical and algebraic, direct and indirect, subjective and objective, compensatory and non-compensatory techniques [30].

2.1. Subjective and Objective Techniques

Subjective, objective, and integrated approaches are widely used for preference elicitation. Subjective weights are determined solely according to the preference of the decision-makers. This type of preference elicitation is mostly based on pairwise comparison methods like AHP (Analytic Hierarchy Process) [31], DEMATEL (Decision-making Trial and Evaluation Laboratory) [32], SWARA (Step-Wise Weight Assessment Ratio Analysis) [33], or PIPRECIA (Pivot Pairwise Relative Criteria Importance Assessment) [34]. Objective weights are typically applied then the influence of the individual decision-makers should be reduced. The most well-known objective weighting approaches are the entropy method [35], CRITIC (Criteria Importance Through Intercriteria Correlation) [36], FANMA methods [37].

Since the subjective judgments are noticeably affected by the knowledge and experience of the decision-makers, most of the time, weights determined by subjective approaches neglect the objective information [38]. The integrated preference elicitation approaches can be used to achieve the more accurate values of the criteria weights [39]. These approaches focus on the principle of integrating the subjective weights based on the expert’s opinion and the information gathered from the criteria data in a mathematical form. For instance, Wang and Lee [40] proposed to integrate objective weights calculated by Shannon’s entropy [35] and the subjective weights determined directly by the decision-makers. Saad et al. [41] proposed to weight the criteria combining the Fuzzy Shannon entropy and the subjective weights calculated as the averages of the direct valuations gathered from three decision-makers. The integrated approach that combines objective and subjective weights calculated from the same survey data will be presented in this paper.

2.2. Direct Weighting Approaches

Most of the currently used subjective approaches are based on the opinions of the specially trained experts [39]. Subjective weights calculation from the survey data is much rarer.

Theoretically, the VAS matrix might be exploited to collect data for the preference elicitation based on the pairwise comparisons. PIPRECIA-E [34] is an example of the pairwise comparison technique that might be used to obtain the attitudes of the respondents that were not specially trained for the criteria weighting. However, it should be mentioned that pairwise comparison is highly sensitive to the data loss caused by the respondent’s unwillingness to assess all the criteria. Since a high level of the missing data is normally generated in the survey-based preference elicitation, application of the pairwise comparisons techniques might be especially challenging. Due to the nature of the pairwise comparison, responses, where at least one criterion is not weighed, should be omitted. Such a data cleaning procedure drastically reduces the number of responses; therefore, it might be an important disadvantage of its exploitation for the survey-based criteria weighting.

Direct weighting techniques are the most commonly used for online preference elicitation. In the direct methods, the decision-maker compares criteria by using a ratio scale, whereas, in indirect methods, criteria weights are calculated based on the preferences of the decision-maker [30]. Direct weighting approaches like the SWING [42], SMARTS [43], SMARTER [43], direct rating [44], and the point allocation [44] were recently used in a survey-based preference elicitation [9,45,46].

SWING method implies the construction of the extreme hypothetical scenarios, where initially a hypothetical worst-case scenario is presented, and then the criterion that might be enhanced to improve the overall situation the most is identified as the most important criterion which gets 100 points. All other criteria are weighted in a similar manner and get the point values less than 100 points.

In SMART (Simple Multi-Attribute Rating Technique) the order of the criteria importance is determined primarily and then, starting from the least important criterion, the relative importance of the criteria is assigned in the ascending order. SMARTS and SMARTER are elaborated versions of SMART [43]. SMARTS imply the procedure for determining criteria weights by comparing criteria with the best and the worst criterion from a defined set of criteria. SMARTER (SMART Exploiting Ranks) uses the centroid method to determine criteria weights [47].

Point allocation (PA) and direct rating (DR) are two relatively simple techniques that have lots in common but produce systematically different weighting results [44]. Decision-makers are asked to allocate 100 points among the analyzed criteria when the PA is applied. In the DR methodology, each object is separately assessed on a scale from 0 to 100. Since DR weights do not add up to 1 (100%), they should be normalized at the final stage of the preference elicitation. Direct rating is highly recommended when the performance evaluation relies on a large number of the criteria and when a respondent does not feel comfortable using complex weighting methods. Moreover, the weights elicited by DR are more reliable than those elicited by PA [44]. However, a little variation of the averaged weights is repeatedly identified as the downside of the straightforward DR technique [48].

2.3. VAS Matrix for the Criteria Weighting

VAS matrix can be used as the data collection tool in the survey-based decision. For instance, VAS scales are implemented in the SEIQoL methodology, which is widely used to nominate, weight, and rate different aspects of life quality [49]. SEIQoL with a direct weighting technique (SEIQoL-DW) is an interview-based tool that involves the interviewer to manage the evaluation process. The respondents are asked to nominate the five most important areas of their life (domains) in these semistructured interviews. For the evaluation of the importance of these domains, point allocation weighting is applied. Vertical VAS matrix with five adjacent VAS scales is used to assess the current functioning in the chosen domains. Finally, five separate indexes are calculated summing up the products of the functioning level and the relative weights.

Such a methodology is widely applied in various studies [50]. However, experiments with the SEIQoL-DW revealed that looking at the VAS matrix respondents comprehends the task as the assessment of the domain importance rather than the scoring of their functioning at the research moment [49].

Burckhardt et al. [46] proposed to employ VAS scales and the direct weighting for both the scoring and the weighting of the chosen domains. He also excluded the interviewer from the experiment and used a self-explanatory paper questionnaire to collect the data. In total, 100 participants were involved in this research. Since the averaged values of the VAS based DR technique showed a tendency toward the low variability of the domain weights, the usefulness of the improved methodology was highly questioned. Nevertheless, it must be noted that neither the subjectivity of the respondents nor the psychometric features of the VAS scales were analyzed in the domain importance assessments. We strongly believe that these aspects should be cautiously analyzed when the survey-based criteria weighting is performed.

3. VASMA Weighting Methodology

VASMA weighting (VAS Matrix for criteria weighting) is an easy to apply survey-based criteria weighting technique. It employs WASPAS-SVNS for the determination of the subjective weights and analyzes information entropy for the determination of the objective weights. VASMA weighting is constructed to decrease the uncertainties noticed in the survey-based criteria evaluation preserving the simplicity of the DR alike data collection. The overall VASMA weighting methodology is presented in Figure 2.

Answers provided by the respondents of the online survey are extracted from the survey database and saved in the data matrix $R$ consisting of the values $r_{nl}$:

$$R=\left[\begin{array}{ccc}{r}_{11}& r_{12}& \begin{array}{cc}\dots & r_{1l}\end{array}\\ r_{21}& r_{22}& \begin{array}{cc}\dots & r_{2l}\end{array}\\ \begin{array}{c}⋮\\ r_{n1}\end{array}& \begin{array}{c}⋮\\ r_{n2}\end{array}& \begin{array}{cc}\begin{array}{c}\ddots \\ \cdots \end{array}& \begin{array}{c}⋮\\ r_{nl}\end{array}\end{array}\end{array}\right],$$

(1)

Here $l=1, 2,\dots L$ denote the number of the criteria and $n=1, 2,\dots N$ denote the number of the respondents.

All the evaluations are automatically transformed from the VAS scales to the integer numbers. The linguistic value at the negative anchor (“Absolutely unimportant”) is determined as 1, and the linguistic value at the positive anchor (“Extremely important”) is determined as 100. Other values are calculated as the distance between these two values. If the respondent $n$ did not move a marker from the default position and left it in the middle of the VAS scales, we assume that he did not express his opinion on the specific criterion $l$, therefore the $r_{nl}=0$. Finally, the simple data cleaning procedure must be done deleting the entries where the respondent $n$ did not evaluate either of the criteria $l$.

Data saved in the matrix $R$ is later exploited to construct two different matrixes $P$ and $X$. Decision matrix $P$ is used to calculate the entropy weights; the decision matrix $X$ is constructed to calculate subjective weights via the WASPAS-SVNS approach. The matrixes $R$ and $X$ and their usage for the VASMA weighting will be explicitly described in the following subsections.

3.1. Entropy Weights Calculation

In most of the survey-based research, respondent characteristics are assumed to be constant across respondents. This assumption should be critically accepted since it is just hypothetically possible that all the respondents would be able to read and interpret survey items unanimously. Since such uncertainty might meaningfully affect the quantity of information in the responses, VASMA weighting is constructed in a specific way, ensuring that valuations providing different amounts of information would be treated differently [51]. In 2016, Friesner et al. [52] made an extensive analysis of how entropy-based information theory might be applied to evaluate survey items with multiple-choice responses. A similar methodology is going to be applied to calculate entropy weights and to deal with the ambiguity among respondents.

3.1.1. Construction of the Decision Matrix

Data from the data matrix $R$ should be transformed into the decision matrix $P$, where $p_{kl}$ is the proportion of response $k$ for the criteria $l$:

$$P=\left[\begin{array}{ccc}{p}_{11}& \cdots & p_{1l}\\ ⋮& \ddots & ⋮\\ p_{k1}& \cdots & p_{kl}\end{array}\right].$$

(2)

For each of the possible responses $k$, $p_{kl}$ is calculated by Equation (3):

$$p_{kl}=\frac{{\sum }_{i=1}^N{D}_{kli}}{N},\mathrm{for}\mathrm{each}k=1,2,\dots 100.$$

(3)

Here $N$ is the number of the non-zero assessments for the criterion $l$. $D_{kl}$ is a binary indicator that gives a value of 1 if the respondent $n$ gave the response $k$ for the criteria $l$, otherwise $D_{kl}=0$. Consistent with most statistical principles, the proportions of the responses should follow three rules: $0\le p_{kl}\le 1$ and ${\sum }_{k=1}^K{p}_{kl}=1$ and $p_{kl}lo{g}_2\left(p_{kl}\right)=0when{p}_{kl}=0$.

3.1.2. The Degree of the Information Entropy

Information entropy $E_l\left(p\right)$ is calculated for each of the $p_{kl}$ elements and aggregated through the set of possible responses:

$$E_l\left(p\right)=-{\sum }_{k=1}^K{p}_{kl}lo{g}_2\left(p_{kl}\right).$$

(4)

Normalization of the $E_l\left(p\right)$ is performed dividing the $E_l\left(p\right)$ by the maximum entropy attainable over the $L$ possible survey items. For every value $k$ entropy is maximized when $p_k=\frac{1}{K}$. Therefore, the normalized entropy is calculated by Equation (5):

$$\widetilde{E_l}\left(p\right)=-\frac{E_l\left(p\right)}{lo{g}_2\left(\frac{1}{K}\right)};l=1,2,\dots L,0\le \widetilde{E_l}\left(p\right)\le 1.$$

(5)

3.1.3. The Entropy Weights

Finally, the entropy weights $W_l$ are calculated as the level of change in each criterion l:

$$W_l=1-\widetilde{E_l}\left(p\right);l=1,2,\dots L,0\le W_l\le 1.$$

(6)

By focusing on the distribution of responses, the entropy measure simultaneously encompasses measures of central tendency and the data variability.

3.2. WASPAS-SVNS for the Calculation of Subjective Weights

The uncertainty caused by the psychometric features of the VAS scales is going to be reduced, employing the Weighted Aggregated Sum Product Assessment extended by single-valued neutrosophic sets (WASPAS-SVNS). WASPAS was initially presented by Zavadskas et al. [53] and later extended by single-valued neutrosophic sets (WASPAS-SVNS) that are the extension of the intuitionistic fuzzy sets. WASPAS and its modifications are widely used for various multicriteria decision-making tasks [29,54,55,56]. We believe that WASPAS-SVNS also might be valuable to deal with the uncertainty caused by the end aversion and the positive skew of the VAS based preference elicitation. To the best of our knowledge, there is not any research where WASPAS would be applied in the criteria weighting process.

3.2.1. Construction of the Decision Matrix

Decision matrix $X$, where $x_{ij}$ is the number of $m^{th}$ variable and $l$ is the number of the criteria $\left(i=1,2,\dots m;j=1,2,\dots l\right)$ has to be constructed prior to the other steps of the WASPAS-SVNS approach:

$$X=\left[\begin{array}{ccc}{x}_{11}& x_{12}& \begin{array}{cc}\dots & x_{1l}\end{array}\\ x_{21}& x_{22}& \begin{array}{cc}\dots & x_{2l}\end{array}\\ \begin{array}{c}⋮\\ x_{m1}\end{array}& \begin{array}{c}⋮\\ x_{m2}\end{array}& \begin{array}{cc}\begin{array}{c}\ddots \\ \cdots \end{array}& \begin{array}{c}⋮\\ x_{ml}\end{array}\end{array}\end{array}\right].$$

(7)

Six variables $m$ are determined to assess each of the preferences $l$. Five variables analyze the nominal aspects of the collected data, and the sixth of them examines the ordinal information extracted from the matrix $R$.

Nominal variables. Nominal variable for the criterion l is expressed as the frequency of the values $r_{nl}$ belonging to the predefined interval [a,b]. $D_{nl}$ is a binary indicator that gives a value of 1 if $r_{nl}ϵ\left[a,b\right]$. Otherwise, $D_{nl}=0.$ Nominal variables V1–V5 for each of the criterions l are determined as the matrix X elements $x_{ml}$ via the Equation (8).

$$x_{ml}=\frac{{\sum }_{n=1}^N{D}_{nl}}{N_l},\mathrm{for}\mathrm{each}\mathrm{m}=1,2,\dots ,5;$$

(8)

here $N$ is the total number of the respondents participated in the survey, $N_l$ is the amount of the non-zero assessments $r_{nl}$ for the criterion $l$.

Ranges [a,b] for the nominal variables V1–V5 were determined based on the medical research where VAS scales are widely used in pain studies. The physical manifestation of the pain is measured as the linear distance in the VAS scales of 100 mm length. It was revealed that VAS ratings of 0–4 mm might be considered as no pain; 5–44 mm—mild pain; 45–74 mm—moderate pain; 75–100 mm—severe pain, and 100 mm means the worst imaginable pain [57]. Similar intervals were determined as the five importance groups of the VAS scales (

Table 1. Variables and their weights determined for the WASPAS-SVNS criteria weighting.

Data Type	Variable Name	Variable Description	SMART Weight	Normalized Weight	Optimum
Nominal values	V1	Frequency of the $r_{nl}$ values ⸦ [1–10]	90	0.161	Min
	V2	Frequency of the $r_{nl}$ values ⸦ [11–49]	50	0.089	Min
	V3	Frequency of the $r_{nl}$ values ⸦ [50–74]	10	0.018	Max
	V4	Frequency of the $r_{nl}$ values ⸦ [75–94]	30	0.054	Max
	V5	Frequency of the $r_{nl}$ values ⸦ [95–100]	100	0.179	Max
Ordinal values	V6	Overanking level	280	0.500	Max

).

Ordinal variable. VAS matrix provides a possibility to rank the several latent criteria visually. Scientific research proved that respondents actively use this feature and increase the precision of their answers. For instance, if the pointer of the VAS scales presenting the criterion $l$ is moved to the right side more comparing with the others (Figure 1), it can be understood as criterion $l$ is the most important for the respondent $n$. This concept can be used to determine the new variable called Overanking level (OVL). The OVL level for the criterion $l$ is calculated individually for all the respondents $n$ by the following algorithm:

$$\begin{array}{l}\mathrm{Let}OV{L}_{nl}=0;j=1\mathrm{and}{C}_{nl}=r_{nl}.\end{array}$$

(9)

$$\begin{array}{l}\mathrm{While}j\le l:\end{array}$$

$$\mathrm{if}\left(C_{nl}>r_{nj}\right)and\left(r_{nj}\ne 0\right),OV{L}_{nl}=OV{L}_{nl}+1,$$

(10)

$$j=j+1.$$

(11)

$$\mathrm{Return}OV{L}_{nl}$$

The ordinal variable V6 of the criterion l (Table 1) is calculated as the average of the $OV{L}_{nl}$ aggregated through the total amount of respondents:

$$x_{6l}=\frac{{\sum }_{n=1}^{N}OV{L}_{nl}}{N_l},$$

(12)

here $N_l$ is the amount of the non-zero values $r_{nl}$ for the criterion $l$.

The final set of the predefined variables and their optimums for the MCDM process is presented in Table 1.

3.2.2. The Weighting of the Predefined Variables

Three experts working as the data analysts were introduced with the different aspects of the VAS matrix. Then they were asked to weight all the variables according to the SMARTS methodology. At first, all the experts found a consensus that both the cardinal and ordinal information is equally important for the final decision, therefore the sum of the weights for the variables V1–V5 should be equal to the variable V6.

At the next step, experts ranked all the cardinal variables according to their importance for the criteria weighting and the psychometric features of the VAS scales. Due to the positive skew that can be typically observed in the VAS based valuations, the lowest importance was set to the preference valuations where $r_{nl}\in \left[50-74\right]$. The highest importance was determined for the VAS values when $r_{nl}\in \left[95-100\right]$. Due to the tendency towards the positive assessment, critical opinions encountered in the variables V1 and V2 were considered more important than the positive ones (V3, V4). The final ranking order of the nominal variables was determined as V3 < V4 < V2 < V1 < V5. The relative scores were assigned to V4, V2, and V1 considering their trade-off to the variables V3 and V5.

3.2.3. Preference elicitation by the WASPAS-SVNS Approach

The WASPAS-SVNS approach can be deconstructed into several steps [54]:

1.

Construction of the decision matrix $X$ where $x_{ij} \left(i=1, 2,\dots m; j=1, 2,\dots , n\right)$ is the value of the of $j^{th}$ variable for the $i^{th}$ ithalternative (criteria).

2.

Vector normalization of the element ${\tilde{x}}_{ij}$:

$${\tilde{x}}_{ij}=\frac{x_{ij}}{\sqrt{{\sum }_{i=1}^m{(x_{ij})}^2}},$$

(13)

3.

The neutrosophication and calculation of the neutrosophic decision matrix ${\tilde{X}}^n$. Matrix ${\tilde{X}}^n$ is composed of the single-valued neutrosophic numbers ${\tilde{x}}_{ij}^n=\left(t_{ij},i_{ij},f_{ij}\right)$, where $t$ means the membership degree, $I$ is indeterminacy degree, and $f$ is a non-membership degree. Standard conversion between crisp normalized values ${\tilde{x}}_{ij}$ and neutrosophic numbers ${\tilde{x}}_{ij}^n$ was applied [29].

4.

Calculation of the first decision component ${\tilde{Q}}_i^{\left(1\right)}$ is done by formula:

$${\tilde{Q}}_i^{\left(1\right)}={{\displaystyle \sum }}_{j=1}^{L_{max}}{\tilde{x}}_{+ij}^n·w_{+j}+{\left({{\displaystyle \sum }}_{j=1}^{L_{min}}{\tilde{x}}_{-ij}^n·w_{-j}\right)}^c.$$

(14)

The sum of the total relative importance of the $i^{th}$ alternative is used to calculate ${\tilde{Q}}_i^{\left(1\right)}$. The ${\tilde{x}}_{+ij}^n$ and $w_{+j}$ are the values related with the criteria that should be maximized; ${\tilde{x}}_{-ij}^n$ and $w_{-j}$ are associated to the criteria that should be minimized. Criteria weights $w_{+j}$ and $w_{-j}$ are the arbitrary positive real numbers, $L_{max}$ and $L_{min}$ are the amount of the maximized and minimized criteria. The following algebra operations should be applied for the single-valued neutrosophic numbers:

$${\tilde{x}}_1^n\oplus {\tilde{x}}_2^n=\left(t_1+t_2-t_1{t}_2,i_1{i}_2,f_1{f}_2\right),$$

(15)

$${\tilde{x}}_1^n\otimes {\tilde{x}}_2^n=\left(t_1{t}_2,i_1+i_2-i_1{i}_2,f_1+f_2-f_1{f}_2\right),$$

(16)

$$w{\tilde{x}}_1^n=\left(1-{\left(1-t_1\right)}^w,i_1^w,f_1^w\right),w>0,$$

(17)

$${\tilde{x}}_1^n{}^w=\left(t_1^w,1-{\left(1-i_1\right)}^w,1-{\left(1-f_1\right)}^w\right),w>0,$$

(18)

$${\tilde{x}}_1^n{}^c=\left(f_1,1-i_1,t_1\right),$$

(19)

here ${\tilde{x}}_1^n=\left(t_1,i_1,f_1\right)$ and ${\tilde{x}}_2^n=\left(t_2,i_2,f_2\right)$.

5.

Calculation of the second decision component ${\tilde{Q}}_i^{\left(2\right)}$ is done by the formula:

$${\tilde{Q}}_i^{\left(2\right)}={{\displaystyle \prod }}_{j=1}^{L_{max}}{\left({\tilde{x}}_{+ij}^n\right)}^{w_{+j}}·{\left({{\displaystyle \prod }}_{j=1}^{L_{min}}{\left({\tilde{x}}_{-ij}^n\right)}^{w_{-j}}\right)}^c.$$

(20)

${\tilde{Q}}_i^{\left(2\right)}$ value is based on the product of total relative importance in the ith alternative

6.

Joint generalized criteria is computed by:

$${\tilde{Q}}_i=0.5{\tilde{Q}}_i^{\left(1\right)}+0.5{\tilde{Q}}_i^{\left(2\right)}.$$

(21)

7.

The final weights of the criteria importance are determined considering the descending order of the score function $S\left({\tilde{Q}}_i\right),$ which is used for the deneutrosophication of the joint generalized criteria:

$$S\left({\tilde{Q}}_i\right)=\frac{3+t_i-2i_i-f_i}{4}.$$

(22)

3.3. VASMA Weights

VASMA weights $w_j$ are calculated as the combination of the entropy weights $W_j$ and the WASPAS-SVNS weights $S_j$:

$$w_j=\frac{S_j{W}_j}{{\sum }_{j=1}^l{S}_j{W}_j}.$$

(23)

here $j=$1, 2 …, l is the index of the analyzed criterion.

4. Numeric Example

Children’s care always has been the focus of governmental institutions since early childhood education is recognized as the basis for lifelong learning and development. Nowadays, kindergartens not only provide childcare but also perform protective, emotional, socializing, and educational functions [58]. Since parent’s contribution to their child’s education is increasingly growing, parental opinion and understanding of the kindergarten quality is becoming an important topic. Besides, a clear understanding of parents’ opinions might help public authorities not only to improve the provision of services but also to ensure proper distribution of the public investments. Since organizing of the parental meetings is a time-consuming and human-intensive process, online surveys are the easiest way to find out what parents think.

4.1. Survey Construction and Distribution

The online survey consisting of 15 separate questions was prepared to find out parents’ opinions on the quality of state kindergartens operating in Vilnius (capital of Lithuania). VAS Matrix was placed as the sixth question, where respondents were asked to indicate how important the analyzed criteria are for the search of the most suitable kindergarten for their children. Thirteen criteria adapted from the research of Malović [58] were presented in the VAS matrix. Continuous bipolar VAS scales with the single tick mark at the center of the scales were used. The anchors were named as “Extremely important” and “Absolutely unimportant” (Figure 1). All the texts were provided in Lithuanian, which is the national language of most of the respondents. The target audience was reached through the parents’ groups already existing in the social networks. The survey took place just before the start of the new school year (at the end of August 2019). Since the survey had to be completed online, respondents were free to choose at what time of day to conduct the survey.

A total of 133 individuals completed the online survey. The results of three respondents were excluded from further study because they did not move any of the sliders in the VAS matrix. The demographic profile of all the rest of the respondents is presented in

Table 2. Demographic profile of the respondents.

Variable	Category	(%)
Gender	Female	97.69
	Male	2.31
Age	24–28	16.5
	29–34	49.4
	35–40	29.8
	41–older	4.3
Degree of study	Secondary	4.3
	Professional	6.1
	Bachelor	43.3
	Masters	43.9
	Doctor	1.2
	Another option	1.2
Language spoken at home	Lithuanian	82.68
	Polish	3.94
	Russian	11.81
	English	1.57

.

4.2. Data Extraction from the Survey Database

Data collected with the VAS matrix was automatically converted to the data matrix $R$, where columns denote the set of criteria, and rows denote the ID of the respondent (

Table 3. Criteria assessments converted from the VAS matrix to the data matrix $R$.

ID	C1	C2	C3	C4	C5	C6	C7	C8	C9	C10	C11	C12	C13
1	100	98	99	100	100	58	82	81	93	100	7	19	0
2	91	95	71	0	95	0	97	98	37	97	13	21	10
3	30	69	64	65	0	68	65	68	69	90	13	83	78
4	73	97	0	0	71	93	86	90	60	80	5	84	0
5	0	0	0	11	0	21	0	0	0	0	0	0	92
⋯	⋯	⋯	⋯	⋯	⋯	⋯	⋯	⋯	⋯	⋯	⋯	⋯	⋯
126	33	95	96	88	93	74	82	94	7	98	51	0	8
127	71	0	13	20	100	0	0	0	4	98	5	3	49
128	97	100	100	100	0	64	77	0	84	83	1	0	0
129	76	94	34	97	0	0	0	76	80	95	5	30	96
130	99	97	95	97	97	3	4	4	10	5	6	7	6

). Records where $r_{nl}=0$ depict situations when neither of the VAS markers were moved from their default position. We assumed these cases as non-response values.

Descriptive statistics of the data collected via the preference elicitation process performed by the VAS Matrix are presented in

Table 4. Descriptive statistics of the criteria weighting performed by the VAS Matrix.

No.	Criteria Descriptions	Count of Responses	Mean	SD	Median	Cronbach’s Alfa
C1	Reputation among parents	117	78.26	24.94	88	0.9381
C2	Skills of the kindergarten teachers	112	82.16	21.44	90	0.9199
C3	Modernity of the teaching methods	104	76.70	26.64	86.5	0.9256
C4	Cooperation with parents	109	77.08	26.41	88	0.9266
C5	Free spaces in the proper age groups	102	74.25	28.50	85	0.9486
C6	Toys and equipment	100	71.19	27.00	79	0.9217
C7	Indoor safety and hygiene	106	78.30	24.01	85	0.9167
C8	Outdoor safety and hygiene	108	79.04	23.11	85	0.9189
C9	Opening hours	113	76.19	25.71	82	0.9311
C10	Distance from home	119	83.83	22.72	93	0.9435
C11	Distance from the bus stop	103	36.73	36.59	18	0.9493
C12	Tolerance for different cultures	81	49.89	38.88	52	0.9491
C13	Price	83	61.17	36.01	78	0.9494

. None of the criteria were assessed by all of the 130 respondents analyzed in the study.

4.3. Reliability of the Collected Data

Analysis of the collected data also revealed that only a quarter (27.16%) of the respondents moved all 13 sliders provided in the VAS matrix. It means that three-quarters of the respondents evaluated less than 13 criteria during the experiment. Since all the responses where single and more criteria are assessed contribute in the construction of the data matrix $R$ (Equation (1)), it is necessary to make sure that the data collected during the survey can be trusted.

Both the Cronbach’s Alpha and the Split-Half techniques (with Spearman and Brown correction) were employed to determine the internal reliability of the collected data. The calculated value for the Split-Half technique was 0.9772, and the total Cronbach’s Alpha reliability coefficient was 0.9861 (Table 4). It means that the overall internal reliability of the collected data is very high. Cronbach’s Alpha for all the 13 criteria also showed very high reliability (0.9189 to 0.9494). It is assumed that internal data reliability is appropriate if the value of the Alpha coefficient/ is at least 0.7.

4.4. Calculation of the Entropy Weights

The objective part of the VASMA weights was calculated applying the principles of the information entropy. Decision matrix $P$, where columns denote the set of criteria and rows denote the possible values k of the VAS scales $\left(k=1 \dots 100\right)$ was constructed from the data matrix $R$ (Table 3).

Values $p_{kl}$ presented in

Table 5. Decision matrix P for the entropy weighting.

k	C1	C2	C3	C4	C5	C6	C7	C8	C9	C10	C11	C12	C13
1	0.009	0.009	0.010	0.009	0.020	0.010	0.009	0.009	0.018	0.000	0.058	0.049	0.048
2	0.000	0.000	0.000	0.000	0.000	0.000	0.000	0.009	0.000	0.000	0.010	0.025	0.012
3	0.000	0.009	0.010	0.000	0.000	0.030	0.009	0.000	0.018	0.000	0.058	0.074	0.000
4	0.000	0.000	0.000	0.009	0.000	0.000	0.009	0.009	0.009	0.000	0.049	0.049	0.000
5	0.026	0.000	0.019	0.000	0.000	0.000	0.000	0.000	0.000	0.017	0.097	0.012	0.012
⋯	⋯	⋯	⋯	⋯	⋯	⋯	⋯	⋯	⋯	⋯	⋯	⋯	⋯
95	0.034	0.045	0.038	0.018	0.029	0.020	0.028	0.009	0.035	0.059	0.039	0.049	0.036
96	0.051	0.036	0.038	0.055	0.059	0.010	0.028	0.083	0.018	0.143	0.010	0.025	0.084
97	0.026	0.054	0.067	0.073	0.039	0.040	0.085	0.037	0.053	0.092	0.010	0.025	0.048
98	0.060	0.143	0.087	0.064	0.039	0.050	0.113	0.065	0.062	0.067	0.029	0.037	0.036
99	0.077	0.071	0.048	0.064	0.029	0.020	0.038	0.046	0.035	0.034	0.000	0.000	0.000
100	0.043	0.027	0.029	0.037	0.069	0.020	0.009	0.019	0.018	0.050	0.010	0.025	0.024

describe the proportion of responses $k$ for the analyzed criterion $l$ ($0\le p_{kl}\le 1$ and ${\sum }_{k=1}^{100}{p}_{kl}=1$). The calculation of the entropy weights presented in

Table 6. Entropy weights calculated from the survey data.

	C1	C2	C3	C4	C5	C6	C7	C8	C9	C10	C11	C12	C13
$\widetilde{E_l}\left(p\right)$	0.7886	0.7395	0.7935	0.7935	0.7828	0.8173	0.7660	0.7641	0.7768	0.7082	0.7813	0.7777	0.7654
$W_l$	0.2114	0.2605	0.2065 *	0.2065 *	0.2172	0.1827	0.2340	0.2359	0.2232	0.2918	0.2187	0.2223	0.2346
Rank	10	2	11	12	9	13	5	3	6	1	8	7	4

* The more precise weight value for the criterion C3 is 0.20652 and for the criterion C4 is 0.20648.

is explicitly described in Section 3.2.

4.5. Calculation of the WASPAS-SVNS Weights

The subjective part of the VASMA weights was calculated as the MCDM task where WASPAS-SVNS is involved for the preference elicitation. Decision matrix $X$ (

Table 7. Decision matrix $X$ for the WASPAS-SVNS criteria weighting.

	C1	C2	C3	C4	C5	C6	C7	C8	C9	C10	C11	C12	C13
V1	3.42	1.79	5.77	2.75	4.90	5.00	3.77	3.70	6.19	4.20	39.81	29.63	16.87
V2	8.55	6.25	7.69	12.84	14.71	15.00	7.55	4.63	6.19	4.20	25.24	19.75	20.48
V3	17.09	16.96	21.15	13.76	11.76	19.00	18.87	20.37	17.70	9.24	6.80	8.64	7.23
V4	41.88	37.50	34.62	39.45	42.16	45.00	39.62	45.37	47.79	37.82	18.45	25.93	32.53
V5	29.06	37.50	30.77	31.19	26.47	16.00	30.19	25.93	22.12	44.54	9.71	16.05	22.89
V6	4.69	5.82	5.12	5.19	4.76	3.95	5.27	5.29	4.75	6.08	1.99	3.12	4.15

), where columns denote variables V1–V6 and rows denote the analyzed preferences, was also constructed from the data matrix $R$ (Table 3). Construction of the decision matrix $X$ and the variables V1–V6 are explicitly described in the Section 3.2.

WASPAS-SVNS weights calculated as the score function for deneutrosophication of the joint generalized criteria are presented in

Table 8. WASPAS-SVNS weights calculated from the survey data.

	C1	C2	C3	C4	C5	C6	C7	C8	C9	C10	C11	C12	C13
$S\left({\tilde{Q}}_i\right)$	0.8492	0.8843	0.8239	0.8402	0.803	0.7935	0.8477	0.8565	0.8205	0.8596	0.3971	0.5381	0.6889
Rank	4	1	7	6	9	10	5	3	8	2	13	12	11

.

4.6. Calculation of the VASMA Weights

VASMA weights were calculated from the entropy weights $W_j$ and the subjective weights $S\left({\tilde{Q}}_i\right)$ by the Equation (23). The final VASMA weights and their ranks are presented in

Table 9. Final VASMA weights.

No.	Criteria Description	VASMA Weight	Rank
C1	Reputation among parents	0.0789	6
C2	Skills of the kindergarten teachers	0.1013	2
C3	Modernity of the teaching methods	0.0748	9
C4	Cooperation with parents	0.0762	8
C5	Free spaces in the proper age groups	0.0767	7
C6	Toys and equipment	0.0637	11
C7	Indoor safety and hygiene	0.0872	4
C8	Outdoor safety and hygiene	0.0888	3
C9	Opening hours	0.0805	5
C10	Distance from home	0.1102	1
C11	Distance from the bus stop	0.0382	13
C12	Tolerance for different cultures	0.0526	12
C13	Price	0.0710	10

.

As can be seen, Distance from home (C10) and the Skills of the kindergarten teachers (C2) were detected as the most important aspects of the kindergarten selection process in Vilnius. Tolerance for different cultures (C12) and the Distance from the bus stop (C11) were identified as the least important criteria.

5. Results and Discussion

VAS matrix is a set of the VAS scales placed in a single question. Since multiple data like the importance value and the ranking information can be gathered from a single survey question, the VAS matrix might be successfully exploited for the survey-based criteria weighting tasks. However, biases of the respondents and the psychometric features of the VAS scales should be carefully treated to avoid uncertainties in the preference elicitation results.

Scatterplots of the two criteria with the highest mean value and with the lowest mean value were generated to illustrate the tendencies in the data collected with the VAS matrix (Figure 3). Analysis of the data distribution shows that the majority of estimates are in the range of 60–100. This is in line with the research showing that direct weighting performed with the Likert-scales or the VAS-scales shows a tendency towards the positive skew of the collected data. On the cognitive side, this may also suppose that parents have a tendency to say that most of the analyzed aspects are important for assessing the quality of kindergartens.

It is also noteworthy to observe that assessments ranging from 40 to 60 were hardly ever provided by the respondents. It might be related to the design of the VAS scales, where the default position of the marker is placed in the middle between the two linguistic anchors. A non-moved marker can be understood either as the non-response situation, or as the cognitive answer that the criteria is neither important nor unimportant (value = 50). To prevent the uncertainties caused by the erroneous interpretation, we consider this situation as the missing data.

In the numerical example presented in this paper, missing data is noticed in 72.84% of the answers. The accuracy of the survey results is usually sought to be improved by ensuring an appropriate sample of the responses. However, recently the significant decrease in the response rate of the online polls can be noticed [59]. In these circumstances, the opinion of each respondent becomes increasingly important. Pairwise comparison approaches like AHP or SWARA are not able to deal with the missing data, but it is not an issue for the VASMA weighting. On the contrary, VASMA weighting exploits the non-response values to achieve the greater accuracy of the preference elicitation results.

5.1. Comparison of the Direct Rating and VASMA Weights

Direct weighting approaches like point allocation, direct rating, SMART, and SMARTER might be considered as the simplest criteria elicitation methods [48]. Direct rating (DR) is probably the easiest of them since criteria weights are assessed by purely asking the respondents to assign absolute values of the criteria. Since DR does not require any prior learning on the preference elicitation process, it might also be easily applied for the survey-based criteria weighting [9]. However, two important disadvantages are recurrently associated with the direct rating methodology: the high potential for biased information [46] and the tendency towards the low variance of the criteria weights [9,44]. A comparison of the DR and VASMA approaches was performed to reveal how the data processing technique integrated into the VASMA weighting methodology affects both the variability and the accuracy of the criteria weights. Both the direct rating and the VASMA weighting techniques employ VAS Matrix as the data collection technique, but the distinctive data processing procedures. While DR simply calculates the averages of the criteria weights proposed by the respondents, VASMA calculates both the subjective and objective weights for the preference elicitation. The criteria weights calculated with the direct rating and VASMA weighting approaches are compared in Figure 4.

Results presented in Figure 4 support the idea that DR is typically associated with the low variation of the criteria weights. DR weights calculated for the criteria C1–C10 slightly vary in the interval (0.0770, 0.0906), while the range of the VASMA weights is much wider (0.0637, 0.1104). Respect for the psychometric properties of the VAS scales and the awareness on the uncertainty of the collected data showed that VASMA weighting demonstrates the positive effect for both the equal weighting and the high bias issues that are the vast disadvantages of the DR technique.

5.2. Sensitivity Analysis

The sensitivity analysis was performed to study the consistency of the obtained ranking. Ranks of the two direct weighting techniques (point allocation and direct rating) and VASMA weighting were determined and compared (Figure 5). Two popular direct weighting techniques SMART and SWING were not included in the comparison because of the methodological differences in the data collection procedure [9].

A comparison of the criteria ranks reveals differences between the point allocation (PA), direct rating (DR), and the VASMA weighting approaches (Figure 5). Due to the different direct weighting methodologies, PA and DR techniques give significantly different results. Greater stability can be observed between the criteria ranks determined by the direct rating and the VASMA weighting approaches. However, the weight values calculated by the DR and VASMA weighting techniques noticeably differ. Since a little variation of the weight values is repeatedly identified as the downside of the straightforward DR technique, VASMA weighting can be chosen as the solution to this issue. Results presented in the Figure 4 prove that the variance of the weighting values for the criterions C10 (Distance from home), C2 (Skills of kindergarten teachers), C8 (Outdoor safety and hygiene), and C7 (Indoor safety and hygiene) are considerably wider when the novel preference elicitation technique VASMA weighting is applied.

6. Conclusions

Criteria weighting is an integral part of the multicriteria decision-making process. When the opinions of the wider audience are needed, electronic surveys may be successfully employed to collect data for the preference elicitation procedure. Since both the psychologists and psychometricians agree that humans are much better at making comparative judgments than at making absolute judgments, visual analogue scales (VAS) have been proposed as the affective data collection tool for the assessment of the respondents’ traits. However, survey-based criteria weighting processes are typically accompanied by the biases of the evaluators and the uncertainty of the experimental conditions. Besides, end-aversion bias and the positive skew are also the companions of the VAS based preference elicitation. The novel criteria weighting technique VASMA weighting respects the psychometric features of the VAS scales and analyzes the uncertainties caused by the survey-based criteria weighting. It is achieved by integrating the WASPAS-SVNS multicriteria decision making approach for the determination of the subjective weights and Shannon entropy for the calculation of the objective weights.

A numerical example analyzing the importance of the criteria that affect parents’ decisions regarding the choice of the kindergarten institution was performed to reveal the practicalities of the proposed methodology. The experiment presented in this paper revealed that the data processing technique integrated into the VASMA weighting methodology is able to overcome the main disadvantages of the direct rating technique—the high biases of the collected data and the low variation of the criteria weights.

In the future, it would be interesting to analyze why the last three criterions presented in the VAS matrix got significantly lower weights than the rest of them. Is it an accidental situation, or is it associated with their position in the VAS Matrix? An optimal number of the criteria that can be weighted with the VASMA weighting methodology also should be analyzed in the future. It would be also interesting to disclose how the number of respondents and their homogeneity affects VASMA weighting values.