Eliciting Weights of Significance of Criteria for a Monitoring Model of Performance of SMEs for Successful Insolvency Administrator’s Intervention

Abstract

Small and medium-sized enterprises (SMEs) are accounted for as a major part of the economy of the EU in terms of part of the population employed, turnover, value-added, etc. Causes of insolvency of SMEs can be different; they are categorized in the paper. A considerable shift from resolving cases of bankruptcy with the sole aim to satisfy creditors’ rights to augmenting and enhancing liquidation and reorganization procedures evolved interest of the authors in creating efficient bankruptcy prediction models and, in particular, methodologies for evaluation and monitoring of the performance of SMEs. In the paper, we reviewed several initiatives and instruments created by the EU for supporting SMEs. The paper laid a foundation for creating a more comprehensive methodology for evaluation of the state of a firm undergoing the process of reorganization. A hierarchy structure of criteria for the evaluation of SMEs was used in the paper; methodologies for eliciting weights of importance of criteria from experts and gauging the level of concordance of opinions of experts were applied. Resulting weights of criteria of performance of an insolvent SME were obtained; the importance of the managerial category of criteria was revealed. Prominent features of hierarchy structures and methodology of using the structure for calculating ultimate weights were described and demonstrated. Gauging concordance of opinions of experts revealed a satisfactory level of concordance of opinions of experts; this allowed to prepare the ultimate weights of criteria for multiple criteria evaluation of SMEs for further research.

1. Introduction

Adopted in the EU and stipulated in its official documents, the definition of small and medium-sized enterprises (SMEs) encompasses the following three criteria and thresholds. The category of micro, small, and medium-sized enterprises consists of enterprises, which employ fewer than 250 persons and have either an annual turnover not exceeding EUR 50 million or the total of its annual balance sheet not exceeding EUR 43 million [1]. Since 2005 in the EU, SMEs have been classified into three categories: micro (less than 10 employees and an annual turnover of under €2 million), small (if it has less than 50 employees, and its annual turnover is less than €10 million), and medium (if it has less than 250 employees, and its annual turnover is less than €50 million). The U.S. Small Business Administration imposed limits on the size in terms of the number of employees setting its maximum to be 500, while the average annual receipts should not exceed $7.5 million for a firm to qualify for the SME. According to the OECD (Organisation for Economic Co-operation and Development), the population of SMEs is diverse in terms of age, size, business model, performance, profile, and aspirations of entrepreneurs [2]. Even if the definition may vary, normally, it should comprise at least the size of the firm in terms of its assets and capital, its legal status, number of employees, degree of dependence, and industrial sector.

SMEs make a major component of the world economy, accounting for more than half of the global formal employment and contributing between 50% and 60% of national GDP on average between the OECD member-states [2]. In the EU-28, SME employment increased by 1% in 2017 and was forecasted to increase by 0.9% in 2018; the value-added increased by 2.5% in 2017 and was predicted to grow by 3.8% in 2018 [3]. In 2016, SMEs in the non-financial business sector of the EU-28 accounted for two-thirds of the total EU-28 employment (66.6%); and slightly less than three-fifths (56.8%) of the value-added generated by the non-financial business sector [3,4]. Numerous academic articles support the notion that efficient performance of SMEs significantly impacts economic growth in both developed and developing countries, and especially contribute to sustainable development [3,4,5,6].

Based on [3,4,7,8,9], SMEs can be characterized by the following distinct features:

• difficulties arising attempting to access sources of financing;
• SMEs are reliant on internal funds or cash from friends and family;
• lack of management competence, skilled staff, lack of regular training;
• lack of efficiency compared to large firms in screening the regulatory environment and dealing with legal requirements;
• comparably low productivity;
• employees perform multiple roles with unclear boundaries and job responsibilities, which results in rather inefficient flexible internal organizational structures;
• lack of financial safety and high reliance on a low number of customers, thus making finding new customers and sustaining established ones a pressing issue for the majority of SMEs;
• dependence on few suppliers, lack of strong networks and links between businesses;
• lagging to utilize new opportunities, such as new markets, supply chains, or simply gaining support and advice;
• isolation, insufficient connection to ecosystems, where ideas and skills can be shared;
• focus on competition rather than collaboration;
• owner’s permanent fear of bankruptcy.

As SMEs are making up a considerable part of the economy achieving their sustainable performance, particularly during periods of crises, they are of a great challenge. Naturally, insolvency and subsequent bankruptcy of SMEs are a major threat to such sustainable performance of SMEs. Consequently, the aim of the bankruptcy law is now undergoing the process of a considerable modification from solely ensuring creditors’ rights against insolvent market participants towards augmenting and enhancing liquidation and reorganization procedures. In this decade, in many countries (e.g., France, Germany, Italy, etc.), the insolvency law was amended to increase the efficiency of liquidation or reorganization of companies after their bankruptcy. In this context, the insolvency administrator plays a very important role in evaluating companies’ performance and creating restructuring plans. Contrary, in many countries, small firms, after facing financial difficulties, are still often rapidly closed down without any proper insolvency procedures. These firms are simply erased from commercial registers [10]. Business entry and business exit are natural processes; quick resolution and/or revival are of utmost importance. Procedures of liquidation or reorganization could be made more efficient if a thorough and rapid SME viability evaluation were performed. From the historical data collected by the World Bank [11], a strong and direct link between the efficiency of bankruptcy procedures and the magnitude of the loss of company value is observed.

Another aspect that still has to be tackled is the traditional fear by the management of consequences of bankruptcy. The possibility of going bankrupt was noted by 49 percent starting business respondents as the foremost risk [12]. Bankruptcy generates huge direct and indirect costs for owners of small businesses, including loss of self-esteem, loss of personal collaterals, loss of employment. It is, therefore, important to mitigate liquidation, to mingle it or make attempts to revert it to reorganization, and reserve such a possibility in legal bankruptcy and debt settlement procedures.

Unique characteristics of SMEs, as well as their importance for economic development, create a strong impetus for policy-makers to improve the favorable environment for SME viability. In the EU, several instruments for supporting SMEs were created. The current policy of the European Commission is primarily concentrated in five priority areas: promotion of entrepreneurship and managerial skills, improvement of SMEs’ access to markets, cutting the red tape, improvement of the growth potential of SMEs, and strengthening dialogue and consultation with SME stakeholders [3]. As it became widely understood that in Europe, in general, business failure leads to social, economic, and legal stigmatization of the failed entrepreneur, creates an obstacle to entrepreneurship and entrepreneurial initiative, in 2000, in its political agenda, the European Commission proclaimed the ‘second chance’ initiative [13]. A number of other efficient policy measures were implemented in the European Union as the ‘second chance principle’ since 2008, such as ensuring the limit of one year for the length of the bankruptcy procedure, ensuring that re-starters are treated equally, etc. As a result, the average time for resolving insolvency has considerably decreased. In most countries of the EU modern, restructuring procedures are implemented, with few exceptions, thanks to through and significant transformations of local insolvency laws over the past decade. Unfortunately, the majority of countries of the EU did not yet ensure that legal bankruptcy procedures are completed within a year. Just a few EU countries launched national information campaigns to reduce the stigma of failure and facilitated successful restarting opportunities for honest bankrupt entrepreneurs. In Europe, around 50% of enterprises do not survive the first five years after establishment. Bankruptcies account in average 15% of all business closures. Such statistics attaches importance to the possibility of either quick resolution or revival of insolvent firms.

Insolvency resolution laws have substantial deviations among countries. Japan, Finland, and US can serve as benchmarks as they were ranked as the top three countries by The Doing Business project of the World Bank in the resolving insolvency category [11]. In the US, bankruptcy has become a historically well-established process. It is designed to help businesses to eliminate or repay their debts under the guidance and protection of the bankruptcy court. Business bankruptcies are described either as liquidations or reorganizations depending on the type of bankruptcy. Chapter 11 of the US Bankruptcy Code is an important part of a legal framework for such businesses that opted for having the future, the businesses that may be reorganized under a court-appointed trustee. The owner of the insolvent company may also be a trustee if agreed by the parties involved. Chapter 13 of the US Bankruptcy Code allows individuals with a regular income for the restructuring of debts for three or five upcoming years. It can also be used for proprietorships [14].

In particular, the diversity of SMEs requires flexibility in the reorganization process. Consequently, the EU initiated a framework of early warning systems followed by three stages of revival of failing SMEs, consisting of:

• out-of-court settlements;
• in-court legal procedures;
• post-bankruptcy treatment of the entrepreneur and creating conditions for a second-chance revival (liquidation or discharge),

which, in many cases, enable to replace liquidation or settlement or reorganization imposed by the court [13].

The European Commission also undertook an initiative to unify bankruptcy laws among member countries. The recommendation encourages the EU member states to “put in place a framework that enables the efficient restructuring of viable enterprises in financial difficulty” and to ensure establishing “minimum standards on…preventive restructuring frameworks” [15].

As mentioned above, insolvency laws were significantly transformed in Europe over the past decades. Nevertheless, considerable disproportions across the EU remain since “there are some countries where such procedures are outdated or may be cumbersome and inefficient and have the effect of transfer wealth to out-of-the-money creditors and shareholders” [15]. Since in EU countries, contemporary restructuring procedures exist not in all member states, such unification is important, especially for the lagging countries. Parameters that represent the efficiency of bankruptcy procedures across the countries also substantially differ. For instance, the recovery time in EU countries in 2016 varied from 0.9 to 4 years, compared to 1.5 years in the US [16].

Described tendencies in the EU legislation encourage scientists to design bankruptcy prediction models, to create methodologies of measurement quality of performance and management of SMEs, both to prevent insolvency and to have an effective monitoring tool. This, in turn, requires establishing a set of criteria, which describe the performance of SMEs.

The current paper laid a foundation for creating a more comprehensive methodology for the evaluation of the state of a firm undergoing the process of reorganization. A hierarchy structure of criteria for the evaluation of SMEs was used in the paper; methodologies for eliciting weights of importance of criteria from experts and gauging the level of concordance of opinions of experts were applied. Resulting weights of criteria of performance of an insolvent SME were obtained; the importance of the managerial category of criteria was revealed. Prominent features of hierarchy structures and methodology of using the structure for calculating ultimate weights were described and demonstrated. The ultimately obtained weights of criteria would be used in subsequent MCDA (multiple criteria decision-aid) evaluation of SMEs.

2. The set of Criteria Intrinsic to Performance of SMEs, and its Hierarchy Structure

In the abundant literature, on the evaluation of the performance of SMEs, there are papers, which categorize criteria and thus lay down a foundation for the creation of a hierarchy structure of criteria. Performance indicators can be derived based on the company’s functions (financial, marketing, etc.). For example, [17,18,19,20] investigated companies active in logistics, production, supply chains, etc.

We used the universal structure of criteria in this paper, suitable for most types of SMEs. Kurschus et al. [21] proposed the set of criteria for SME performance measurement, which was taken as the baseline in this analysis (

Table 1. List of soft criteria of SME performance.

Category	Soft Sub-Criteria
S1. Shareholders/owners	(a) Shareholders are anxious about total loss; (b) Shareholders’ capability to generate capital, or a potency to involve new shareholders and to raise equity; (c) Shareholders’ capability to make a valuable personal commitment
S2. Management	(a) Experience with crisis situations; (b) Lost confidence; (c) Projects development know-how; (d) Product know-how; (e) Process know-how; (f) Market know-how; (g) Capability to measure the extent of crisis; (h) Capability to communicate to all the groups of interests; (i) Interests conflicts among involved partners; (j) Value of the intangible property; (k) Availability of trustful information; (l) Communication quality; (m) Globalization problem; (n) Technical and technological changes; (o) Dependability from customers; (p) Dependability from suppliers; (q) Political developments; (r) Economic developments; (s) License risk; (t) Patent risk; (u) Development of products
S3. Personnel	(a) Confidence of the depending employees in management; (b) Anxiety about losing the workplace; (c) Anxiety about losing the remuneration; (d) The anxiety of the employees’ representatives due to the reduced influence; (e) Management’s capability to develop initiatives also during the crisis; (f) Human resources
S4. Customers	(a) Compromising of performance relationships tends to seek for substitution; (b) Compromising of performance relationships leads to extended payment terms; (c) A negative influence of the customers upon the competitive situation
S5. Suppliers	(a) A threat of the bad-debt losses leads to advance payments and eventually to the suspension of deliveries; (b) Product reliability and requirements of the service agreement
S6. Competition	(a) Market pressure; (b) Price pressure; (c) Product pressure
S7. Finances	(a) Third-party concern about losing its security and/or collateral; (b) Concern about the value adjustment pressure; (c) Concern about the high pressure to be sold to the third party; (d) The reaction of the credit institutions; (e) Financial resources; (f) Currency-related risks; (g) Investment-related risks; (h) Risks of borrowing; (i) Adequate coverage of the company assets by insurance; (j) Fire outbreak, energy crises, and other emergencies
S8. Rehabilitation concept	(a) Quality of the newly developed concept; (b) Optimal involvement of the remaining potential of the whole enterprise

Source: [21].

and

Table 2. List of hard criteria of SME performance.

Category	Hard Sub-Criteria
H1. Adverse balance, negative balance	(a) Level of depreciation and amortization; (b) Possession of the share capital; (c) Possession of the equity capital
H2. Liquidity	(a) Cash liquidity (1st degree liquidity); (b) Current ratio (2nd degree liquidity); (c) Quick ratio (3rd degree liquidity); (d) Working capital; (e) Measures for securing liquidity
H3. Net sales and profit	(a) Decrease in profit; (b) The decrease in net sales
H4. Personnel intensity	(a) Personnel intensity-personnel costs/operational performance; (b) Personnel costs resulting from wages; (c) Personnel costs resulting from salaries; (d) Personnel costs resulting from social security expenses
H5. Material intensity	(a) Material investment; (b) Material costs; (c) Operating performance
H6. Funding ratio	(a) Funding ratio I; (b) Funding ratio II
H7. Debt ratio	(a) Debt ratio-borrowed capital/equity capital; (b) Statistical debt ratio to perform analysis of the capital structure; (c) Dynamic debt ratio
H8. Equity ratio	(a) Equity ratio; (b) Financial stability of the enterprise; (c) Financial dependence of the enterprise
H9. Yield key figures	(a) Return on equity; (b) Return on total assets; (c) Cash flow
H10. Turnover key figures	(a) Return on sales; (b) The efficiency of plant and equipment, material, and manpower; (c) Turnover rate

Source: [21].

). As the number of criteria appeared to be considerable, the categorization of criteria was important both to comprise all areas of activity of an SME investigated and to be able to elicit weights of criteria.

There are different approaches in the literature, both in terms of the set of criteria used in the analysis of firms and their categorization, like the seminal balanced scorecard (BSC) performance managerial system, which combines hard and soft ratios, was introduced by Kaplan and Norto [22]. The BSC provides criteria and a framework for measuring the performance of a company based on four main aspects: financial, internal business process, customer, and learning and growth. The BSC assesses data that represents the financial state of a company, evaluates information about customer’s perception of the firm’s products, and appraises the possibilities of the firm to educate its human resources [22,23]. Most prediction models also use microeconomic financial ratios (e.g., net income to total assets, total liabilities to total assets, etc.), some use macroeconomic indicators (e.g., inflation, interest rate changes, etc.), and variables that describe market (e.g., firm’s equity price, lagged cumulative security residual return, etc.) [24,25,26]. In order to create a universal structure of criteria, the initial broad categorization, to hard (financial) and soft (managerial) categories, was made.

A group of researchers is evaluating firms based on hard criteria, and such criteria might be derived from the scientific papers by this group. The economic value added (EVA) developed by Stern and Stewart in 1991 emphasized the creation of value by management for the owner’s accounting cost of capital employed [27]. Bahri et al. [28] presented the EVA model for SMEs where financial ratios used for evaluation were expanded and augmented net operating profit after tax, cost of capital rate, and cost of invested capital. Hard criteria are usually used in bankruptcy prediction models. Altman and Sabato [29] are often mentioned as being pioneers in developing a bankruptcy prediction model for SMEs.

Ausloos et al. [30] demonstrated that the level and timing of investment allows making a forecast of performance of a small or medium-sized enterprise at the time of the financial crisis. The evaluation of business performance in that paper is based on four financial or “hard” variables: two variables belong to the “growth” category: sales variations and total assets variations, and two belong to the ‘‘profitability’’ category: returns on investments and returns on sales. The authors concluded that robust resistance to crises could be found in firms, which assets grew substantially over the few years of activity, such a dynamic variable as the growth of assets is appropriate to use.

Kotanea and Kuzmina-Merlinob [20] suggested to use the following financial indicators for the evaluation of SME performance in the transport sector: gross profitability, turnover of receivables, return on sales (ROS), return on equity (ROE), turnover of payables, turnover of inventory, and return on assets. They also suggested to include non-financial, or “soft”, indicators in the future studies. Thus, the evaluation of the performance of firms should be considered holistically, reflecting the importance of managerial strategic criteria, which are drivers of their future performance. The firm’s managerial performance described by soft criteria influences its performance, described by hard criteria [31]. Sanchez and Marin [32] also used a broader set of categories for the criteria of performance of SMEs: level of competitiveness, financial, and social. Bianchi et al. [33] included profitability, productivity, and market criteria into their analysis, thus comprising both soft and hard criteria. Sarwoko et al. demonstrated that the entrepreneurial competencies of the management, such as conceptual competency, opportunity competency, relationship competency, learning competency, and personal competency, as well as entrepreneurial characteristics of management, such as psychologic traits, experience, the influence of the family [34], and desire for business growth [35], make effects on financial performance of the firm. The role of the management of SMEs is crucial to successfully compete and survive during a turbulent period.

Financial ratios are often recognized to be the most important indicators used for measuring the performance of firms, but they are built based on historical financial data, which, in most cases, are lagging variables.

The new trend to incorporate the concept of sustainable development into research boosted the inclusion of sustainable competitive strategies into the evaluation criteria of firms’ performance [36,37,38]. Seo and Chae [39] emphasized the importance of innovative activities that are related to experiences, knowledge, etc. Ruiz-Mallorqui and Aguiar-Diaz [40] showed another aspect of soft managerial criteria. Maintaining relations with only one major bank or the largest bank in a country proved to increase the likelihood of reorganization versus liquidation of an enterprise.

The system is formed in the hierarchy structure containing two broad types of criteria, qualitative and quantitative, which are also called soft and hard criteria, respectively (Table 1 and Table 2). The division of criteria into a multi-layer hierarchical system is plausible whenever an object of evaluation is described using a large number of criteria of different dimensions. For example, decision-making problems in the realm of sustainability are described using three sustainability dimensions: environmental, social, and economic in renewable energy technologies [41]; or in civil engineering, construction, and building technologies [42]. In [43], there are outlined criteria of quality of a hierarchical structure-completeness, redundancy, operationality, decomposability, minimum size. Such qualities of a good hierarchy should be kept in mind at the stage of creating a set of criteria because constructing a hierarchy considerably helps to solve a problem of dealing with a large set of criteria by reducing its complexity at the lowest level of sub-criteria. Inherent to hierarchical structures, flexibility allows one to effectively modify and adjust the set of criteria to satisfy the desires of a decision-maker.

In view of the above, and observing criteria outlined in Table 1, we might derive the specific feature that discerns SMEs: such enterprises more heavily rely on management; their results, failure, or sustainable development depend on the quality of management in various aspects. This leads to the outcome that such categorization of criteria into “hard” and “soft” is particularly justified for SMEs.

Completeness factor requires all objectives important to the decision-maker insolvency administrator to be included in the structure. The inclusion of both soft and hard criteria results in the fact that both qualitative and quantitative factors are encompassed. The hierarchical structure comprises elicited from the literature comprehensive set of categories, thus making it possible to collect the sub-criteria at the lowest level easier; in our case, the number of criteria falls in between two and twenty-one in each category. Consequently, the task of obtaining the structure with the minimum-size quality of a hierarchical structure is reduced to checking only criteria within each category. Thus, such a hierarchy structure allows having a relatively moderate (with two exceptions for soft categories of management and finances) number of sub-criteria within each category. This makes it possible to satisfy both redundancy and operationality criteria of quality of a hierarchical structure: duplicative criteria can be easily excluded to retain distinction, and meaningful criteria added. The decomposability criterion is met by the logical categorization of criteria to four levels.

3. Methodology and Results

3.1. Eliciting Weights of Criteria

After the hierarchical structure of criteria is observed, their weights can be technically derived more easily. Weight-allocation methods can be broadly categorized as subjective, where knowledge of experts is used, and objective, based on the analysis of data. Such methods could be also called a priori and a posteriori weights, respectively [44]. Usually, objective weights are used in such cases when a quality cast of experts cannot be created. There are interesting combinations of both types of weight-elicitation methods [45,46].

The cast of experts was widened from 20 [47] to 26. These are qualified experts working in bankruptcy administration of SMEs, employees of rescue divisions of credit institutions, court officials for insolvency, and rescue managers; the subjective methods were chosen.

The ability to quantify the opinions of respondents is restricted by psychological factors and particularities of the functioning of the human brain. Various quantitative scales of measurement, which translate qualitative perceptions into real or integer numbers, began to appear only in the 20th century. The foundation of the scaling was laid by Stevens, who provided four types of scales of measurement - nominal, ordinal, interval, and ratio [48,49]. Methods of eliciting the power of judgment of respondents that make an assumption that responses are distributed in accordance with the normal distribution, such as the Thurstone scale [50] or Guttman scale [51], appear complicated and time-consuming. For unfamiliar users, the method AHP (analytic hierarchy process), proposed by Thomas L. Saaty, also is rather complicated. For S2 (management), S3 (personnel), and S7 (finance), which contain 21, 6, 9 criteria, respectively, the usage of the AHP is not possible because the method uses the 5-point scale, only sometimes the 9-point scale, which allows discerning only, limited by the range of the scale number of criteria. The pairwise comparison of criteria within each category is thus impossible using the Saaty scale [52]. A method FUCOM (full consistency method) uses a similar idea of pairwise comparison of the significance of criteria but expresses magnitudes in the scale of real numbers [53,54].

The experts were surveyed for the estimation of weights of importance of criteria. The percentage scale was used. The weights of hard and soft categories and the weights of criteria within each category made a total of 100% in each set of responses. The average values were calculated among all the participating experts. Both third (the lowest) and the second (the middle) levels of the hierarchy were joined using the following formula (1) for each expert r participated [55]:

$${\omega }_{j,r}={\omega }_{j_{k,r}}·{\omega }_{k,r},$$

(1)

where the index k denotes categories; $j_k$ is the index of a criterion within the category k. It is easily shown that the sum of all weights of evaluation equals to 100% as well (for every expert), which satisfies the requirement for weights per se.

Aiming to obtain consistent estimations with the group of experts, the Delphi technique was chosen [36,56,57]. It is based on the aggregation-disaggregation MCDA (multiple criteria decision-aid) paradigm presented in [58] (Figure 1). The model increases the probability of success in eliciting concordant opinions from a group of experts.

3.2. Statistical Analysis of Correlation

As already mentioned, both weights related to the evaluation of an SME before the crisis and after the crisis. We processed these two sets of elicited data from the 26 experts separately. We denoted the number of experts as r; r = 26. In our analysis, we used weights obtained at the two levels of the hierarchy joined using formula (1). At the initial stage, the pairwise correlation analysis of final weights corresponding to each pair of experts was performed. The results of the correlation of hard criteria before the crisis are presented in

Table 3. Correlation between final weights (before crisis case).

No. Experts	2	3	4	5	6	7	8	9	10	11	12	13	14	15	16	17	18	19	20	21	22	23	24	25	26
1	0.52	0.42	0.39	0.41	0.56	0.49	0.74	0.33	0.41	0.44	0.19	0.42	0.44	0.74	0.61	0.70	0.63	0.39	0.59	0.67	0.67	0.58	0.45	0.53	0.44
2	–	0.45	0.45	0.52	0.57	0.54	0.77	0.27	0.53	0.25	0.19	0.44	0.08	0.70	0.53	0.41	0.33	0.22	0.52	0.51	0.68	0.29	0.13	0.56	0.45
3		–	0.65	0.21	0.52	0.52	0.31	0.01	0.23	0.18	0.35	0.36	0.36	0.53	0.59	0.57	0.54	0.64	0.57	0.63	0.40	0.60	0.40	0.53	0.99
4			–	0.38	0.53	0.80	0.45	0.05	0.39	0.02	0.06	0.32	–0.02	0.47	0.82	0.34	0.26	0.29	0.47	0.55	0.51	0.23	0.06	0.79	0.65
5				–	0.82	0.61	0.74	0.65	1.00	0.24	0.13	0.65	–0.01	0.38	0.47	0.28	0.18	–0.01	0.74	0.52	0.86	0.17	0.12	0.65	0.23
6					–	0.74	0.74	0.50	0.82	0.36	0.49	0.74	0.23	0.57	0.69	0.57	0.45	0.33	0.82	0.83	0.87	0.43	0.34	0.77	0.53
7						–	0.66	0.41	0.61	0.02	0.13	0.36	0.04	0.55	0.85	0.44	0.31	0.21	0.64	0.67	0.74	0.27	0.16	0.99	0.51
8							–	0.49	0.74	0.25	0.08	0.51	0.12	0.75	0.65	0.51	0.39	0.14	0.66	0.60	0.94	0.35	0.22	0.68	0.31
9								–	0.65	0.15	0.09	0.28	–0.14	0.21	0.11	0.01	–0.04	–0.10	0.40	0.18	0.54	–0.02	–0.04	0.45	0.01
10									–	0.24	0.13	0.66	–0.01	0.39	0.47	0.29	0.19	0.00	0.75	0.53	0.86	0.18	0.13	0.65	0.25
11										–	0.43	0.66	0.05	0.08	0.09	0.25	0.24	0.20	0.13	0.48	0.29	0.18	0.05	0.08	0.18
12											–	0.55	0.35	0.25	0.13	0.28	0.29	0.56	0.34	0.52	0.18	0.34	0.35	0.13	0.35
13												–	0.14	0.36	0.30	0.32	0.32	0.34	0.54	0.72	0.64	0.31	0.29	0.42	0.38
14													–	0.61	0.26	0.73	0.80	0.79	0.59	0.45	0.18	0.81	0.96	0.05	0.40
15														–	0.64	0.71	0.70	0.66	0.75	0.64	0.70	0.70	0.67	0.55	0.55
16															–	0.71	0.57	0.31	0.67	0.73	0.71	0.49	0.28	0.83	0.58
17																–	0.96	0.62	0.73	0.76	0.58	0.90	0.67	0.46	0.57
18																	–	0.75	0.70	0.72	0.48	0.97	0.75	0.35	0.54
19																		–	0.58	0.58	0.23	0.80	0.80	0.22	0.64
20																			–	0.76	0.81	0.71	0.67	0.67	0.58
21																				–	0.73	0.68	0.52	0.70	0.63
22																					–	0.46	0.30	0.77	0.40
23																						–	0.79	0.31	0.60
24																							–	0.17	0.45
25																								–	0.52

.

The average correlation appeared to be 0.459, which is positive and quite large. For the chosen level of precision α = 0.05, 224 or 68.92% of cases satisfied the following test statistics TS of correlation for variables distributed in accordance with multivariate normal distribution, as stated in formula (2):

$$\mathrm{TS} = \frac{\mathrm{R}\sqrt{\mathrm{n}-2}}{\sqrt{1-{\mathrm{R}}^2}}$$

(2)

as in such cases, TS appeared to be larger than the threshold value of the one-sided t-distribution for r–2 = 24 degrees of freedom, which is 1.71. This means that in 71.79% cases, the value of test statistics appeared to fall into the interval with a relatively small probability of 5%, and we, therefore, in such cases might reject the hypothesis that weights elicited from the pairs of experts are non-correlated with 5% probability of making a mistake.

Analogously, for soft criteria before the crisis, the average correlation was 0.46. For the chosen level of precision α = 0.05, 226 or 69.54% of cases satisfied the test statistics TS for the significance of correlation (2) for variables distributed in accordance with a multivariate normal distribution.

Consider now the case after the crisis. For hard criteria after the crisis, the average correlation was 0.52. For the chosen level of precision α = 0.05, 261 or 80.31% of cases satisfied the following test statistics TS of correlation for variables distributed in accordance with a multivariate normal distribution. For soft criteria after the crisis, the average correlation was 0.44. For the chosen level of precision α = 0.05, 229 or 70.46% of cases satisfied the following test statistics TS of correlation for variables distributed in accordance with a multivariate normal distribution. The results appeared to be quite satisfactory and, in most, pairs of experts’ opinions correlated; nevertheless, the results applied not to all pairs of experts.

Under a recommendation of an anonymous reviewer, we performed the analysis of the significance of average values of the correlation coefficient, for each expert, and each case of four in terms of time to crisis and type of criteria (hard and soft). It was revealed that all average values were above the significance thresholds of the chosen level of precision α = 0.05.

A rank correlation test simultaneously between all 26 experts based on Kendall’s theory of concordance [59] should answer whether opinions of experts are concordant in the whole group of experts.

3.3. Nonparametric Statistical Analysis of Concordance of Opinions of Experts

The performance of the non-parametric test of concordance of opinions of experts allows estimating the likelihood of the fact that opinions of experts within the group are similar. A decision-maker decides after such an estimation which steps should be taken in case opinions appear to be non-concordant. Usually, after opinions appear to be concordant, no extra steps are required, and averages of the opinions of experts expressed in terms of weights are taken. Such resulting weights are used in the subsequent research as in [60,61,62,63].

For the theory of concordance to be applied, weights of criteria were transformed to ranks, for each expert. Such ranks we denoted as e_ik, where i = 1, 2,..., m is the index of criteria, while k = 1, 2,..., r is the index of denoting experts (r is the number of responded experts, counting 26 in our case). Chi-squared test statistics use Kendall’s variable W, which was calculated by summing of squared deviations of all ranks e_ik by all experts

$$e_i={\displaystyle \sum _{k=1}^r{e}_{ik}}$$

(3)

and by measuring the sum of squared deviations from the mean of such sums

$$\overline{e}=\frac{{\displaystyle \sum _{i=1}^m{e}_i}}{m}.$$

(4)

Consequently, Kendall’s variable W equals to the ratio between the largest deviation observed in the concordant case denoted by S_max (5)

$$S_{\max }=\frac{r^2\cdot m\cdot \left(m^2-1\right)}{12}$$

(5)

and the sum S mentioned above, calculated by the formula (3).

$$S={\displaystyle \sum _{i=1}^m{\left(e_i-\overline{e}\right)}^2}$$

(6)

Consequently,

$$W=\frac{S}{S_{\max }}=\frac{12\cdot S}{r^2\cdot m\cdot \left(m^2-1\right)}$$

(7)

Chi-squared test statistics for this variable is as follows [59]:

$${\chi }^2=W\cdot r\cdot \left(m-1\right)=\frac{12\cdot S}{r\cdot m\cdot \left(m+1\right)}$$

(8)

The number of degrees of freedom υ = m – 1. For performing the test statistics, the level of significance α = 0.05 was chosen. For the cases when ranks were equal, the following adjustment of the coefficient of concordance was applied [58]:

$$W=\frac{12\cdot S}{r^2\cdot m\cdot \left(m^2-1\right)-r\cdot {\displaystyle {\sum }_{\phi }\left(t_{\phi }^3-t_{\phi }\right)}}$$

(9)

where φ denotes the sets of equal ranks, and t_φ denotes the number of equal ranks within a set within φ. Equal ranks were found in the matrix of ranks. Nevertheless, it is seen from formula (7) that the equal-rank adjustment will only increase Kendall’s variable and, simultaneously, the resulting test statistics.

The calculations of values of test statistics are presented in

Table 4. χ² test statistics for the null hypothesis of the discordancy of opinions of experts.

Group of Elicited Weights of Criteria	χ2 Test	χ2 Threshold at α = 0.05	χ2 Threshold at α = 0.01
Hard before crisis, 30 degrees of freedom	481.41	43.77	50.89
Soft before crisis, 49 degrees of freedom	669.08	66.34	74.79
Hard after crisis, 30 degrees of freedom	502.83	43.77	50.89
Soft after crisis, 49 degrees of freedom	677.35	66.34	74.79

for all the four cases. For the case of weights of hard criteria before the crisis, χ² test statistics appeared to be equal to 484.41; for the case of weights of soft criteria before the crisis, the test statistic value result was 669.08; for the case of weights of hard criteria after the crisis, the value of the test was 502.83; and for the case of weights of hard criteria after the crisis, the value of the test was 677.35. The threshold χ² for the level of significance α = 0.05 was 43.77 for hard criteria case with 30 degrees of freedom, and 66.34 for soft criteria with 49 degrees of freedom. Test statistics appeared to be much larger than the corresponding thresholds. Therefore, as there is a low probability of making a statistical mistake to treat a discordant case as concordant, with a high degree of probability, we might reject the null hypothesis about the assumption that opinions of the experts were discordant in all four cases considered.

The averages of weights elicited from experts as they were found to be concordant would be used in the monitoring model of successful post-crisis intervention in the case-study of 26 SMEs, which would be described in a subsequent paper. The final weights are graphically depicted in Figure 2 and Figure 3. The average weights of importance that express the state of a firm before the crisis that experts allocated are depicted in blue color, while weights of importance that express the state of a firm after the crisis are depicted in red.

It is seen that weights of importance of criteria before and after the crisis are different. We plan to evaluate how weights affect the dynamics of the shift of the state of the firm in a subsequent paper.

4. Conclusions

SMEs are accounted for as a major part of the economy of the EU in terms of the employed part of the population, turnover, value-added, etc. Considerable particularities of SMEs were noted in the paper, in particular, the importance of the performance of the management of SMEs, for ensuring their sustainable development. The managerial dimension of performance characteristics of SMEs was added to the list of criteria of performance of SMEs along with the financial dimension. Causes of insolvency of SMEs were analyzed in the paper; particular causes for SMEs were listed. A substantial shift from resolving cases of bankruptcy with the sole aim to satisfy creditors’ rights to augmenting and enhancing liquidation and reorganization was observed in the EU; several initiatives and instruments for supporting SMEs were created. This induces interest in creating efficient bankruptcy prediction models and, in particular, methodologies for the evaluation and monitoring of the performance of SMEs. The paper attempted to build a foundation for such a methodology. It proposed a hierarchy structure of criteria for the evaluation of SMEs and applied methodologies for eliciting weights of importance of criteria from experts and gauging the level of concordance of opinions of experts. The statistical non-parametric test on concordance of opinions of experts retrieved satisfactory results, and opinions of the experts appeared to be concordant. This allows using the ultimately obtained weights of criteria for multiple criteria MCDA evaluation of SMEs in further research with guidelines as were set out in [64,65,66].