An Index to Measure the Sustainable Information Society: The Polish Households Case

Abstract

Since the 1960s, a turning point for civilization related to the increasing role of information and communication technologies (ICT) in creating the modern society can be observed. The sustainable information society (SIS) entails a new phase of information society development, in which the ICT adoption by the society stakeholders is the key enabler of sustainability. One of the most important issues in the debate about the SIS is the means to measure it. In this paper, a methodological framework for creating SIS evaluation indexes is presented. In practical terms, an exemplary adjustable index for evaluating SIS in households is created based on the presented approach. During the empirical research, the presented index is used to evaluate SIS in Polish households. The study showed the usefulness of the proposed approach and confirmed the fact that it can be used to build other SIS indices.

1. Introduction

The turning point for civilization, initiated around the 1960s, is related to the increasing role of information and communication technologies (ICT) in creating a modern society. This society is referred to as the information society [1,2]. In this society, business, public administration, and everyday life have been profoundly influenced by ICT. The potential of the information society for economic growth and social welfare is enormous [3], but, regrettably, it can also be a source of threats and dangers. It can contribute to information and digital exclusion, new social divisions and social stratification, economic diversification, loss of privacy, information and computer crimes [4,5].

Due to the aforementioned causes, researchers have explored the areas where the information society, the ICT adoption, and sustainable development come together [6,7,8]. The concept of the sustainable information society (SIS) derives from this exploration [9,10,11,12,13]. The SIS entails a new phase of information society development in which the ICT adoption by society stakeholders, namely household, enterprises, and public administration, is becoming a key enabler of sustainability and its various kinds, namely economic, social, cultural, political, and environmental [11,14].

After extensively searching the literature it can be noticed that the SIS requires in-depth studies. The measurement is one of the important issues in the debate about the SIS. Already a few decades ago, Drucker (or maybe Deming) said, “if you can’t measure it, you can’t manage it” [15]. DeMarco [16] also pointed out the need to measure performance: “You can’t control what you can’t measure.” To make rational personal, business, and political decisions regarding various kinds of sustainability, we need to have quantitative tools for measuring the state of SIS. These tools should allow definition of the direction of desirable actions aimed at developing SIS and evaluate their effects.

Around the world, inter alia in UN and OECD agencies, many institutions, organizations or European Union committees, intensive theoretical and practical research has been underway to create society measurement indicators. Among the available sources of information, the following research and reports deserve mentioning: International Communication Union [17], Organization for Economic Cooperation and Development [18], World Economic Forum [19] or European Union [20]. Unfortunately, however, the indicators, indices and methods developed and used so far are often very controversial [21]. Moreover, the continuous development of technologies leads to the necessity of their constant update and modification. Regardless of the measuring tool selected, the measurement can provide the following functions [22,23,24,25,26]:

• Monitoring of the SIS in households, enterprises, and public administration;
• Providing the foundations for formulating the objectives and development strategies for SIS;
• Supporting the business and political decision-making processes in enterprises, public administration, or government regarding the investment in SIS;
• Evaluation of the progress towards achieving the stated milestones of SIS development.

A profound literature review allowed identification of a set of carefully selected multiple-criteria decision analysis (MCDA) methods as a feasible solution fulfilling the functions mentioned above, thus confirming their aptness to sustainability measurement. The authors’ contribution in this paper is to propose a methodological framework supporting SIS evaluation and, at the same time, supporting decision-making and adjusting the stakeholders’ preferences at different levels of criteria hierarchy structure. A practical contribution in the SIS domain is also provided, in the form of an MCDA-based households’ SIS index AdjSI, which demonstrates the fulfillment of the requirements above by the proposed methodological framework. The practical aim is achieved by means of survey questionnaires and MCDA methods, especially Analytic Hierarchy Process (AHP), Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS) and Preference Ranking Organization Method for Enrichment Evaluation (PROMETHEE). Explicitly, the practical case is presented on a set of over 600 households from one of the Polish geopolitical regions.

Consistent with the aim, the remainder of the paper is organized as follows. Firstly, the state of the art regarding the SIS and its measurement is presented. Secondly, MCDA application to sustainability issues is identified and explained, followed by MCDA-based sustainability indexes. Next, the methodological foundations of the authors’ MCDA-based adjustable index of SIS are presented, followed by an empirical study of the Polish households’ SIS. Discussion, implications, limitations, and future works conclude the paper.

2. State of Art

The terms of sustainability and sustainability science as a whole are considered to be an emerging discipline nowadays [22]. The challenges of the discipline are oriented on handling environmental, social and economic aspects/issues and relationships between nature and humankind [22]. Furthermore, sustainability offers a holistic approach, capable to gather and manage the sectorial knowledge as well as to entail multidisciplinary aspects [27]. Consequently, the complexity and the multidimensional facets of sustainable development force the researchers to discover new models and approaches to the emerging field of sustainability science. Former research concerning the sustainability science was determined by seven core questions proposed by Kates et al. in 2001 [28] (updated in 2011 by the same authors) and by the conceptualization of Komiyama and Takeuchi [29,30]. Moreover, the previous research on sustainability has been mostly limited to environmental and economic concerns. Presently, however, the sustainability science is a dynamically developing field, offering new perspectives [31] and capable of addressing both environmental and economic dimensions, and also focusing on the social issues, recognized as a fundamental component of the sustainable development.

2.1. Social Sustainability

The social dimension is also known in the literature as the social sustainability. According to Chiu [32], three main approaches to the interpretation of social sustainability exist. The first interpretation equates social sustainability to environmental sustainability, whereas the second refers to the social preconditions required to achieve environmental sustainability. The aim of the third one concerns improving the wellbeing of people and the fair distribution of resources, while mitigating social exclusions and negative conflict [33]. Identifying the main key themes of social sustainability encompasses especially equity [34], poverty reduction and livelihood [35], human rights, employment [36], social homogeneity [37], social security and basic needs. These key themes have been renewed or replaced by more vague and less measurable concepts such as identity [23], sense of place and the benefits of social networks [38]. This can be caused by the significant breakthrough and innovation caused by information technology that forces the social progress and the balance, ensuring social equality, freedom and a healthy standard of living [33]. The consideration of the social aspects creates the consciousness of the society by defining guidelines, issues and policies essential to respect [39]. Subsequently, the synergy of the available actions and intentions may provide the comprehensive assumptions to social sustainability assessment.

The sustainability assessment approaches can be used for various dimensions of sustainability, but the analysis is performed concerning social dimension with elements of sustainable society, however. To a certain extent, sustainable society is considered to be a society that is economically viable, environmentally sound and socially responsible [40,41]. The sustainable society aims to reinforce policies and guidelines to protect the environment, build scaled growth economies with regard to achieving a high quality of life [42,43]. Thus, it ensures the balance between social wellbeing, economic growth, and environmental quality.

2.2. Assessment of Sustainability and SIS

Sustainability assessment (SA) is a complex appraisal problem [30]. It is a form of assessment that aims to inform and improve strategic decision-making [44]. The goal of SA is to perform the activities and assumptions to make an optimal contribution to sustainable development [30,45]. Over the last few decades, a multitude of approaches and methods for the assessment of sustainability have been devised by an increasing body of literature. The approaches typically address the three generally accepted dimensions of sustainable development [46]. They differ in the context of the offered features and the conceptualization of the object of evaluation. The SA process can be performed using various types of measures and metrics, indicators, methods, tools, and frameworks.

The aim of the measures and metrics is to provide a comprehensive and complete evaluation of the selected aspects and activities. This group allows for quantitative assessment [31], and, contrary to indicators (indexes), is offering a narrative description in addition to the qualitative characterization [47]. Indicators are simple measures, most often quantitative that represent a state of sustainability dimension development. There is many examples confirming that the term of metrics and indicators is regularly used interchangeably. When indicators are aggregated in some manner, the resulting measure is an index. Naturally, indicators may contain one or more metrics [48]. What is more, the frameworks may contain a series of indicators. This set provides the guidelines for the achievement of sustainable development. Additionally, frameworks extend the functionalities by offering both a comprehensive approach [39,48] and methodological base for sustainability development [40].

Particularly, since the 1990s, many substantial and often promising SA and sustainability indicator efforts have been made. However, in terms of indicators, the analysis suggests that the development of new sustainability indicators is increasingly focused on measuring emerging themes rather than on improving the assessment of more traditional concepts. Nonetheless, that kind of approaches occurs as well.

Moreover, within the last two decades, the ICT became commonplace. This, in turn, lead to its increased role in the social, economic, cultural, or personal development and growth. This increased significance of information and ICT, established the concept of SIS, which immediately became a research topic in the USA [49,50] and, subsequently, in Europe [51,52].

The foundation of SIS measurement should conform to good practice for information society measurement. Generally speaking, there are two approaches to the quantitative description of the information society. The first one comprises the list of indicators characterizing information society. The second relates to the so-called composite indexes which are aggregate measures and are based on the chosen set of indicators.

The review of the SA approaches reports that the most well-known ones, for example Sustainability Society Index (SSI [42]), Global Reporting Initiative (GRI [53]), United Nations Commission on Sustainable Development (UNCSD) Framework [54], Life Cycle Assessment (LCA), and development of standards [55], were the key driver for adoption of sustainability [31]. The United Nations Environment Programme (UNEP) launched the Global Reporting Initiative in 1997 for improving the quality, structure and coverage of sustainability reporting, whereas a compatible to GRI framework for sustainability indicators was provided by Azapagic [56] and dedicated to the mining and minerals industry. Former work, developed by Holmberg and Karlsson in 1992 [57], referred to the concept of socio-ecological indicators (SEIs) in order to establish the linkage between society and the environment. Then, the set of sustainability indicators for companies covering all main aspects of sustainable development was collected and improved by Krajnc and Glavič [58]. Warchust [48] suggested a two-step approach for measuring the sustainability, offering development in different areas measured through Sustainable Development Indicators (SDIs) individually and assessing the overall improvement achieved towards sustainable development by aggregating these individual areas concerning their respective dimensions [59].

Furthermore, the SDIs are used to monitor the EU Sustainable Development Strategy (EU SDS, Eurostat [60]), whereas the Sustainable Society Index integrates human wellbeing and environmental wellbeing. SSI contains the data from different countries to manage the meaningful information in the dynamic environment. Another example is the set of 58 national indicators used by the UNCSD. These indicators extend further than just the common economic indicators, to include, social, environmental and institutional monitoring mechanisms [54].

SSI aims to describe societal progress along all three dimensions: human, environmental, and economic. The SSI comprises eight policy categories and three wellbeing dimensions (Human, Environmental, Economic) and is calculated for 151 countries accounting for 99% of the world population. Apart from SSI, there is several indices of societal progress, encompassing more than 30 aggregated measures that attempt to describe either sustainability or more general societal progress in a quantitative way. These indices are used to compare, rank, and evaluate the selected countries or regions. Mostly, they refer to measuring healthy life expectancy [61], environment protection [62], migration policies [63], life expectancy [64], personal consumption, population [65] and the level of welfare.

It is evident that there is a vast number of indices that endeavor to quantify sustainability in its broad sense or to highlight particular issues. These elaborations vary in the number of indicators and updating, however. The predominant aspect is a comparative analysis of selected countries and existing scoring rank (benchmarking) of countries. Although the indices are mostly focused on assessment of the current level of sustainability of countries worldwide, the national, regional and company levels/aspects are also considered. As an example, European Cities Monitor [66] is based on the survey of 500 companies from nine European countries. Then, European Green City Index [67] enables comparing the environmental performance of 30 major cities in 30 European countries with regard to water, land, energy and waste use resources.

Similarly, the index Best Performing Cities [65] measures growth in jobs, wages and salaries, and technology output. It contains 200 US Metropolitan areas. Assessing the perspectives of development of the medium-sized cities is possible using European Smart Cities [68] indicator which contains 70 European cities. Furthermore, Australian RobecoSAM Sustainability Index [69] contains the largest sustainability-driven companies from each of 21 industry clusters covering the entire Australian economy, providing the measures in three sustainability dimensions.

Analyzing the methods of measuring information society development, the ones which do not embrace the sustainable imperative shall not be considered. Moreover, one can observe an imbalance between researching the ICT adoption and its impact on sustainability. Also, the universality of these methods is assumed, regardless of the economic, political, social, or cultural situation in various countries. Some of the indicators proposed in the measurement approach do not diagnose the most important factors affecting the ICT adoption within households, and thus they do not identify the development of a SIS in a given place and time. Given these limitations, an original proposal for measuring the information society should be developed. Such measurement should have such essential functions as [14]:

• Defining the development priorities and objectives of the SIS [70];
• Assessment of progress in achieving the defined objectives of the SIS [71];
• Monitoring of the processes in the SIS [24];
• Support for investment decisions regarding the development of the SIS [72];
• Measurement of selected parameters that characterize the SIS [73];
• Giving the opportunity for benchmarking [74].

3. MCDA Application to Sustainability Measurement

The MCDA has been a prevalent approach in various decision-making and evaluation problems’ areas. A development of two main groups of methods and directions can be observed in the research, differing significantly in the technique of preference modelling and data aggregation (often taking the form of the best alternatives’ choice) [75]. The first group, so-called American MCDA school [76], is based on the utility theory and includes the methods such as Multi-Attribute Utility Theory (MAUT), AHP or TOPSIS. These methods provide a utility value of each of the considered alternatives, which can later be used to create a ranking. The second group, European MCDA school [77], is based on the outranking relations and includes methods such as ELimination Et Choin Traduisant la REalite (ELECTRE) or Preference Ranking Organization Method for Enrichment Evaluations (PROMETHEE). These methods provide rankings of the considered alternatives, but no utility value is calculated. Furthermore, fuzzy developments of some of the MCDA methods are available, where linguistic assessments and more advanced knowledge of the input data are used instead of the mean numerical values.

The MCDA approach has been successfully used in various topics related to sustainability, which include the challenges of air pollution [78,79] and water pollution and sanitation [25], land use and misuse [80,81], drought and desertification [82], industrial development and energy crisis [83,84], hazardous wastes management [79], population growth and urbanization [85], as well as in various approaches to sustainable development, such as changing the consumption and production patterns [86,87,88], educating and increasing the awareness of sustainability [89], sustainable use of natural resources [90,91,92].

Table 1. Literature review of MCDA methods in sustainability problems.

Topic	MCDA Method	Other Met.	Alt.	Crit.	Ranking	Index	Sens. Anal.	Ref.
consumption and production patterns	SMART	Y			N	Y	N	[86]
	AHP	Y		10/8	Y	N	N	[87]
	DESIRES–AHP, MAVP	Y	4	36	Y	Y	Y	[88]
sustainable development policies	NAIADE		4	9	Y	Y	N	[95]
	NAIADE, AHP		5	17	Y		N	[96]
	ELECTRE III		3	3	Y	N	Y	[93]
education and awareness of sustainability	MAUT		4–6/scenario		Y			[89]
sustainable use of resources	SCORE		4	15	Y	Y	Y	[90]
	MCA, AHP		19	13				[91]
air pollution	PROMETHEE II	Y	10	6	Y	N	N	[78]
	Reference Point Technique		5	5	Y	N	N	[79]
water pollution	ELECTRE III, ELECTRE IV, CP		14	8	Y	N	N	[25]
land use and misuse	MCDA		2	6	Y	N	N	[80]
	PROMETHEE II		3	6	Y	N	N	[81]
desertification and drought	ELECTRE I, PROMETHEE, AHP		5	8	Y	N	N	[82]
industrial development and energy crisis	AHP, CP		4	13	N	Y	N	[84]
population growth, urbanization	AHP	Y		5	N	Y	Y	[85]

presents the literature review results of MCDA application in various sustainability problems.

Two groups of research results can be observed in the literature review. The first group is focused on evaluating a group of alternatives and generating a ranking based mainly on the outranking relations between the alternatives. In this group, one can find mostly the methods from the European MCDA school, such as ELECTRE [25,93], PROMETHEE [78] or reference point technique [79]. The second group, based mostly on the American MCDA school, contains research where composite indicators (CIs) are computed, with absolute values which can later be used for rankings generation [88,90] or be put on a map to perform a spatial analysis [85,94]. The MCDA application for composite index creation is further described in Section 4.

De Luca et al. developed a comprehensive literature review of MCDA methods and their combination with life cycle tools in agricultural sustainability [97]. Cinelli et al., performed a comparison of five selected MCDA methods to conduct SA. MAUT, AHP, PROMETHEE, ELECTRE and DRSA methods are compared in respect to ten crucial criteria that SA tools should satisfy, such as ease of use, software support, uncertainty management, thresholds and a life cycle perspective [98].

Myllvita et al., assessed the environmental impacts of alternative biomasses [86]. Yue et al., used AHP to evaluate the carbon footprints and environmental impacts of China’s paper producing and printing services industries [87]. Azapagic et al., presented the DESIRES method, evolving from AHP and MAVP, and used it to assess the sustainability of different future technologies and scenarios for electricity generation in the United Kingdom [88].

Munda used the NAIADE method to conduct a social multi-criteria evaluation of urban sustainability policies in four cities, two from developed countries and two from developing countries [95]. Similarly, Omann used the NAIADE and AHP methods to develop core environmental strategies in five policy fields in Germany [96]. Khalili and Duecker applied ELECTRE III to select the most relevant and sustainable waste management solution for a waste stream [93].

The MAUT was successfully incorporated into the transdisciplinary case study approach (TCS) by Scholz et al., in [89]. Experts and stakeholders were individually asked to assess 4–6 scenarios for each industry, first intuitively and then based on a series of sustainability criteria. As a result, two rankings of desired and undesired scenarios were generated.

Rosén et al., used SCORE [90], and Koschke et al., used AHP in the subject of sustainable use of resources, whereas Mavrotas et al., and Duijm et al., applied PROMETHEE II and Reference Point Technique respectively for the air pollution problem. Ganoulis evaluated alternative strategies for wastewater recycling and reuse in the Mediterranean area with ELECTRE III, ELECTRE IV and Compromise Programming (CP). A set of 14 strategies was considered, based on five sites and four disposal site actions, and the research produced a ranking of the best strategies.

Fontana et al., compared three land-use alternatives resulting from land-use change caused by socioeconomic pressures in the central Alps [81]. The PROMETHEE II method was used to generate a ranking of alternatives, based on six criteria. Grau et al., used PROMETHEE, ELECTRE I and AHP methods to select among different alternatives to prepare an integral plan to solve the soil erosion and desertification problem in Argentina [82].

Over the last decade, some pilot attempts to MCDA use in digital sustainability problems can also be observed. Mondlane et al., demonstrated the possible contribution of ICT to the decision-making process that can contribute to minimizing the waste of resources and prevent the loss of both social and economic assets by developing scenario strategic planning based on AHP method and backcasting [99]. Giesen presented a methodological framework for ICT-Driven sustainability planning and management support with the focus on small and medium enterprises (SME) [100]. Halog and Manik describe the development of an integrated computational methodology connecting LCA, LCC, SLC, stakeholders’ analysis supported by MCDA, dynamic system modelling and ICT technologies, and demonstrate its application on development of biofuel supply chain networks [101]. Albadvi used PROMETHEE method to define a national strategy model for information technology development in developing countries. The research was structured around a three-dimensional configuration of the strategy development process: key technologies, socioeconomic sectors and applications [26]. Thomaidis et al., demonstrated the application of fuzzy multi-criteria decision-making approach to establishing a sustainable computer infrastructure for the local-area hospital of Chios Island [102].

4. MCDA-Based Sustainability Index

The literature review shows that the usage of MCDA is not limited to rankings creation. Some MCDA methods, especially those from the so-called American school, provide utility value functions, which can be used to measure the absolute performance of various alternatives, as opposed to only showing the outranking relations.

Table 2. A literature review of MCDA methods usage in sustainability indices creation.

Method	Use Case	Alt.	Crit.	Index	Ranking	Map	Sens. Anal.	Ref.
AHP	steel company	1	34	Y	N	N	N	[59]
AHP	countries	28	9	Y	Y	N	Y	[103]
AHP + GIS	agricultural development	map	10	Y	N	Y	N	[94]
AHP + GIS	urbanization	map	5	Y	N	Y	Y	[85]
LIN, GME, NCMC	brewery	5	99	Y	N	N	Y	[104]
MCDA-DEA	energy	18	3	Y	Y	N	Y	[105]
MCDA-DEA	energy	18	3	Y	Y	N	N	[106]
SMAA	water	8	3	Y	N	N	Y	[107]
SWING + GIS	country areas	92	18	Y	Y	Y	N	[108]

contains the literature review of such application of the MCDA methods.

Most of the reviewed work was based on the AHP method. Singh et al., developed a composite sustainability performance index (CSPI) for a major steel company in India [59]. Cucchiella et al., calculated the sustainability index for 28 European countries to obtain their sustainability ranking. A comprehensive sensitivity analysis was performed [103]. Kumar et al. [94] and Al-Shalabi et al. [85] used AHP along with GIS to put sustainability index values on maps. Similarly, Boggia and Cortina used SWING and GIS to classify and map different Italian Region areas based on environmental and socioeconomic performance indices [108]. Zhou et al., constructed a composite sustainability index for a brewery for a 5 years’ time span. Linear Aggregation (LIN), Geometric Aggregation (GME) and non-compensatory multi-criteria approach (NCMC) methods were used to aggregate data from [104]. Hatefi et al., used a combination of MCDA and DEA methods to create sustainable energy index and then evaluated 18 country economies [105,106]. Linhoss and Ballweber developed a water-sustainability index based on the stochastic MCDA (SMAA) method. Sironen aggregated 21 indicators into a Sustainable Society Index and ranked 151 countries over the world [71].

The conducted literature review clearly presents MCDA as a useful tool for SIS evaluation, which justifies further research aiming at using these methods to construct a MCDA-based framework for the construction of various SIS indexes, which constitutes the authors’ methodological contribution of this paper.

5. MCDA-Based Adjustable Index of SIS

5.1. Methodological Foundations

As mentioned above, MCDA methods are a powerful tool for obtaining quantitative results of data aggregation and final evaluation, as well as they provide the possibility to express the preferences of the decision maker (for example in the form of weight or criteria evaluation). It is worth noting that in the context of the fundamental assumptions of the MCDA methodology, when a multi-criteria problem is formulated, a decision-making situation is considered, where a finite set of decision variants (actions) A is evaluated according to n criteria g₁, g₂, …, g_n, forming a family $G=\left\{1,2,\dots ,n\right\}$. It can be assumed, without loss of generality, that the higher the value of criterion function g_i(a), the better the variant a ∊ A is regarding criterion g_i, for each i ∊ G. In such a situation, the decision maker (or system analyst) wants to construct their ranking from the best to the worst, according to the preferences [77,109,110]. Based on that, the authors provide a methodological guideline in the form of a framework to construct an adjustable SIS index (AdjSI).

As noted, the proposed AdjSI index was developed using the classical methodology foundations provided by Guitouni [111,112] and Roy [77,109]. The process of AdjSI construction was divided into four successive stages (see Figure 1). In the first step of the process, the purpose of building AdjSI and the defining an object of the evaluation process is determined. In the assumptions of the MCDA methodology, it takes the form of process structuring. In this case, in addition to identifying the objective and stakeholders of the process, it is necessary to identify and build a cohesive family of assessment criteria (most commonly in the form of a hierarchy). Obtaining high reliability of research is usually carried out by identifying the base set of criteria, based on the relevant literature of the subject, which is confirmed by numerous publications [77,92,109,110,112].

After the problem structuring phase, having an identified family of assessment criteria, one can proceed to the next stage of the process. In this case, it includes data collection. The data can be collected in different ways, for example, the Delphi method, expert panels, semi-structured interviews, focus groups, workshops or survey questionnaires [113,114,115,116].

In the third stage, to build a general synthetic AdjSI, MCDA aggregation techniques are selected. For this purpose, a subset of MCDA methods providing the aggregation of data in numerical form, e.g., utility function, was selected. In case of the European school’s methods, the PROMETHEE II method can be adapted to allow this (see Appendix D), whereas in case of the American school’s methods, besides the well-known techniques such as simple additive weighting (SAW) or multiplicative utility functions, the AHP and TOPSIS methods can be used. The choice of these methods was, apart from the possibility to obtain the quantitative result of data aggregation, additionally based on:

• Possibility to express the complex structure of evaluation criteria (AHP method) [117];
• Possibility to express the values and reference points of the constructed index (TOPSIS method) [118];
• Possibility to model the decision maker’s (DM) preferences (along with imprecision) and vast analytic possibilities (PROMETHEE method) [119].

The latter two were eventually used in the process of construction and determination of the AdjSI index (see Appendix A and Appendix C). It was also decided that the aggregation process would be based on the set of criteria defined in step 1 (see the * link in Figure 1). The final value of the AdjSI was determined with the use of the Formula (1):

$$U\left(a\right)=g_1\left(a\right)·w_1+g_2\left(a\right)·w_2+\dots +g_n\left(a\right)·w_n$$

(1)

where U(a) is the index value, $g_i\left(a\right)$ represents the a-th variant’s evaluation regarding the i-th criterion and $w_i$ represents the weight assigned to the i-th criterion. Because of such a form, the index can be easily adjusted to the DM’s preferences or strategy.

It was decided that the fourth step of the procedure should be divided into two stages, to expand the possibilities of AdjSI’s practical use. In the presented form, AdjSI is an effective tool for assessing the degree of sustainability for each assessed object, and, moreover, it provides a possibility to aggregate survey data into groups, for example by the respondents’ education level or province or city of living. This allows division of the test sample into unequivocal profiles where, subsequently, AdjSI can be determined for each profile object.

The “Adj” term in the AdjSI name requires some additional explanation. The use of MCDA methodology, in contrast to the classical statistical research, allows not only determination of the aggregate value for each evaluated object from the given set of collected data, but also the possibility of carrying out deliberate manipulations in the individual criteria and their groups. In practical terms, this does not require interference with the collected data set but allows for a relatively smooth transfer of the DM/Analyst’s preferences and priorities, for example in the form of relative or absolute weights of criteria. Such preferences can be illustrated, in practical terms, for example by (1) strategies oriented to increase the share of individual criteria or their groups in the final value of AdjSI; or (2) manipulating criteria values to obtain the desired maximum or minimum value for a given profile object. Therefore, the AdjSI addresses the research question specified in Section 2.2 by, inter alia, providing the means to define the development priorities and objectives of the SIS, to monitor the processes in the SIS, and, even more importantly, to provide support for investment decisions regarding the development of the SIS. The practical studies presenting the process of determining the AdjSI values for various possibilities and scenarios are presented in Section 5.2.

5.2. Empirical Research Foundations

The framework proposed in this paper is based on a quantitative approach. Therefore, a positivistic (quantitative) research paradigm was adopted in this study [120]. Thus, it was assumed that a theory of SIS is a set of interrelated variables, definitions, and a proposition that presents a systemic view of SIS phenomena by specifying relationships among variables. Those variables were indicated based on the literature. Then they were examined by using statistical analyses of collected data and their interpretation was presented in an objective way. According to the argumentation of Crowther and Lancaster [121], this study adopted deductive approach to guide the selection of variables and constructs measured.

In the authors’ empirical research SIS for households was constructed and evaluated. By using the first stage of the framework, identification of the criteria for evaluating SIS in households was performed. As the SIS is a multidimensional concept encompassing various kinds of sustainability, all of which could be strongly influenced by adopting ICT by society stakeholders, among other households. Based on a stream of research on sustainability information society [11,12,122], Ziemba proposed expanded sustainability which includes four sustainability components, i.e., ecological, economic, socio-cultural, and political [123]. Regarding households, the sustainability components are:

• Ecological sustainability (Ecl) is the ability of households to maintain rates of renewable resource harvest, pollution creation, and non-renewable resource depletion by means of conservation and proper use of air, water, and land resources [39,57,124,125];
• Economic sustainability (Eco) of households means that households can gain some economic benefits, i.e., revenue growth and cost reduction, facilitated everyday life, increased satisfaction in relations with enterprises and public administration [18,88,126,127,128];
• Socio-cultural sustainability (Soc) is based on the socio-cultural aspects that need to be sustained. It means improving knowledge and skills, developing and improving teaching and working environment, increasing household’s security, and reducing social exclusion [36,57,129,130]; and
• Political sustainability (Pol) must rest on the fundamental values of democracy and active appropriation of all rights. It is related to the engagement of households in creating a democratic society [108,123,131,132].

Thus, 15 primary criteria for measuring SIS were distinguished and are presented in

Table 3. The dimensions of sustainability construct in households with the criteria describing them.

Primary Criterion	Summary	Description of Primary Criterion
Ecological sustainability (Ecl)
Ecl1	Sustainability in ICT	Reducing energy consumption and increasing the protection of the environment through ICT consuming less energy as well as built with fewer materials (miniaturization), and more easily recyclable and disposable.
Ecl2	Sustainability by ICT	Reducing the consumption of energy and other resources (e.g., paper, gasoline) and increasing the environmental protection by “de-materializing” production and consumption, e.g., the use of e-documents instead of paper documents, shopping in e-shops instead in traditional stores, participating in e-training instead of traditional training, using e-products instead of traditional products.
Economic sustainability (Eco)
Eco3	Cost reduction	Reducing costs, e.g., through lower purchase prices of goods/services on the Internet, eliminating travel expenses, lower costs of communication over the Internet than telephone or personal communication.
Eco4	Income growth	Income growth, e.g., obtaining higher interest rates on electronic deposits, gaining an additional source of income from employment on the Internet (e.g., paid surveys, content creation).
Eco5	Facilitating everyday personal life	Facilitating everyday personal life, e.g., learning, shopping, extracting information, running administrative errands, communicating, socializing, using entertainment.
Eco6	Facilitating everyday working life	Facilitating everyday professional life, e.g., performing work at home without the daily commute to work, participating in virtual business meetings, more efficient and faster execution of professional tasks and instructions.
Eco7	Increased satisfaction—business	Increased satisfaction with the services and goods provided by businesses, including e-services and e-products.
Eco8	Increased satisfaction—public services	Increased satisfaction with public services provided by public administration, including public e-services.
Socio-cultural sustainability (Soc)
Soc9	Competence extension	Expanding existing knowledge and skills along with gaining new ones (including digital knowledge and skills), as well as better aligning the way of thinking and acting in response to changing reality, labor market requirements and the needs of employers and customers.
Soc10	Increasing activity	Increasing activity in different socio-cultural groups, whose members share a common passion, exchange knowledge and ideas, disseminate the culture, support each other, and build up their reputation.
Soc11	Working and teaching environment improvement	Development and improvement of own learning and working environment, e.g., lifelong learning with e-learning and real-time learning, working with different people from own work/school environment, working with professionals from different cultural backgrounds, better communication, creating employee/student ties with the enterprise/school.
Soc12	Society security	Increasing the security of people and social groups together with participating in its improvement through access to information and dissemination of information on various dangers and disasters (e.g., weather, food, illness) as well as undertaking and organizing humanitarian and social justice actions.
Soc13	Reducing social exclusion	Reducing social exclusion due to age, education, place of residence or disability, which causes lay in limited and difficult participation in social or collective life and limited or difficult access to resources, public goods, and social institutions, as well as to businesses and their products services and jobs.
Political sustainability (Pol)
Pol14	E-democracy	Increased participation in public consultation and democratic public decision-making along with the development of cooperation, communication, partnership and networking with public administration, social welfare institutions, social enterprises.
Pol15	E-public services	Increasing and facilitating access to public administration services as well as to the rules of law and their interpretation, e.g., regarding work, social protection, health, family life, participation in cultural life and education, as well as personal and political rights.

.

Based on the literature studies above, a cohesive family of assessment criteria was built in the form of a hierarchy, which is presented on Figure 2.

In the authors’ approach, in step 2, the data is collected. Survey questionnaires built based on the Likert scale were selected for the data collection. A Likert-type instrument (questionnaire) was developed that studied SIS in households. The task of the respondents was to assess the primary and detailed criteria describing the four components of the sustainability construct, i.e., ecological (Ecl), economic (Eco), socio-cultural (Soc), and political sustainability (Pol), in accordance with Figure 2 and Table 3. The respondents answered the question: Using a scale of 1 to 5, evaluate the following benefits for your household resulting from the efficient and effective ICT adoption? The scale’s descriptions were: 5—strongly large, 4—rather large, 3—neither large nor disagree, 2—rather small, 1—strongly small. The data collection process was preceded by a pilot study, which was conducted in March and April 2016 to verify the survey questionnaire. In total, 15 experts participated in the pilot study, i.e., five researchers from an information society, five people acting for digital inclusion in Poland and five persons from five households. Finishing touches were put into the questionnaire, especially of a formal and technical nature. No substantive amendments were required.

The subjects in the study were enterprises from the Silesian Province in Poland. The choice of this region was driven by the fact of its continuous and creative transformations related to restructuring and reducing the role of heavy industry in the development of research and science, supporting innovation, using know-how and transferring new technologies, as well as increasing importance of services. In response to the changing socioeconomic and technological environment intensive work on the development of the information society has been undertaken in the region for several years. In the next development strategies of the information society it was and is assumed that the potential of the region, especially in the design, provision and use of advanced ICT will be increased [133]. All this means that the results of this research can be reflected in innovative efforts to build a SIS in the region and, at the same time, constitute a modus operandi for other regions throughout the country and other countries.

Selecting a sample is a fundamental element of a positivistic study [132]. The stratified sampling and snowball sampling were therefore used to obtain the sample that can be taken to be true for the whole population. The following strata were identified based on household’s size (defined in terms of the number of members), age and educational background of household’s members, place of residence, and the main source of household’s income.

The subjects were advised that their participation in completing the survey was voluntary. At the same time, they were assured anonymity and guaranteed that their responses would be kept confidential.

Having applied the Computer Assisted Web Interview and employed the SurveyMonkey platform, the survey questionnaire was uploaded to the website. The data were collected during a two-month period of intense work, between 24 April 2016 and 24 June 2016. After screening the responses and excluding outliers, there was a final sample of 679 usable, correct, and complete responses. The sample ensured that the error margin for the 98% confidence interval was 4%. Data was stored in Microsoft Excel format, using the Statistica package and Microsoft Excel, and analyzed in three stages.

Steps 3 and 4 of the proposed framework were performed sequentially—the AHP, TOPSIS and PROMETHEE methods were used to compute the value of the AdjSI index based on the initial DM preferences. For the purposes of the empirical study, it was decided that each cluster of criteria will be assigned an equal weight. Similarly, the weights of criteria in each cluster were also equal. The detailed calculations, results and their discussion are presented in the following sections.

Initially, the index value was computed for all particular households. However, such approach turned out to provide little practical value to the DM. More valuable insights could be provided if the survey results were grouped based on the collected data, as pointed out in Section 5.1. Therefore, in the following steps, the data collected from the survey was further aggregated prior to the index computation, based on the data available. At first, the data was aggregated based on the household members’ education level, thus providing the index for four variants: elementary, vocational, secondary, and higher education. Afterwards, the average age of household members was used for aggregation, thus forming four variants: under 21, 21–35, 36–50, over 51. The third aggregation method used was by the town/city size: rural, towns with less than 20 thousand inhabitants, cities with 20–100 thousand inhabitants and cities over 100 thousand inhabitants. In this case, the index computation was followed by a research how a modification of the DM’s preferences would influence the index—the sensitivity analysis. Additionally, for comparison purposes, in the last scenario apart from AHP, also the TOPSIS and PROMETHEE II methods were used to compute the SIS index.

6. Empirical Study of the Polish Households’ SIS

6.1. Index Based on Unaggregated Data

The empirical study, as explained in the prior sections, was performed with the use of three MCDA methods: AHP, TOPSIS and PROMETHEE. However, to keep the paper concise and the narrative consistent, where the results were similar, only the output of a single method is discussed.

The research was commenced by computing the index values for all the surveyed households. Although the usage of the framework allowed the obtaining of a complete, quantifiable ranking of all individual households, for the studied class of objects—households, as opposed to companies or local organizations—the results obtained in such a manner have a limited practical value. Therefore, the authors decided that the full set of results (source data and AdjSI values for all households) will be presented only in a form of a raw XLS data supplement (see details in Appendix E), whereas in the manuscript the discussion would be limited to only 10 objects, which would clearly exhibit the methodological properties of the proposed framework, yet would allow the avoidance of any possible practical implications of the full set of data.

For the AHP method, all clusters in the SIS analysis, i.e., Ecl, Eco, Soc and Pol, were initially assigned an equal weight. The criteria Ecl1—Pol15 were then compared pairwise, and the results were scaled to the range $1\dots 9$ or $\frac{1}{1}\dots \frac{1}{9}$. The results of the AHP analysis are presented in

Table 4. Results of SIS indices calculation for arbitrarily selected ten households, obtained with: (a) the AHP method; (b) the TOPSIS method; (c) the PROMETHEE method.

Household	A1	A2	A3	A4	A5	A6	A7	A8	A9	A10
(a) AHP
Weight	9.10%	6.00%	11.10%	9.90%	10.50%	12.80%	11.30%	11.60%	6.80%	11.00%
Rank	8	10	4	7	6	1	3	2	9	5
(b) TOPSIS
Cci	0.5172	0.3463	0.7144	0.6108	0.5768	0.8212	0.6031	0.7479	0.4027	0.6222
Rank	8	10	3	5	7	1	6	2	9	4
(c) PROMETHEE
Phi	−0.0237	−0.1308	0.0422	0.0274	0.0094	0.0826	0.03	0.0488	−0.0973	0.0115
Phi+	0.0458	0.015	0.0601	0.0688	0.0523	0.0866	0.0752	0.0684	0.0221	0.0588
Phi−	0.0695	0.1458	0.0179	0.0414	0.0429	0.004	0.0452	0.0196	0.1195	0.0472
Rank	8	10	3	5	7	1	4	2	9	6

a. Its analysis results in an observation that the households A6, A8 and A7 are the leading ones in the arbitrarily selected group. Moreover, the leading variant A6 obtained over twice as much score than the worst variants A9 and A2. The TOPSIS and PROMETHEE methods also point out the households A6 and A8 as the leading ones; however, the order in the remaining part of the ranking is slightly different (see Table 4b,c).

Nevertheless, it is important to note that such comparisons of individual variants would be beneficial if public buildings or industries were compared, while for the individual households, such information provides little practical usage. Therefore, the future research of the authors focused on the analysis of aggregate data.

6.2. Index Based on Aggregate Data

Section 6.1 demonstrates how AdjSI index can contribute to the SIS evaluation. However, the practical usefulness of AdjSI based on unaggregated data representing individual households is limited. Much more insight can be provided to the DM by applying AdjSI to generate an index based on data previously aggregated. Consequently, after the individual households’ evaluation stage, the data was aggregated (grouped) based on various characteristics of the surveyed households. Initially, the level of education of the most educated household member was considered: higher (A1), secondary (A2), vocational (A3) and elementary (A4) education. The aggregation by the A4 variant resulted in no households. Therefore this alternative was excluded from the AdjSI calculation.

Similarly to the study of the individual households, in the study based on aggregate data, all clusters in the AHP method, i.e., Ecl, Eco, Soc and Pol, were assigned equal weights. The criteria Ecl1-Pol15 were then compared pairwise. The comparison was performed by computing the ratio of the two compared values and then scaling the result to the range $1\dots 9$ or $\frac{1}{1}\dots \frac{1}{9}$. The results of the analysis, along with the results produced by the TOPSIS and PROMETHEE methods, are presented in

Table 5. AdjSI index values for SIS evaluation in Polish households aggregated by the household members’ education (a); age (b) and town size (c).

	AHP		TOPSIS		PROMETHEE
(a) Education
Alternative	Score	Rank	Score	Rank	Score	Rank
A1	34.50%	1	0.7974	1	0.0053	1
A2	33.20%	2	0.6883	2	0.0025	2
A3	32.30%	3	0.2443	3	−0.0078	3
(b) Age
A1	25.50%	2	0.7354	1	0.0016	2
A2	27.00%	1	0.5520	2	0.0045	1
A3	24.70%	3	0.3160	3	−0.0001	3
A4	22.70%	4	0.2323	4	−0.006	4
(c) Town Size
A1	23.90%	4	0.1711	4	−0.0016	4
A2	24.80%	3	0.4351	3	−0.0001	3
A3	26.10%	1	0.7347	1	0.0013	1
A4	25.20%	2	0.5905	2	0.0004	2

.

The analysis of Table 5a shows that the highest AdjSI index values (with all three methods) were obtained by the households members with higher education, followed by the ones with secondary education. The lowest index values were obtained by the households’ group where members had vocational education only.

Subsequently, the data was re-aggregated, based on the average age of the household members. Thus, four variants were formed: under 21 (A1), 21–35 (A2), 36–50 (A3) and over 51 (A4) years old. The results of the analysis are presented in Table 5b. The AHP and PROMETHEE methods assigned the highest AdjSI values to the A2 group, the households with average members’ age in the range 21–35, followed by the A1 group (under 21), A3 (36–50) and A4 (over 51). In case of the TOPSIS method, the ranks of groups A1 and A2 were swapped.

Last, but not least, the data was aggregated based on the size of the town/city. Four alternatives were obtained: rural areas (A1), towns with less than 20 thousand inhabitants (A2), cities with 20–100 thousand inhabitants (A3) and cities over 100 thousand inhabitants (A4). It can be observed, that the AdjSI index based on all three methods returned the same ranking of variants: A3, A4, A2 and A1.

It is important to note, that one of the advantages of the AHP method is its ability to produce the utility function not only of the individual criteria, but also of the criteria’s clusters. This, in turn, in case of the partial effects of SIS, fulfills the research question pointed out in Section 2.2.

The aforementioned ability of the AHP method is illustrated on the data aggregated based on the town size. The results of the SIS index calculation are presented in

Table 6. AdjSI index values for the SIS evaluation for Polish households aggregated by town size.

	A1	A2	A3	A4
SIS	23.90%	24.80%	26.10%	25.20%
¦--Ecl	5.70%	5.90%	6.90%	6.50%
¦ ¦--Ecl1	2.90%	2.80%	3.40%	3.30%
¦ °--Ecl2	2.80%	3.10%	3.50%	3.20%
¦--Eco	5.90%	6.20%	6.60%	6.40%
¦ ¦--Eco3	1.00%	1.00%	1.10%	1.00%
¦ ¦--Eco4	1.00%	0.90%	1.20%	1.10%
¦ ¦--Eco5	1.00%	1.10%	1.00%	1.00%
¦ ¦--Eco6	1.00%	1.00%	1.00%	1.20%
¦ ¦--Eco7	0.90%	1.00%	1.20%	1.10%
¦ °--Eco8	1.00%	1.10%	1.10%	1.00%
¦--Soc	6.00%	6.50%	6.30%	6.20%
¦ ¦--Soc9	1.20%	1.30%	1.30%	1.20%
¦ ¦--Soc10	1.20%	1.40%	1.20%	1.20%
¦ ¦--Soc11	1.20%	1.30%	1.30%	1.30%
¦ ¦--Soc12	1.20%	1.30%	1.30%	1.20%
¦ °--Soc13	1.20%	1.30%	1.30%	1.20%
°--Pol	6.40%	6.20%	6.30%	6.20%
¦--Pol14	3.10%	3.20%	3.10%	3.10%
°--Pol15	3.20%	3.10%	3.10%	3.10%

. The analysis of Table 6 shows that the leading alternative A3 obtained the score of 26.1%, followed by 25.2% for alternative A4, 24.8% for alternative A2 and 23.9% for alternative A1. When scores obtained in individual clusters are considered, the alternative A3 was best in all clusters but Pol, where it was preceded by alternative A1, which was the worst alternative in the overall ranking. The alternative A2, on the other hand, was best regarding to the Soc cluster criteria, followed by the alternatives A3, A4 and A1. Table 6 contains the precise score achieved by each alternative in individual criteria and clusters of criteria.

6.3. Adjustment of the Index to Adhere to the SIS Priorities and Objectives

As outlined in Section 2, a proper SIS measurement method should allow definition of the development priorities and objectives of the SIS, as well as provide support for decision-making in the area of the SIS. One of the advantages of the Authors’ adjustable index is that it allows studying how the evaluation would change if the DM considered one of the criteria clusters to be more important than the others. Two scenarios are presented in this section, demonstrating what effect on the results would have a strategy in which the DM evaluated one of the clusters of criteria as moderately more important in the first one, and extremely more important than the other clusters in the second one. The previously studied case of SIS index evaluation for data aggregated by town size was taken as an example.

In the first step, a DM’s strategy which gave the Soc cluster moderately more importance (1 in Saaty’s scale [117]) than the rest of the clusters was tested. The results of the original equal weights strategy and the modified strategy are presented in

Table 7. AdjSI index values for the SIS evaluation for Polish households aggregated by town size for various strategies: (a) equal weights of all clusters; (b) Soc cluster moderately more important; (c) Soc cluster extremely more important.

	Variant	A1	A2	A3	A4
(a)	Equal weights
	Score	23.90%	24.80%	26.10%	25.20%
	Rank	4	3	1	2
(b)	Soc moderately more important
	Score	23.90%	25.20%	25.80%	25.00%
	Rank	4	2	1	3
(c)	Soc extremely more important
	Score	24.00%	25.70%	25.50%	24.80%
	Rank	4	1	2	3

a,b respectively. The analysis of the presented results shows that although the criteria from the Soc cluster were given moderately more importance than the criteria from the other groups, the SIS index would still be highest in the households in the cities with 20–100 thousand of inhabitants (A3), and lowest in the rural areas (A1). Nonetheless, we can observe that the DM’s strategy would cause a change in the rank of the remaining two groups of households—small towns with less than 20 thousand of inhabitants (A2) would outrank the cities with over 100 thousand inhabitants (A4). When the scores of each group of households are analyzed, it can be observed that the score of the alternative A1 remained unchanged, whereas the weight of the alternative A2 increased by 0.40% and the scores of the alternatives A3 and A4 decreased by 0.30% and 0.20% respectively.

In the second step, the DM’s strategy which gave the Soc cluster extremely more importance (9 in Saaty’s scale [117]) was tested. The results of this strategy are presented in Table 7c. The analysis of the obtained results suggests that this intense change of Soc cluster’s weight would cause the A2 group (small towns with less than 20 thousand inhabitants) to advance to the first position in the ranking, outrunning the remaining alternatives, A3, A4 and A1, in order from having the best to the worst score. Additionally, it can be observed that the change of the Soc cluster weight resulted in a slight increase of the A1 group’s score (0.10%), 0.50% increase of the now-leading A2 group’s score and a decrease of the A3 and A4 groups’ scores by 0.30% and 0.20% respectively.

The aforementioned two examples of strategies demonstrate how a change in the weight of a simple cluster of criteria allows the adjustment of the AdjSI index to the DM’s aimed strategy, to better monitor the progress in achieving the defined objectives of the SIS. In this case, it showed how the Soc cluster significantly supports the A2 group, slightly supports the A1 group, and how it conflicts with the A3 and A4 groups (see Figure 3). Such influence of the clusters of criteria on various alternatives can be further benchmarked by performing a full sensitivity analysis, which is presented in Section 6.4.

6.4. Benchmarking the SIS Objective Achievement Strategies with Sensitivity Analysis

The AdjSI’s sensitivity analysis component allows the researchers to analyze the effect of employing virtually any strategy on the index behavior. In this section, a sensitivity analysis is presented for the SIS index of the previously demonstrated aggregation scenario—households aggregated by the town’s size.

At first, to verify how changes in weights of individual criteria affect the ranking of alternatives, the sensitivity analysis of the TOPSIS ranking was performed. For each criterion, its weight was modified in the range from 0 to 1, while the rest of the criteria remained unchanged. The results of the sensitivity analysis are presented in Appendix F. It can be observed, that the obtained ranking is very stable. Regardless of the criterion’s weight being changed, no change in the order of the alternatives can be observed. This is probably caused by the fact that the differences in scores between the alternatives are too considerable for a change in a single criterion to affect the final order of the alternatives. However, if the weight of a complete cluster of criteria is modified, the outcome is notably different, as presented below.

In the second step, the changes of the alternatives’ scores brought with the changes of weights assigned to each cluster were studied in a sensitivity analysis of the AHP ranking. In each step of the analysis, the preferences matrix was modified in such a manner, that the analyzed cluster received consecutively the score of $\frac{1}{9},\frac{1}{8},\frac{1}{7},\frac{1}{6},\frac{1}{5},\frac{1}{4},\frac{1}{3},\frac{1}{2},1, 2, 3, 4, 5, 6, 7, 8, 9$, while the results of the comparisons of the other clusters remained unchanged at the value of 1. The pairwise comparisons of criteria remained unchanged, i.e., the ratio of the compared values was calculated and scaled to the range of $\frac{1}{9}\dots 9$. Thus, the problem was solved 17 times for each cluster. The resulting scores and ranks were then plotted (see Figure 4).

The analysis of Figure 4 shows that the alternatives A3 and A4 are supported by criteria from the clusters Ecl and Eco, whereas the alternatives A1 and A2 are supported by criteria from the alternatives Soc and Pol. The leading alternative A3 would be preceded by alternative A2 only if in the comparison between the Soc cluster and other clusters resulted in score more than $7$. When the weight of the Eco cluster is increased, the difference between the winning alternatives A3 and A4 and the losing alternatives A2 and A1 grows, nevertheless, no difference in ranks is observed. The alternative A2 can precede the alternative A4 if the weight of the Soc cluster grows above $2$ or if the weight of the Ecl cluster is decreased to below $\frac{1}{3}$ when compared to other clusters. A significant swap of ranks 2 and 4 can be observed between the alternatives A4 and A1, with the alternative A2 remaining unchanged on rank 3 if the weight of the Pol cluster grows above $8$ when compared to other clusters.

The example presented above demonstrates how the possible strategies and their impact on achieving the SIS objectives can be benchmarked with the use of the proposed approach. The example focused on aggregate data. Nevertheless, the same mechanism can be used to research the full set of raw data. Sample calculations and analysis for a limited set of 10 randomly selected households are presented in Appendix G.

6.5. GAIA Analysis

While some basic analytics of how each alternative is supported (or not) by a set of criteria were presented in Section 6.4, the PROMETHEE component of the proposed approach and its GAIA (Geometrical Analysis for Interactive Assistance) tool provide much more sophisticated means to perform the analysis of selected parameters that characterize the SIS. Figure 5 depicts the GAIA analysis of the data aggregated by town size, broken down into criteria or clusters.

The analysis of Figure 5a results in observation that the alternative A3 is supported by most of the criteria, except Pol15, which is opposite to it, and Soc12, Soc13, Pol14 which are rather not affecting the rank of A3. The alternatives A4 and A2 (rank 2 and 3 respectively) are supported by different sets of criteria, as they are in opposite quarters of the chart. Although A2 seems to be directly supported by a higher number of individual criteria, the alternative A4 is supported directly and indirectly by the criteria which have more effect on the final ranking, such as Ecl1, Ecl2, Eco4, Eco6 and Eco7. The worst alternative A1 is supported only by criterion Pol15 and, to some minor degree, by Eco4 and Eco6. While criterion Ecl1 has no noticeable effect on A1 rank, the rest of the criteria are in opposition to this alternative. When criteria are considered, we can notice that the criterion Ecl2 almost precisely covers the decision stick, which makes it the most determining criterion in this decision problem. Also, most criteria are similar to Ecl2 regarding preferences, except Pol15, which conflicts with it in terms of preferences. When the influence of clusters of criteria on the final ranking is analyzed (see Figure 5b), it can be observed that the leading alternatives A3 and A4 are supported the most by the Ecl cluster. Further analyzing the relations between the clusters, some similarity between Ecl and Eco can be observed in terms of preferences, and lack of relation between Ecl and Pol. The Soc cluster is in small conflict with Ecl and in considerable conflict with Pol regarding preferences. It can also be noticed, that the Pol cluster of criteria has the least influence on the final ranking.

7. Conclusions

The SIS measurement is a current and important topic. There are numerous challenges related to the measurement of the new forms of sustainability, SIS being one of them. While much of the research effort has been focused on information society, sustainable development, and ICT adoption, there is still lack of profound, quantitative method-based studies about the SIS measurement. This study is the first to investigate theoretically and empirically this issue, especially in the context of the main SIS stakeholder, i.e., households. The major contribution of the research was to:

• Construct an MCDA-based framework for handling different aspects of SIS measurements, such as, but not limited to defining the development priorities and objectives of the SIS, accommodating various strategies of the stakeholders, or giving the opportunity for benchmarking;
• Propose an MCDA-based SIS index of households; and last, but not least
• Use the proposed SIS index to evaluate the SIS of a sample set of Polish households, thus facilitating future benchmarking with other regions.

As demonstrated in the research, the presented theoretical framework provides methodological indications to build a wide range of arbitrary SIS indices, which can facilitate the explanation to the issue of how households can participate in the creation of sustainable development and sustainable information society. It is worth noting that the generalized literature-based household SIS evaluation model presented in Table 3 can be easily reused or transferred for evaluation of different entities. The presented empirical verification of the research was divided into multiple steps. In the first step, ten households randomly selected from the group of over six hundred were evaluated to demonstrate the most basic functionality of the presented framework. However, much more interesting results were obtained from the next step, i.e., the multi-criteria evaluation of households based on aggregate data. The results showed that the highest level of SIS can be observed in households which inhabitants are highly educated from towns sized 20–100 thousand inhabitants, followed by the cities with over 100 thousand inhabitants. Depending on the method used, either households with an average age of 21–35 or below 21 were indicated as having the highest SIS level.

Subsequently, the influence of various strategies on the final SIS assessment was demonstrated, followed by a comprehensive sensitivity analysis of the complete range of possible DM’s preferences regarding the clusters of evaluation criteria. It was presented that, depending on the DM’s preferences, the AdjSI index adjusts the evaluation scores accordingly, thus resulting in modified rankings of the alternatives.

Eventually, the GAIA analysis allowed study of the relations between the clusters of criteria, exhibiting a considerable conflict between the social and political criteria, yet, at the same time, marking small significance of the latter cluster on the overall ranking.

With regard to the presented results, it is reasonable to conclude that this study expands the existing research on the SIS provided by Schauer [11], Fuchs [9], and Hilty [8]. Additionally, the findings of this study can be used by scholars to improve and expand the research on the SIS. Researchers may use the proposed theoretical model and methodology to do similar analyses with different sample groups in other countries, and many comparisons between different countries can be drawn. Moreover, the theoretical model and methodology constitute a very comprehensive basis for measuring the sustainability, but researchers may develop, verify, and improve them. Furthermore, the findings can help the public administration to develop sound information society plans and receive funding from the European Union that set itself the target of implementing the 2030 Agenda for Sustainable Development [20].

The presented study opens several questions for future work. First, the sustainability criteria used in empirical research are new constructs that have yet to be further explored and exposed to repeated empirical validation. Second, the studied sample was exclusively comprised of Polish households, especially from the Silesian Province. Caution should be taken when generalizing the findings to other regions and countries, as well as some further validation of the presented approach on a larger sample comprising households from more varied areas should be performed. Additionally, SIS measurement in the context of other SIS stakeholders, e.g., enterprises and public administration could be explored, which would require building and verifying the SIS indices for such stakeholders based on the indices presented in this study. Moreover, universal indices including criteria for all SIS stakeholders could be provided and analyzed. Last, but not least, the MCDA methods used in the proposed approach are simple in terms of understanding and application, which provides an easy start for researchers not well acquainted with MCDA methods; however, their replacement with more advanced methods can be verified. There exist fuzzy variants of the AHP and TOPSIS methods, which can be used for the improvement of the aggregation techniques of the survey data. Moreover, all methods used in this paper are prone to a so-called rank-reversal phenomenon, which means that the introduction of a new evaluated objects (in the presented case household) can lead to a reversal of the ranks of other objects. Supplementing the method’s set with a rank-reversal free method, such as COMET (Characteristic Objects METhod) [134,135], could be considered.