Assessing Digital Performance of Public Services in the EU: E-Governance and Technology Integration

Abstract

This research aims to rank the regions of the European Union based on their digital performance in public services, offering a new perspective on the digitalization process and its effects on the European public system. Therefore, the study examines the evolution of intensity and the impact of digitalization through the e-government framework, specifically from the perspective of the 27 member states of the European Union. Thus, the research analyzes the interplay between digital intensity, institutional quality, and adoptingadopting e-government services to understand the challenges and disparities across different regions. To evaluate the digital performance of the European Union regions, the analysis employs the following methods: (i) Principal Component Analysis to construct a composite Institutional Quality Index based on governance indicators, (ii) Gaussian graphical models to assess the relationships between digital intensity, institutional quality, and e-government highlighting the interconnections between variables, (iii) Data Envelopment Analysis to measure the relative efficiency of public service delivery, and (iv) cluster dendrograms to identify the clusters of countries with similar performance levels and challenges. The findings reveal disparities among nations and the contemporary challenges facing EU member states in implementing and adopting e-government initiatives. While Denmark and Finland often excel in digital service efficiency, countries like Bulgaria, Greece, and Romania may lag due to citizens’ limited digital infrastructure and digital literacy. However, it is to note that the quality of regulations, as reflected in the Institutional Quality Index, plays a significant role in successfully integrating digitalization. The study underscores the crucial role of digital adoption in enhancing the public sector’s performance, particularly when coupled with strong institutional frameworks.

1. Introduction

The primary objective of this research is to evaluate and rank the regions of the European Union based on their digital performance in terms of public service delivery. In addition, the research investigates the digital performance of the public sector in the countries of the European Union, incorporating different dimensions of digital efficiency and public service delivery and governance: digital infrastructure, digital economy, digital skills, e-government, and the Institutional Quality Index within a complex methodological framework.

Member states of the European Union have emphasized the importance of digital solutions and highlighted how digital innovations contribute to the optimal allocation of resources, obtaining a competitive advantage, and stimulating the digital performance of the public sector. However, it is necessary to evaluate the effects of digitalization because not all forms of digitalization can improve the performance of the public sector. The imperative of monitoring the efficiency of the public sector has led to the exploration of new methods and approaches for performance evaluation. Therefore, the increasing share of public services determines the intensification of the performance measurement process.

The complexity of these challenges, amplified by the current macroeconomic events: economic and political uncertainties, the COVID-19 pandemic, and conflicts that have affected economies globally, underscores the need for advanced methodologies to capture the multilateral nature of public sector performance. All these events have influenced citizens’ perceptions of the efficiency of the state in organizing and carrying out its main characteristics. Therefore, measuring the sector’s performance, which has led to a simplified perspective of its complexity, drives the evaluation of the state of the public sector. However, this endeavor requires a comprehensive conceptual framework as well as the use of robust evaluation methods. Due to the complexity of the public sector, there are also various instruments for determining its performance. Furthermore, the importance of developing precise performance measurement models supported by solid empirical evidence to refine policies and strategies in the public sector also arises.

Although the existing literature often constructs performance indices associated with the public sector, our research suggests that measuring the public sector’s performance through composite indicators faces significant obstacles due to insufficient data input. This can lead to unrealistic results and potentially flawed public policies. Furthermore, identifying the research hypotheses behind developing composite performance measurement indicators represents an entirely different challenge and can lead to measurement errors.

Our study is structured around the following research questions:

(i)

What are the key challenges in implementing e-government and achieving efficient digitization of public services in the member states of the European Union?

(ii)

How does the level of public sector digitization influence the efficiency of public service delivery in EU member states, and what policies can be implemented to maximize its positive impact on the public sector’s performance?

To address these questions, our research highlights the potential of comprehensive evaluations and the application of innovative econometric models, such as Principal Component Analysis (PCA), Gaussian graphical models, cluster dendrogram analysis, and Data Envelopment Analysis, as well as introducing the RTS Region calculation method, to assess the efficiency of the public sector at the EU-27 level and identify the best practice models and provide policy recommendations aimed at improving the public sector’s performance.

Our study addresses the existing gaps in the scientific literature, highlighting the importance of integrating multiple dimensions within a coherent framework for evaluating the performance of the public sector in numerous European Union member states. It goes beyond traditional one-dimensional assessments to provide a more holistic understanding of the factors determining performance.

This study offers a new analytical framework for evaluating the efficiency of the public sector in a multilateral EU context. The research distinguishes itself from other studies in the field by utilizing advanced methodological approaches that provide a comprehensive overview of measuring the public sector’s performance. Specifically, we focus on identifying significant variations in the digitalization of public services across EU member states, with a focus on advanced digital skills.

The innovative aspect of this study stems from the opportunity to examine new methodologies and perspectives that can contribute to the conceptualization and perspectives of innovations related to the progress of the public sector. By combining different methodological approaches, the research provides a more robust and nuanced analysis than would be possible with a single research method.

This research is not limited to the simple application of various methodologies to construct a relevant methodological framework. It also explores, tests, and provides various proxy indicators for measuring the public sector’s performance while formulating possible policy implications. However, there remain gaps in understanding how various country-specific factors interact with the implementation of e-government technology and digital technology to provide actionable insights. This methodological approach facilitated the measurement of the efficiency of organizational/individual decision-making units, with the member states of the European Union being divided into efficient and inefficient regions, with a focus on determining the digital performance. This approach highlights the specific positions of each country and the regional differences within the EU, thus contributing to tailored policy recommendations for improving performance. Therefore, the contribution of our study is the detailed examination of performance levels in the public sector of European Union countries. It has incorporated a composite indicator of institutional quality while calculating the efficiency using the RTS region calculation method introduced through the application of Data Envelopment Analysis.

Our empirical findings are aligned with the model developed by Androniceanu et al. [1], Yera et al. [2], and Doran et al. [3], highlighting the performance dynamics of the public sector in the European Union countries. Thus, our main findings highlight significant variations in the level of digitization of services among the EU-27 member states. These results also highlight the need to establish a stable yet flexible framework for regulating digital policies tailored to the specific level of digitalization of each member state.

Our findings support the hypothesis that the digitalization of the public sector has a positive impact on the provision of public services. However, our analysis shows significant national differences that need to be addressed in order to increase the digital efficiency and thus the generation of public services. Highlighting these disparities, we emphasize the importance of developing strategies to improve performance in specific contexts, thus promoting more significant equity in the provision of public services across the EU. A one-size-fits-all approach is unlikely to be effective given the diversity of digital landscapes across the EU. Instead, customized strategies are needed to address the unique challenges and opportunities facing each member state.

The presentation of the structure of the paper is shown in Figure 1. Thus, after an introduction that establishes the research context, highlighting the main tools for studying performance in the public sector, the second section presents the methods and techniques for measuring the public sector’s performance identified in the literature, ranging from non-parametric to parametric analyses. This review synthesizes key theoretical frameworks and empirical studies relevant to the subject, emphasizing the progress in understanding the analytical framework of the public sector’s performance in terms of efficiency and effectiveness. Section 4 describes the source of the data used for analysis and also details the econometric methods used to analyze data, such as Gaussian graphical models, Data Envelopment Analysis and clustering techniques, as well as other statistical tools that facilitate a thorough examination of the relationships between digitalization and e-government variables. Section 5 systematically reports the results in tabular and graphical form to illustrate key trends and relationships. The main findings provide empirical evidence of the implications of using digital technology on the performance of the public sector, documenting new perspectives on how various factors interact in the context of e-government. This approach allows for a deeper understanding of the regional disparities among EU member states. The final section concludes with the study’s findings, highlighting the main insights derived from the analysis.

2. Literature Review

2.1. Conceptual Delimitations Regarding Performance of Public Sector

Nowadays there is an increased interest in evaluating and measuring the performance of the public sector, both at the academic level and among international institutions. This trend results from many macroeconomic events, namely economic recessions, financial crises, and conflicts that have affected economies globally. These events have thus influenced citizens’ attitudes towards the state’s efficiency in the organization and conduct of its main characteristics. Therefore, the measurement of the performance of the public sector is based on the incorporation of methods borrowed from the private sector for the evaluation and improvement of performance, including the use of performance indicators [1].

There are a variety of forms for measuring performance in the public sector, and according to Djellal and Gallouj [2], these are represented by: effectiveness (the ratio between results and inputs), economic effectiveness (the ratio between results and costs), and technical/productive efficiency (related to outputs and inputs), as well as allocation efficiency (the ratio between results and costs). On the one hand, effectiveness refers to the degree of achievement of the proposed objectives. We are talking about economic efficiency when considering the costs involved in this process. On the other hand, efficiency highlights the capacity to achieve these objectives while minimizing the use of available resources. Consequently, efficiency is an indicator obtained by relating the effects of the results to the efforts expended. At the same time, effectiveness is the indicator given by the ratio between the result obtained and the one planned to be achieved. Additionally, Drucker [3] considers that there is no efficiency without effectiveness. Hence, the relationship between efficiency and effectiveness is that of a part to the whole, with effectiveness becoming necessary for achieving efficiency. Thus, efficiency is determined by the ratio between inputs and outputs. Therefore, effectiveness illustrates the success with which resources were used to achieve objectives, so effectiveness is more complex than efficiency [4]. However, a distinction appears between technical efficiency and allocation efficiency. Technical efficiency implies a relationship between inputs and outputs on the production frontier curve. However, not every form of technical efficiency is economically significant—this deficiency is captured by allocation efficiency, which requires a cost/benefit ratio. Furthermore, Profiroiu and Profiroiu [5] consider that the performance measures are represented by evaluating resource efficiency, evaluating costs, measuring efficiency, determining effectiveness, evaluating service quality, examining financial performance, and measuring overall performance.

According to the scientific literature, there are three general ways to determine the efficiency of the public sector, as a measure of performance in the public sector, namely: the use of composite indicators [6,7], the use of non-parametric methods, such as Data Envelopment Analysis (DEA), to obtain efficiency-based scores and thus identify best practices [8], and the use of stochastic frontier analysis for evaluating the performance of the public system [9]. Since performance measurement is a complex process, it requires appropriately considering many associated subdomains when calculating composite indicators. According to van Thiel and Leeuw [10], the decline in performance indicators is attributed to four processes. The first process is named positive learning. As performance improves, the indicators lose their sensitivity in detecting poor performance, suggesting that everyone has become so good at what they do that the indicator becomes outdated. The second process is represented by perverse learning, which occurs when organizations or individuals have learned what aspects of performance are measured and can use this knowledge to manipulate their evaluations. They can focus all efforts on what is measured, leading to increased performance. However, overall improvement may not materialize, and other aspects of performance may even deteriorate. The third process, called selection, assumes replacing the weakest performers with the best performers, reducing performance differences. As a result, only the best performers remain, and the indicator loses its discriminatory power, almost resembling a consequence of the survival of the fittest mechanism. The fourth process is represented by a suppression that arises when performance differences are ignored.

As a result of the complexity of the public sector, various instruments emerged for studying performance in the public sector, which are differentiated at two levels: management and performance measurement instruments at the organizational level and instruments for measuring the overall performance of the public sector. Thus, in the international specialized literature, there are a variety of approaches regarding the analysis of the public sector’s performance. Afonso et al. [6] focus on assessing the public sector’s performance in the new EU member states and emerging markets, using econometric techniques to measure the efficiency of public spending. They also propose economic and institutional factors as key explanations for performance differences between these emerging economies. Moreover, Afonso and Fernandes [11] measure the efficiency of local government spending in the Lisbon region, using Data Envelopment Analysis to compare the efficiency of different administrative units and to identify the variables that influence local government performance and how public spending can better align with community needs. Data Envelopment Analysis is also used by Afonso et al. [12], who use the public sector efficiency in Latin American regions and highlight the importance of regional context and institutional factors in assessing public sector performance, offering insights into the diversity of public sector performance across emerging countries. Furthermore, Rouag and Stejskal [13] analyze the public sector performance in North African and Middle Eastern countries, using quantitative approaches to evaluate the efficiency of governmental institutions. The study identifies key factors such as political stability, public administration efficiency, and economic development as essential in evaluating the public sector’s performance in these regions. Lobonț et al. [14] have another interesting approach, conducting a factor analysis to assess the public sector’s performance in EU member states and identifying significant differences between older and newer EU members. The study focuses on economic and institutional variables that influence government performance, highlighting the challenges faced by new member states in achieving optimal performance. Moreover, Muștea et al. [15] highlight multiple approaches to measuring performance at all levels of the public sector.

Therefore, determining performance is a complex process, particularly in the public sector, as public services present a wide range of activities and services provided to citizens and enterprises, and these services are difficult to evaluate using traditional measures based on productivity and efficiency. Moreover, public services are not sold on the market, and the prices of these services are not available, making their production challenging to quantify. Although the inputs for public services can be measured in physical units or financial resources, the issue arises when defining the output, which can face difficulties due to the varied perspectives of the consumers. Thus, being a complex process, academic literature presents the undesirable consequences associated with measuring the performance of the public sector. In this regard, the notion of the “Performance Paradox” emerges. The performance paradox is a theory presented by Meyer and Gupta [16], which suggests that organizations can maintain control by not having exact knowledge about the nature of performance. It is also important to have a basic understanding of performance evaluations and certifications (the reasons for it, the measures taken, responsiveness and efficiency). The performance paradox describes the limited relationship between performance values and performance [10]. This is due to fluctuations over time, as performance values tend to decrease as a precise measure of actual performance, making it difficult to distinguish between good and poor performers. Consequently, the link between reported performance and actual performance diminishes. It is also important to note that the paradox does not refer to performance but rather performance reporting. Having fewer performance indicators makes obtaining an accurate performance report more challenging. Another factor leading to unintended performance paradoxes is the elusive nature of policy objectives [17]. The emphasis on organizational monitoring and efficiency increases the likelihood of an unintended performance paradox. The extensive use of performance indicators can lead to employees understanding which aspects of their work are significant for management, with Smith [18] labelling this measure as “fixation”.

Measuring the performance of the public sector also faces a significant obstacle due to there being insufficient input data, which leads to unrealistic results and potentially flawed public policies. Additionally, identifying the research hypotheses behind developing performance measurement indicators is an entirely different challenge and can lead to measurement errors. This approach allows for a global understanding of public sector performance across dimensions. Thus, considering the variety of approaches to measuring the public sector’s performance, the theoretical framework for this study is primarily rooted in Efficiency Theory and Institutional Theory, focusing on Digital Governance and e-government. The overall theoretical model for this research combines the Efficiency Analysis (DEA) to assess performance metrics, the Institutional Theory to understand the institutional and regulatory factors that influence digital implementation, and the Digital Governance and e-government model to frame the role of technology in public sector service delivery.

2.2. Framework for Analyzing the Performance of the Public Sector in Terms of Efficiency

Numerous studies have highlighted the importance of the performance of the public sector and the need to measure it in many EU member states. They did not just focus on the different methodologies applied to build composite indicators by exploring, testing, and providing various proxies to measure the performance in terms of the efficiency of the public sector, but also on the possible implications. However, a traditional technical approach has evaluated the performance in public services based on the concept of efficiency [7,19,20,21].

According to the scientific literature, there are three general methods for determining the efficiency of the public sector. One of the methods involves using composite indicators [22,23]. These indicators facilitate the identification of a common trend in a set of separate indicators, providing a more straightforward interpretation of the results. As a result, they are becoming increasingly recognized as valuable tools in policy-making and the public communication of countries’ performance in various areas, such as the environment, economy, society, and technological development.

Principal Component Analysis is a multivariate statistical method that involves an orthogonal linear transformation of the data into a coordinate system. The projection of the data with the greatest variance becomes the first coordinate, called the first principal component. The second greatest variance becomes the second principal component, and the subsequent principal components follow.

$$\begin{gathered} W_{\left(k\right)}{=(W_1,\dots ,W_p)}_{\left(k\right)} \\ {(t_1,\dots ,t_m)}_{\left(i\right)} \end{gathered}$$

(1)

$$t_{k\left(i\right)}=x_{\left(i\right)}·w_{\left(k\right)}$$

(2)

$$sub-vertical{w}_{\left(1\right)}={}_{\left|\left|w\right|\right|=1}{}^{argmax}\{{\Sigma }_i{\left(t_i\right)}_{\left(i\right)}^2\}=\{{}_{\left|\right|w\left|\right|=1}{}^{argmax}{\Sigma }_i{(x_{\left(i\right)}·w)}^2\}$$

(3)

The equivalent matrix is:

$$w_{\left(1\right)}={}_{\left|\right|w\left|\right|=1}{}^{argmax}\{{\left|\left|Xw\right|\right|}^2\}={}_{\left|\right|w\left|\right|=1}{}^{argmax}\{w^T{X}^{T}Xw\}$$

(4)

$w_1$ is defined as a vector, relating to the equation:

$$w_{\left(1\right)}=argmax\{{\displaystyle \frac{w^T{X}^{T}Xw}{w^{T}w}}\}$$

(5)

The principal component $k$ is determined by removing $k_{-1}$ the principal component $X$:

$${\widehat{X}}_k=X-\sum _{s=1}^{k-1}{Xw}_{\left(s\right)}{w}_{\left(s\right)}^T$$

(6)

Then, the vector that removes the maximum variation of the new matrix is defined:

$$w_{\left(k\right)}={}_{\left|\right|w\left|\right|=1}{}^{argmax}\{{\left|\left|Xw\right|\right|}^2\}=\mathit{arg}\mathit{max}\{{\displaystyle \frac{w^T{\widehat{X}}_k^T\widehat{X_k}w}{w^{T}w}}\}$$

(7)

Then, the principal components $X^{T}X$ are proportional to the covariance matrix $X$ of the dataset, and the covariance $Q$ between the two principal components:

$$\begin{gathered} Q\left({PC}_{\left(j\right)},{PC}_{\left(k\right)}\right)\propto ({{Xw}_{\left(j\right)})}^T·({Xw}_{\left(k\right)}) \\ =w_{\left(j\right)}^T{X}^T{Xw}_{\left(k\right)} \\ =w_{\left(j\right)}^T{\lambda }_{\left(k\right)}{w}_{\left(k\right)} \end{gathered}$$

(8)

The covariance matrix of the initial values can be written as follows:

$$Q\propto X^{T}X={W\wedge W}^T$$

(9)

The covariance matrix of the two principal components can be written as follows:

$$W^{T}QW\propto =W^T\wedge W^{T}W=\wedge$$

(10)

$$X^{T}Xand{\lambda }_{\left(k\right)}=\sum t_{k\left(i\right)}^2=\sum {(x_{\left(i\right)}·w_{\left(k\right)})}^2$$

(11)

Therefore, evaluating the public sector’s performance involves considering the state’s key areas and main activities.

Additionally, the study by Afonso et al. [23] determines the performance of the public sector in general terms, the Public Sector Performance indicator can be determined using the following formula:

$${PSP}_i=\sum _{j=1}^n{PSP}_{ij}$$

(12)

$${PSE}_i={\displaystyle \frac{{PSP}_i}{{PXP}_i}}=\sum _{j=1}^n{\displaystyle \frac{{PSP}_{ij}}{{PXP}_{ij}}}$$

(13)

Afonso et al. [23] mention using specific indicators, such as economic performance, income distribution, economic stability, quality of governance, and health and education systems. Each indicator is calculated by averaging the standardized values of the sub-indicators that contribute to its definition.

Summarizing the entire spectrum of the public sector’s performance measurement through composite indicators, Lobonț et al. [14] document the importance of differentiating between parameters and indices. These performance measurement indices result from aggregating multiple weighted average parameters, allocating reciprocally relevant weights in constructing the indices. This approach allows for a comprehensive understanding of the public sector’s performance across dimensions. However, it should be noted that if these composite indicators are constructed improperly or misinterpreted, they can lead to misleading political messages or simplistic public or political decisions, or can be used abusively, for example, to support a desired policy, if the construction process is not transparent or lacks understanding of statistical or conceptual principles. Therefore, selecting indicators and weights could be the subject of political disputes or tensions. Moreover, they can hide gaps in some dimensions and can lead to a reduction in the process of identifying appropriate remedial measures if the construction process is not transparent and understood. Tracing inappropriate policies in cases where performance dimensions that are difficult to measure is ignored or not included.

Smith and Street [24] discuss the limitations of analysis in measuring public service efficiency, reminding us that qualitative assessments and contextual understanding should complement quantitative measures. The measurement of public service productivity is in a pre-paradigmatic phase. This implies testing different approaches based on the distinction between inputs, outputs, and outcomes of public services (outcome). Pestieau [25] argues for the necessity of introducing performance measurement of public services, in addition to input and output measures, which reflect both quantity and quality, and it is equally important to introduce information on the institutional environment. Therefore, the performance of public services depends not only on inputs and outcomes but also on institutional and regulatory aspects. Non-parametric methods are often preferred, as they can be applied to ordinal, nominal, or interval data. In contrast, parametric methods, such as Stochastic Frontier Analysis (SFA), Distribution-Free Analysis (DFA), and the Thick Frontier Approach (TFA), are applied to interval data with a normal Gaussian distribution, making them suitable for specific analyses and evaluations in this field.

Additionally, non-parametric methods, such as Data Envelopment Analysis (DEA) and Free Disposal Hull (FDH), are used to obtain efficiency-based scores and thus identify best practices [8]. Furthermore, Stochastic Frontier Analysis is noteworthy for evaluating the performance of the public sector [26]. Therefore, numerous parametric and non-parametric research techniques are employed to determine the level of performance in the public sector.

Frontier analysis methods are among the most sophisticated performance measurement tools that enable the investigation of complex multidimensional production processes. Frontier analysis methods can be divided into two groups: Stochastic Frontier Analysis (SFA), Distribution-Free Analysis (DFA), and the Thick Frontier Approach (TFA), and the non-parametric Data Envelopment Analysis (DEA) and Free Disposal Hull (FDH). Retaining the proposals of Mihiț et al. [27], non-parametric tests are more suitable for measuring the performance and efficiency of the public sector, given that the analyzed data will not follow a specific distribution.

Additionally, non-parametric frontier estimation methods, such as DEA, are used to measure the efficiency of a production unit in comparison to an estimated non-parametric frontier. This involves avoiding assumptions about the analytical structure of the production function and the use of production units. The model equations can be expressed as follows:

$$E_0={\displaystyle \frac{{\Sigma }_r{u}_r{y}_{rj0}}{{\Sigma }_j{v}_i{x}_{ij0}}}\to max$$

(14)

$$E_0={\displaystyle \frac{{\Sigma }_r{u}_r{y}_{rj0}}{{\Sigma }_j{v}_i{x}_{ij0}}}\le 1$$

(15)

$$u_r\ge 0$$

(16)

$$v_i\ge 0$$

(17)

The variables $u_r$ and $v_i$ are used to indicate the significance attributed to each input and output. It is difficult, if not impossible, to establish a universal set of weights for evaluating all variables. Moreover, determining the significance for each unit involves subjectivity, as different units perceive the importance of inputs and outputs differently. Nevertheless, this method also has some disadvantages, such as subjectivity, which could lead to inaccurate efficiency assessments for some units. In contrast, when inefficiency is identified to some degree, it provides certainty about the result within the framework in which indicators favoring efficiency have been selected.

The DEA model is a multivariate non-parametric technique that constructs a frontier function through a linear approach by comparing ratios based on multiple inputs and outputs of similar units in the observed dataset [28]. At the same time, DEA benefits from the advantages similar to a production function but avoids the errors related to the specification of the form. Within the DEA model, the productivity score of any unit is calculated as the maximum ratio between weighted outputs and inputs. Thus, the condition is imposed that, for all other units in the dataset, the similar ratios must be less than or equal to one. Productivity can be obtained through the following model [29]:

$$Max{h}_0={\displaystyle \frac{\sum _{r=1}^t{U}_r{Y}_{rj0}}{\sum _{i=1}^m{V}_i{X}_{ij0}}}$$

(18)

$$\begin{gathered} {\displaystyle \frac{\sum _{r=1}^t{U}_r{Y}_{rj}}{\sum _{i=1}^m{V}_i{X}_{ij}}}\le 1;j=1,\dots ,n \\ {U}_r{V}_i\ge 0;r=1,\dots ,s;i=1,\dots ,m \end{gathered}$$

(19)

$Y_r$ and $X_{ij}$ represent the quantity of outputs and inputs, while the weights are to be estimated based on all the data in the model. The model includes output variables, input variables m and n, and the total number of units. If the DMU is on the efficiency frontier (that is, it is part of the reference set), its productivity score will indicate maximum efficiency compared to the other units analyzed. Other less efficient properties selected from these data will be oriented below the efficiency frontier, and their scores will be less than 100%.

The BCC model, named after the authors Banker, Charnes, and Cooper, considers returns to scale and also includes the dimension of deviation from constant returns to scale, $qq0$. The range of variables included in this method allows for a more in-depth evaluation of performance.

The concept of returns to scale is related to the production function. Thus, if the output increases in the same proportion as all the inputs increase, we identify a constant return to scale, and an evolution of different proportions implies a variable return to scale. Input or output orientation refers to measures to improve the efficiency of units through solutions of increasing or decreasing inputs or outputs.

$$Max{E}_K=\sum _{n=1}^N{U}_n{Y}_{jn}-u_0$$

(20)

$$\mathrm{s}.\mathrm{t}. \sum _{m=1}^{nM}{V}_m{X}_{im}-u_0=1$$

(21)

$$\begin{gathered} \sum _{n=1}^N{U}_n{Y}_{rn}-\sum _{m=1}^M{V}_m{X}_{rm}-u_0\le 0 \\ {U}_n,V_m\ge 0;n=1,\dots ,N;m=1,\dots ,M;r=1,\dots ,R \\ {U}_0\mathrm{c}\mathrm{a}\mathrm{n}\mathrm{t}\mathrm{a}\mathrm{k}\mathrm{e}\mathrm{b}\mathrm{o}\mathrm{t}\mathrm{h}\mathrm{n}\mathrm{e}\mathrm{g}\mathrm{a}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{e}\mathrm{a}\mathrm{n}\mathrm{d}\mathrm{p}\mathrm{o}\mathrm{s}\mathrm{i}\mathrm{t}\mathrm{i}\mathrm{v}\mathrm{e}\mathrm{v}\mathrm{a}\mathrm{l}\mathrm{u}\mathrm{e}\mathrm{s} \\ \mathrm{e}\mathsf{\lambda } \end{gathered}$$

(22)

Returns to scale are related to the production function. If outputs increase in the same proportion as inputs, input or output orientation implies improving the efficiency of the unit through solutions that increase or decrease the total number of inputs and/or outputs.

However, the FDH model allows for free availability to construct the production possibility set. Consequently, the frontier line for the FDH model is developed from the observed inputs and outputs that allow for free disposal. Thus, two constraints are added in the FDH model: $e\lambda =1$ and ${\lambda }_j\in \left\{\mathrm{0,1}\right\}$. Performance must have zero or unit values, so that, together with $e\lambda =1$, only one of the actually observed performances will be chosen. Consequently, Equation (23) must be solved, followed by Equation (24).

$$\begin{gathered} \mathit{min}{\theta }_k \\ \mathrm{s}.\mathrm{t}.{\theta x}_k-X\lambda \ge 0 \\ Y\lambda \ge y_k \\ e\lambda =1,{\lambda }_j\in \left\{\mathrm{0,1}\right\} \end{gathered}$$

(23)

$$\begin{gathered} \mathit{max}\sum s^{+}+s^{-} \\ \mathrm{s}.\mathrm{t}.{\theta x}_k-X\lambda \ge 0 \\ Y\lambda \ge y_k \\ e\lambda =1,{\lambda }_j\in \left\{\mathrm{0,1}\right\} \end{gathered}$$

(24)

For the analysis of costs, revenues, and profits, we use a stochastic frontier analysis model. The Stochastic Frontier Analysis method starts from a production frontier model without the random component [30,31], as follows:

$$Y_{it}=f(X_{it};\beta )\mathit{exp}\left(v_{it}-u_{it}\right);i=1,\dots ,N;t=1,\dots ,T$$

(25)

$Y_{it}$ is the form of production associated with time t, where $X_{it}$ denotes a vector of inputs, and $\beta$ represents a vector coefficient of inputs for the independent variables. $v_{it}$ represents random variations in the production function. These variations can result from measurement errors, specification effects, or exogenous shocks. Another element within the disturbance term, $u_{it}\left(X_0\right)$, reflects the productive inefficiency specific to a particular economic unit compared to the stochastic production function.

At the same time, the non-negative character of the disturbance term $u_{it}$ indicates that production is above or below its maximum capacity. Any deterministic deviation in the stochastic production function can be attributed to factors as follows: (i) symmetric random variation ${f(X}_{it};\beta$), represented by the component $v_{it}$, and (ii) asymmetric variation or productive inefficiency. The term $u_{it}$ quantifies productive inefficiency, as it measures the production deficit $Y_{it}$ compared to what is suggested by its maximum frontier determined by $f(X_{it}\mathit{exp}{v}_{it}$).

Moreover, there are numerous parametric and non-parametric research methods used in the construction of performance indicators in the public sector. Since non-parametric methods apply to ordinal, nominal, or interval data, they are considered more suitable for the social sciences, while parametric methods generally apply to interval data, with a Gaussian normal distribution [32]. Furthermore, there are numerous parametric and non-parametric research methods used in the construction of performance indicators in the public sector. Since non-parametric methods apply to ordinal, nominal or interval data, they are considered more suitable for social sciences, while parametric methods generally apply to interval data, with a normal Gaussian distribution [14]. Current approaches to measuring performance in the public sector take into account the multidimensional nature of public sector performance and have abandoned the supremacy of financial indicators. However, performance measurement systems as a whole are still confusing, but because they are necessary, their construction can give rise to unintended effects, contrary to expectations, being subject to the risk that negative effects may result from the beneficial effects [33]. Unintended effects of performance measurement in the public sector can take various forms, and knowledge of these is useful in building an optimal system for managing and measuring performance.

Speklé and Verbeeten [34] emphasize the existence of multiple approaches to measuring performance at all levels of the public sector. Therefore, determining performance is a complex process, particularly in the public sector, as public services present a wide range of activities and services, provided to both citizens and enterprises, and these services are difficult to evaluate using traditional measures based on productivity and efficiency.

Considering the proposals of Mihiț et al. [27] and Muștea et al. [15], non-parametric tests are considered more suitable for measuring the performance and effectiveness of the public sector, especially when the analyzed data do not follow a specific distribution. The decision to use non-parametric tests should focus on various factors: small sample sizes; analysis of ordinal data or their performance, data classification, or higher values; using the median as an appropriate measure of central tendency instead of the mean; and variables and attributes. Therefore, the need for a robust framework for comparing countries is recognized, in terms of the quality of public policies, institutions, and decision making.

3. Data and Methodology

The main objective of this research is to identify the most efficient European region in terms of public sector digital performance by calculating the efficiency of organizational units within each region relative to best practices observed within the same group. Therefore, this study provides an overview of ways to determine the digital performance while also identifying the issues that can arise in implementing e-government. The empirical analysis involved several complementary research stages. First, to simplify the complex indicators of Worldwide Governance Indicators (Regulatory Quality, Government Effectiveness, Political Stability and Absence of Violence, and Rule of Law), the Institutional Quality Composite Index (IQI) was built.

After that, we demonstrated the relationship between five key dimensions, digital infrastructure, digital economy, digital skills, e-government, and institutional quality, using the Gaussian graphical model. This was followed by a non-parametric approach employing Data Envelopment Analysis. The final stage represented the digital performance of the public sector by applying vector quantization through hierarchical clustering. This allowed for the grouping of EU regions based on the efficiency of implementing digital technology in public service delivery. Consequently, three different research methods were used to classify EU member states in terms of their digital efficiency, as follows:

(a) Principal Component Analysis (PCA) is a statistical technique used to reduce the dimensionality of a large dataset, transforming a set of correlated variables into a smaller number of uncorrelated variables, called principal components. PCA identifies the directions in which the data variation is maximized. Composite indicators of institutional quality for European countries between 2017 and 2022 reflect their performance and evolution in terms of governance and stability. Positive values above 1 suggest stronger and more efficient institutional quality, while negative values around −1 indicate challenges and deficiencies. A composite indicator is constituted based on a set of relative indicators, grouped by domains, capable of providing a relative, comparative view of the performance level of research units. Thus, the following steps can be individualized [35]: (i) defining the phenomenon to be measured; (ii) selecting a group of individual indicators; (iii) normalizing the individual indicators; and (iv) aggregating the normalized indicators. The use of composite indicators facilitates the identification of a common trend in a set of separate indicators, providing an easier interpretation of the results. As a result, they are becoming increasingly recognized as useful tools in policy-making and in the public communication of countries’ performance in various fields.

(b) Gaussian Graphical Models evaluate the global interconnections between various factors that influence digital performance in the EU. To capture the complex interplay between digital infrastructure, digital economy, digital skills, and the institutional quality index affecting digital performance (e-government), we employ two complementary graphical modelling approaches: Gaussian graphical models. GGMs are efficient at revealing conditional dependencies among continuous variables.

(c) Data Envelopment Analysis (DEA)—to identify the most efficient European region in terms of the digital performance of the public sector, we calculated the efficiency of an organizational unit within a group, in relation to the best practices observed within the same group, through Data Envelopment Analysis (DEA). Additionally, the DEA method assesses the efficiency of similar entities that use multiple resources to produce multiple results on a production frontier that represents the optimal level of performance or efficiency. It should be noted that this methodology has some limitations, as DEA does not refer to a universally accepted set of best practices when determining efficient units and efficiency measures. Additionally, it presents large differences in results depending on the assignment of the quality of the input/output variables.

Within this analysis based on the DEA method, eleven indicators were selected (11 input indicators and 3 output indicators) that can highlight the disparities among the member states of the European Union from the perspective of digital progress. To initiate this DEA-based analysis, the following steps are necessary: selection of input and output variables, and efficiency analysis. It is also important that the inputs and outputs are correlated with the number of decision-making units. A generally valid rule assumes that the sum of the number of inputs and outputs should be less than or equal to 1/3 or 1/11 of the total number of DMUs. The correct selection of input and output units is essential in using the DEA method. We will further apply the DEA method at the level of the European Union. In order to obtain an optimal predictive model, we have reduced the total number of inputs to 11 and 3 outputs. The 3 units related to the e-governance dimension represent the output units. Selecting input and output variables for Data Envelopment Analysis (DEA) models involves a complex approach. Based on the literature, Data Envelopment Analysis is used in e-government research to assess efficiency and performance. Studies employ DEA to: evaluate the technical efficiency of public libraries using technology-based outputs [36]; and assess the levels of digital society and e-government in different countries [37]. Krejnus et al. [38] reveal that DEA has been used in the context of digital transformation to evaluate the levels of digital society and e-government in different countries. Hsieh et al. [39] evaluate the performance of government web portals, considering factors like information, interaction, and transaction capabilities as outputs, and organizational and institutional factors as inputs.

The selection of indicators along with the application of models oriented towards either inputs or outputs also enables the grouping of countries into distinct subregions. Furthermore, the analysis based on the DEA method allows for the evaluation of the level of digitalization for each country, as well as the division of countries into efficient regions to compare efficiency scores. The application of the CRS and VRS models allows for the evaluation of e-governance performance in the EU. The fundamental purpose of this analysis is to evaluate the efficiency of decision-making units (DMUs) by analyzing the inputs to generate one or more outputs. The formula for calculating the efficiency of decision-making units with a single input and a single output is defined by the following relationship (26) and (27):

$$Efficiency={\displaystyle \frac{output}{input}}$$

(26)

For multiple inputs and outputs, the following relationship will be used:

$$Relativeefficiencyscore={\displaystyle \frac{weightedsumofoutputs}{weightedsumofinputs}}$$

(27)

The BCC model is:

$$\begin{gathered} E_0=\sum _r{u}_r{y}_{r{j}_0}+q_0\to max \\ \sum _i{v}_i{x}_{i{j}_0}=1 \\ -{\sum }_i{v}_i{x}_{i{j}_0}+{\sum }_r{u}_r{y}_{r{j}_0}+q_0\le 0 \end{gathered}$$

(28)

Scale returns are associated with each type of data envelopment surface determined by each DMU, decision-making unit, which corresponds to the efficiency frontier. Units that are on this surface are considered efficient, while those that do not fit within this surface are considered inefficient. The following steps are considered to transform inefficient decision-making units into efficient units: reducing inputs while at the same time keeping outputs constant (input-oriented analysis), increasing outputs while inputs remain constant (output-oriented analysis), or increasing outputs while reducing inputs (the dual version). It is important to note that input-oriented measurement is equivalent to output-oriented measurement of technical efficiency (TE) only in the case of constant returns to scale (CRS). For a more comprehensive analysis of efficiency, multiple DEA methods are recommended. In this context, the CRS and VRS models oriented towards maximizing outputs will be used to determine the efficiency of the e-governance process in the European Union member states, considered homogeneous decision-making units.

(d) The statistical analysis focused on disparities, using the R software, will apply the cluster and hierarchical clustering techniques to partition the EU-27 countries into homogeneous groups. Group analysis is a statistical data processing method, a quantitative form of grouping data into categories based on their specific differences and similarities. It includes many different algorithms and methods for clustering data and creating data groups. Thus, the cluster dendrogram was applied to determine the impact of digital technology integration on e-government and to measure the efficiency of public services by the effects of various digitalization and e-government indicators in the European Union member states.

Consistent with the methodologies utilized in prior studies by Noja et al. [40], Bartolacci et al. [41], and Bánhidi and Dobos [42], the present investigation incorporated comparable metrics and associated methodological specifications. To achieve our overarching objective and in alignment with the established methodological procedures, we have formulated the following two hypotheses (H) for testing:

H1: 

There is a positive association between the level of public sector digitization and the efficiency of public service delivery in the member states of the European Union.

H2: 

EU member states with a higher level of digital technology adoption and digital skills among the population achieve superior performance in implementing and utilizing e-government services.

E-governance is a multifaceted phenomenon that encompasses various interconnected dimensions that influence the entire governance process. This research evaluates the efficiency of the European Union regions in terms of the progress of the digitalization of public services. Therefore, rather than attributing e-governance to a single dimension, viewing it as a multifaceted process where these dimensions are interconnected and influence each other is more accurate.

To achieve the study’s overall objective, we compiled a dataset based on the relevant literature. For the methodological operationalization, the variables retained for analysis were grouped into dimensions. Data were extracted for the years 2017 and 2022 for all European Union member states (EU-27), from established databases such as Eurostat, the European Commission, and the World Bank. Consequently, the analysis is not limited only to digitalization indicators: we also included the institutional quality component to capture the multifaceted aspects of public sector performance, as shown in

Table 1. Description of digitalization and e-government indicators.

Index	Code	Source	Attribution
A. Digital Efficiency and Public Service Delivery
I. Digital Infrastructure
(i) Infrastructure and Access
Level of Internet Access	LIA	Eurostat	Input
Fixed Broadband Subscription	FB	The World Bank	Input
(ii) Usage and Adoption
Individuals using the Internet	IUI	The World Bank	Input
(iii) Integration of Technology
Integration of Digital Technology	IDT	Digital Economy and Society Index	Input
Connectivity	Connect	Digital Economy and Society Index	Input
Communication	COM	Digital Economy and Society Index	Input
II. Digital Economy
High-Technology Exports	HTE	The World Bank	Input
ICT Service Exports	ICT	Eurostat	Input
III. Digital Skills (Social Digital)
(i) Skills and Education
Human Capital	HC	Digital Economy and Society Index	Input
Advanced Skills and Development	ASD	Digital Economy and Society Index	Input
IV. e-Government
(i) Digital Service Delivery
Digital Public Services by e-government	DPS	Digital Economy and Society Index	Output
(ii) Open Data and Citizen Engagement
e-Government Users-Open Data	GUOD	Digital Economy and Society Index	Output
(iii) Overall e-Government Usage
e-Government Users	e-gov	Digital Economy and Society Index	Output
B. Governance
V. Institutional Quality
Regulatory Quality	RQ	The World Bank	Input
Government Effectiveness	GE	The World Bank	Input
Political Stability and Absence of Violence	PSAV	The World Bank	Input
Rule of Law	RL	The World Bank	Input

Source: authors’ own processing.

.

The indicators in our dataset reflect various aspects of the relationship between two main dimensions: digital efficiency and public service delivery and governance in the EU. The digital efficiency and public service delivery dimension has been divided into four sub-dimensions: Digital Infrastructure, Digital Economy, Digital Skills, and e-government. Additionally, the governance dimension has been incorporated as a related sub-dimension based on a composite Institutional Quality Index, as illustrated in Figure 2.

The first dimension, A. Digital Efficiency and Public Service Delivery, is grouped into these following sub-dimensions: I. Digital infrastructure: (i) Infrastructure and Access: World Bank—Fixed broadband subscriptions; Eurostat—Level of Internet Access (% of households); (ii) Usage and Adoption: World Bank—Individuals using the Internet (% of population); (iii) Integration of Technology: Digital Economy and Society Index—Integration of Digital Technology (weighted score: 0 to 100); Connectivity (weighted score: 0 to 100); World Bank—Communication (% of service imports); II. Digital Economy: World Bank—High-tech Exports (% of manufactured exports); Eurostat—ICT Service Exports (current USD); III. Digital Skills (Social Digital): i) Skills and Education: Digital Economy and Society Index—Human Capital (weighted score: 0 to 100), Advanced Skills and Development (weighted score: 0 to 100); IV. e-government: Digital Economy and Society Index- e-Government Users-Open Data (% of people who have used the Internet in the last 12 months), Digital Public Services—e-Government (weighted score: 0 to 100), Index of the Digital Economy and Society—e-Government Users (weighted score: 0 to 100). The second dimension, B. Governance, is grouped into the sub-dimension V. Institutional quality: World Bank—Regulatory Quality (score); Government Effectiveness (score); Political Stability and Absence of Violence; Rule of Law (score).

4. Results

In the empirical analysis, we have focused on the segmentation of the European Union regions based on digital performance, providing a new perspective on the role of digitization and its effects on the operationalization of public services.

To test hypothesis H1, we first constructed our own composite indicators of institutional quality for European countries, using Principal Component Analysis for two separate time periods, 2017 and 2022, based on four governance indicators (Regulatory Quality, Government Effectiveness, Political Stability and Absence of Violence, Rule of Law). These new indicators reflect the country’s performance and evolution in terms of institutional quality. Positive values above 1 suggest stronger and more efficient institutional quality, while negative values around −1 indicate challenges and deficiencies.

After that, we constructed Gaussian graphical models (GGMs). The Gaussian graphical models were analyzed based on partial correlation, extended Bayesian information criteria/contraction operator, and absolute minimal contraction selection for 2017 and 2022. Each indicator in the configured network represents an interconnected node linked to other nodes, reflecting their interdependence. The estimates were performed using partial correlation estimation and the Bayesian EBIC information criterion-based method.

In the 2017 Gaussian graphic model (

Table 2. GGM 2017 key variable relationships.

Node_A	Node_B	Link_Type	Color	Intensity
COM	FB	positive	blue	very strong
COM	IQI	positive	blue	very strong
Connect	DPS	positive	blue	very strong
DPS	e.gov	positive	blue	very strong
HC	ICT	positive	blue	very strong
IUI	LIA	negative	red	very strong
IDT	HTE	negative	red	very strong
FB	HTE	negative	red	very strong
e.govOD	ICT	negative	red	very strong
IUI	COM	negative	red	very strong

COM: Communication; Connect: Connectivity; IQI: Institutional Quality Index; DPS: Digital Public Services; e.gov: E-government users; FB: Fixed Broadband Subscription; HTE: High-Technology Exports; IUI: Individuals Using the Internet; IDT: Integration of Digital Technology; LIA: Level of Internet Access; HC: Human Capital; ICT: ICT Service Exports; e.govOD: E-government Users-Open Data. Source: authors’ own processing.

), shown in Figure 3a, stronger links capture various associations related to digital infrastructure, digital economy, digital skills, and e-government and governance. The GGM network includes 14 variables/nodes and links or edges between each pair, thus distinguished, differentiated nodes: ASD, COM, Connect, IQI, DPS, e.gov, FB, HTE, IUI, IDT, LIA, HC, ICT, e.govOD. And the connection between the edges for each pair of vertices is determined by a connection. The blue color indicates positive associations, while the red color indicates negative associations, and the thickness of the edge reflects the strength of the bond.

The strongest positive associations are as follows (intense blue): communication (COM) and Fixed Broadband Subscription (FB); communication (COM) and Institutional Quality Index (IQI); Connectivity (Connect) and Digital Public Services (DPS); Digital Public Services (DPS) and e-Government Users (e.gov); and Human Capital (HC) and ICT service exports (ICT). The strongest negative associations are as follows (intense red): Individuals using the Internet (IUI) and Level of Internet Access (LIA); Integration of Digital Technology (IDT) and High-Technology Exports (THE); Fixed Broadband Subscription (FB) and High-Technology Exports (THE); e-Government Users-Open Data (e.govOD) and ICT service exports (ICT); and Individuals using the Internet (IUI) and Communication (COM).

The Gaussian graphical model for 2022 (

Table 3. GGM 2022 key variable relationships.

Node_A	Node_B	Link_Type	Color	Intensity
ASD	COM	negative	red	very strong
ASD	FB	positive	blue	very strong
ASD	IUI	positive	blue	very strong
COM	Connect	negative	red	very strong
COM	IQI	positive	blue	very strong
Connect	HTE	positive	blue	very strong
ICT	e.gov	negative	red	very strong
HC	e.gov	positive	blue	very strong
HC	e.govOD	positive	blue	very strong
IDT	LIA	positive	blue	very strong

ASD: Advanced Skills and Development; COM: Communication; Connect: Connectivity; IQI: Institutional Quality Index; DPS: Digital Public Services; e.gov: E-government users; FB: Fixed Broadband Subscription; HTE: High-Technology Exports; IUI: Individuals Using the Internet; IDT: Integration of Digital Technology; LIA: Level of Internet Access; HC: Human Capital; ICT: ICT Service Exports; e.govOD: E-government Users-Open Data. Source: authors’ own processing.

), shown in Figure 3b, highlights the significant interconnections between all variables, with 14 nodes connected to each other (ASD, COM, Connect, DPS, FB, HTE, HC, ICT, IDT, IUI, IQI, LIA, e.gov and e.govOD), Each node has exactly 13 connections, indicating a symmetric distribution of the links. The Gaussian graphical model implies very strong interconnections between the variables/nodes considered in this research, as reflected by the width/thickness of the links. The maximum interdependence between the variables is 1, measured by the average distance between two variables. Advanced Skills and Development (ASD) is negatively associated with COM (communication). It also has a partially positive correlation with fixed broadband (FB) and people who use the internet (IUI). Communication (COM) shows relatively negative associations with connectivity (Connect) but a positive relation with the Institutional Quality Index (IQI). Meanwhile, connectivity (Connect) is positively related to high-technology exports HTE). ICT specialists are negatively associated with e-government (e.gov), but human capital (HC) along with e-government and e-government online delivery (e.govOD) present a positive relationship. Information and Communications Technology (IDT) and Level of Internet Access (LIA) are positive associations. There are significant overall implications of the dimensions of digitalization and innovation on e-governance in the EU-27 countries. There are significant overall implications of the dimensions of digital efficiency and public service delivery and governance: digital infrastructure, digital economy, digital skills, e-government, and governance: institutional quality in the EU-27 countries.

Furthermore, we investigated centrality indicators to better understand how the variables influence the behavior of the estimated networks. Additionally, we explored and compared the resulting models to understand their differences and similarities, thus providing a deeper perspective on the structure and relationships within the GGM networks.

According to the analysis of the centrality index, Figure 4, digital intensity (IDT) and our composite Institutional Quality Index (IQI) are the central powers in the network of interactions between the analyzed variables. This suggests that the quality of regulation plays a crucial role within the integration of digitalization. Additionally, the results indicate that advanced skill development has a lesser influence in this network. These findings highlight the significant implications of the diverse aspects of digitalization and innovation within the e-government landscape across the EU-27 member states. Such insights suggest that enhancing digital capabilities and fostering innovative practices can improve public service delivery and economic performance. Based on these results, hypothesis H1 is validated, namely “There is a positive association between the level of digitalization in the public sector and the efficiency of public service delivery in the European Union member states”.

The findings align with those of Criste et al. [43] and Crăciun et al. [44], highlighting that digital technology is generally considered a relative, but not definitive, solution for reducing pressure and improving the performance of the public sector. While digitalization involves a series of advantages, such as streamlining processes and improving the delivery of public services, some limitations must be considered to ensure the efficiency of public service delivery among taxpayers.

For the second research hypothesis (H2), the EU member states with a higher level of digitalization will achieve superior performance in the implementation and use of e-government services; we used DEA, which performs a relative efficiency assessment by relating inputs to outputs for each decision-making unit (DMU), to measure their efficiency compared to the other decision-making units. To select the most relevant input and output variables, we first applied correlation analysis (Appendix A). This allowed us to detect multicollinearity and provided a statistical basis for using DEA. By detailing the relationships between variables, we ensured the robustness of the DEA model, an analysis that validated its applicability.

The basic models of the DEA methodology are the CRR input-oriented and output-oriented model, the input-oriented and output-oriented BCC model, and the SBM model. However, the present research focuses on constant (CRS) and variable returns to scale (VRS) for minimizing inputs and maximizing outputs. DEA models are based on the set of admissible possibilities, represented by all possible combinations of inputs and outputs, and are also limited by the efficiency frontier. Efficient units are those units that have input and output combinations located on the efficiency frontier. These efficiency frontiers are characteristic of both the CRS and VRS models.

Table 4. DEA Analysis Results for the year 2022.

			Input-Oriented	Input-Oriented			Output-Oriented	Output-Oriented
			VRS	CRS		Input-Oriented	VRS	CRS		Output-Oriented
DMU No.	DMU Name	RTS Region	Efficiency	Efficiency	Sl	RTS	Efficiency	Efficiency	Sl	RTS
1	Austria	Region II	1.00000	1.00000	1.00000	Constant	1.00000	1.00000	1.00000	Constant
2	Belgium	Region II	1.00000	1.00000	1.00000	Constant	1.00000	1.00000	1.00000	Constant
3	Bulgaria	Region II	1.00000	1.00000	1.00000	Constant	1.00000	1.00000	1.00000	Constant
4	Croatia	Region II	1.00000	1.00000	1.00000	Constant	1.00000	1.00000	1.00000	Constant
5	Cyprus	Region II	1.00000	1.00000	1.00000	Constant	1.00000	1.00000	1.00000	Constant
6	Czechia	Region II	1.00000	1.00000	1.00000	Constant	1.00000	1.00000	1.00000	Constant
7	Denmark	Region II	1.00000	1.00000	1.00000	Constant	1.00000	1.00000	1.00000	Constant
8	Estonia	Region II	1.00000	1.00000	1.00000	Constant	1.00000	1.00000	1.00000	Constant
9	Finland	Region II	1.00000	1.00000	1.00000	Constant	1.00000	1.00000	1.00000	Constant
10	France	Region II	1.00000	1.00000	1.00000	Constant	1.00000	1.00000	1.00000	Constant
11	Germany	Region III	0.87808	0.84015	1.16676	Decreasing	1.06886	1.19026	1.38875	Decreasing
12	Greece	Region II	1.00000	1.00000	1.00000	Constant	1.00000	1.00000	1.00000	Constant
13	Hungary	Region II	1.00000	1.00000	1.00000	Constant	1.00000	1.00000	1.00000	Constant
14	Ireland	Region II	1.00000	1.00000	1.00000	Constant	1.00000	1.00000	1.00000	Constant
15	Italy	Region II	1.00000	1.00000	1.00000	Constant	1.00000	1.00000	1.00000	Constant
16	Latvia	Region II	1.00000	1.00000	1.00000	Constant	1.00000	1.00000	1.00000	Constant
17	Lithuania	Region II	1.00000	1.00000	1.00000	Constant	1.00000	1.00000	1.00000	Constant
18	Luxembourg	Region II	1.00000	1.00000	1.00000	Constant	1.00000	1.00000	1.00000	Constant
19	Malta	Region II	1.00000	1.00000	1.00000	Constant	1.00000	1.00000	1.00000	Constant
20	Netherlands	Region III	1.00000	0.92938	1.41560	Decreasing	1.00000	1.07599	1.52317	Decreasing
21	Poland	Region II	1.00000	1.00000	1.00000	Constant	1.00000	1.00000	1.00000	Constant
22	Portugal	Region II	1.00000	1.00000	1.00000	Constant	1.00000	1.00000	1.00000	Constant
23	Romania	Region II	1.00000	1.00000	1.00000	Constant	1.00000	1.00000	1.00000	Constant
24	Slovakia	Region II	1.00000	1.00000	1.00000	Constant	1.00000	1.00000	1.00000	Constant
25	Slovenia	Region II	1.00000	1.00000	1.00000	Constant	1.00000	1.00000	1.00000	Constant
26	Spain	Region II	1.00000	1.00000	1.00000	Constant	1.00000	1.00000	1.00000	Constant
27	Sweden	Region III	1.00000	0.74503	1.34206	Decreasing	1.00000	1.34223	1.80135	Decreasing

Source: authors’ own processing in DEAFrontier.

and

Table 5. DEA analysis results for the year 2017.

			Input-Oriented	Input-Oriented			Output-Oriented	Output-Oriented
			VRS	CRS		Input-Oriented	VRS	CRS		Output-Oriented
DMU No.	DMU Name	RTS Region	Efficiency	Efficiency	Sl	RTS	Efficiency	Efficiency	Sl	RTS
1	Austria	Region II	1.00000	1.00000	1.00000	Constant	1.00000	1.00000	1.00000	Constant
2	Belgium	Region IV	0.81211	0.78241	0.83656	Increasing	1.25551	1.27810	1.06920	Decreasing
3	Bulgaria	Region II	1.00000	1.00000	1.00000	Constant	1.00000	1.00000	1.00000	Constant
4	Croatia	Region II	1.00000	1.00000	1.00000	Constant	1.00000	1.00000	1.00000	Constant
5	Cyprus	Region II	1.00000	1.00000	1.00000	Constant	1.00000	1.00000	1.00000	Constant
6	Czechia	Region I	0.90842	0.72513	0.71529	Increasing	1.36396	1.37906	0.98643	Increasing
7	Denmark	Region II	1.00000	1.00000	1.00000	Constant	1.00000	1.00000	1.00000	Constant
8	Estonia	Region II	1.00000	1.00000	1.00000	Constant	1.00000	1.00000	1.00000	Constant
9	Finland	Region II	1.00000	1.00000	1.00000	Constant	1.00000	1.00000	1.00000	Constant
10	France	Region II	1.00000	1.00000	1.00000	Constant	1.00000	1.00000	1.00000	Constant
11	Germany	Region I	0.97097	0.96342	0.92579	Increasing	1.03353	1.03797	0.96094	Increasing
12	Greece	Region II	1.00000	1.00000	1.00000	Constant	1.00000	1.00000	1.00000	Constant
13	Hungary	Region IV	0.89122	0.81296	0.87788	Increasing	1.22473	1.23008	1.07987	Decreasing
14	Ireland	Region II	1.00000	1.00000	1.00000	Constant	1.00000	1.00000	1.00000	Constant
15	Italy	Region II	1.00000	1.00000	1.00000	Constant	1.00000	1.00000	1.00000	Constant
16	Latvia	Region II	1.00000	1.00000	1.00000	Constant	1.00000	1.00000	1.00000	Constant
17	Lithuania	Region II	1.00000	1.00000	1.00000	Constant	1.00000	1.00000	1.00000	Constant
18	Luxembourg	Region II	1.00000	1.00000	1.00000	Constant	1.00000	1.00000	1.00000	Constant
19	Malta	Region II	1.00000	1.00000	1.00000	Constant	1.00000	1.00000	1.00000	Constant
20	Netherlands	Region II	1.00000	1.00000	1.00000	Constant	1.00000	1.00000	1.00000	Constant
21	Poland	Region II	1.00000	1.00000	1.00000	Constant	1.00000	1.00000	1.00000	Constant
22	Portugal	Region II	1.00000	1.00000	1.00000	Constant	1.00000	1.00000	1.00000	Constant
23	Romania	Region II	1.00000	1.00000	1.00000	Constant	1.00000	1.00000	1.00000	Constant
24	Slovakia	Region II	1.00000	1.00000	1.00000	Constant	1.00000	1.00000	1.00000	Constant
25	Slovenia	Region II	1.00000	1.00000	1.00000	Constant	1.00000	1.00000	1.00000	Constant
26	Spain	Region II	1.00000	1.00000	1.00000	Constant	1.00000	1.00000	1.00000	Constant
27	Sweden	Region III	0.78378	0.70707	1.19329	Decreasing	1.09092	1.41428	1.68766	Decreasing

Source: authors’ own processing in DEAFrontier.

present the results of the CRS and VRS models. The efficiency of the EU regions was calculated using the DEAFrontier software program.

Therefore, based on the DEA analysis, the efficiency value for 2022 was also calculated (Table 4), and the European Union countries were grouped into two regions based on efficiency: increasing, constant, decreasing. No country is integrated into the first efficiency region (increasing). The analysis conducted for 2022 does not reveal any countries exhibiting increasing returns to scale. While the Data Envelopment Analysis method assesses relative efficiency based on the input and output variables considered, the overall socio-economic landscape of the European Union during this period must be taken into account. Region II (with constant returns) includes Austria, Belgium, Bulgaria, Croatia, Cyprus, the Czech Republic, Denmark, Estonia, Finland, France, Greece, Hungary, Ireland, Italy, Lithuania, Latvia, Luxembourg, Malta, Poland, Portugal, Romania, Slovenia, Slovakia, and Spain. Region III includes three countries Germany, the Netherlands, and Sweden (decreasing). The DEA analysis revealed efficiency differences in the provision of digital public services among EU member states. Many countries have maintained their efficiency, while others have room for improvement. Developed nations demonstrate higher efficiency in digital public service delivery than less developed European Union member states.

In 2017 (Table 5), the member states of the European Union were divided into four distinct regions based on efficiency. The efficient states belong to Region I (increasing) include Germany and the Czech Republic, Region II (constant) includes Austria, Bulgaria, Croatia, Cyprus, Denmark, Estonia, Finland, France, Greece, Ireland, Italy, Latvia, Lithuania, Luxembourg, Malta, Netherlands, Poland, Portugal, Romania, Slovakia, Slovenia, and Spain, Region III (decreasing) includes Sweden, and the last region, Region IV, includes the most sub-efficient countries, Belgium and Hungary. Following the analysis, at the 2017 level there are four regions, so the analysis showed the difference between growing, steady, and decreasing profitability at scale, as well as a region with the most sub-efficient countries, so this analysis offers a new way of understanding digital efficiency across the EU.

Many countries, especially those in Region II, exhibit consistent efficiency scores in input-oriented and output-oriented models and VRS and CRS assumptions. This suggests that these countries achieved optimal performance in providing digital public services in 2022. Regarding the potential for improvement through scale expansion, Germany, the Netherlands, and Sweden recorded efficiency scores of less than 1, indicating the potential for improvement through scale adjustments. At the same time, a significant number of countries positioned in Region II maintained their efficiency scores, suggesting a high level of performance in the provision of digital public services.

In 2017, most countries demonstrated efficiency in both Constant Returns to Scale and Variable Returns to Scale, regardless of whether the orientation was input- or output-oriented. This suggests a generally high-efficiency level across the entire EU in 2017. While many countries have maintained efficiency for 2022 as well, there has been a decline in the number of countries considered efficient, particularly at the CRS level. Several countries exhibit a different RTS, indicating potential inefficiencies related to the scale of operations. Therefore, many countries have recorded significant changes in RTS between 2017 and 2022. For example, Germany, the Netherlands, and Sweden have transitioned from CRS in 2017 to Decreasing Returns to Scale in 2022. This implies that these countries may have exceeded the optimal scale for digital public services. On the other hand, Hungary and Belgium have optimized their Increasing Returns to Scale, suggesting efficiency in the operationalization of their digitalization initiatives.

Region II consistently encompasses the most significant number of efficient countries in both 2017 and 2022. However, the observed changes in efficiency and RTS in this region highlight the importance of examining each country’s performance individually rather than relying solely on regional classifications. Countries such as France, Bulgaria, Croatia, Cyprus, Estonia, Finland, France, Greece, Hungary, Ireland, Italy, Latvia, Lithuania, Luxembourg, Malta, the Netherlands, Poland, Portugal, Romania, and Slovenia maintained constant returns to scale and optimal efficiency in both years. At the same time, the Czech Republic, Germany, and Sweden present potential for improvement through the expansion of their digitalization efforts.

In the context of the COVID-19 pandemic, the public system has recorded new discrepancies at the level of European countries. These were caused, on the one hand, by the dynamic changes in digital technology, its adoption and integration, and the various software required to provide digital public services [45]. On the other hand, these discrepancies between EU countries were contributed to by the limited digital skills of professionals and the population, as well as the lack of knowledge, the low credibility of the information transmitted, and the low level of digital literacy [46,47]. In this situation, the role of the European Union has become even more crucial in providing policies, strategies, and financial support to address the gaps between the member states, ensuring the uniform development of the digital public administration sector [48]. Continuing the analysis, the final part includes the dendrogram along with the results of the estimates. These new research settings have allowed us to observe the impact of digital integration on e-government and measure public service efficiency by illustrating how different digitalization and e-government indicators affect EU member states and re-evaluating our final hypothesis (H2). This method groups similar countries based on their digital performance indicators, creating a tree-like structure where the proximity of branches reflects the similarity between countries.

A dendrogram is used to determine the hierarchical relationships between countries and clusters. The height indicates the order in which the clusters were merged. The lowest height indicates the strongest link, i.e., the closest similarity between countries, while the highest height indicates the most distinct values. The dendrograms in Figure 5 use a scale from 0 to 8 to establish the link height. We can observe the most similar countries in terms of public sector performance level in the 2017 (Figure 5a) dendrogram. The blue cluster includes Romania, Bulgaria, and Greece. The yellow cluster exhibits a medium level of similarity and intermediate performance, encompassing countries such as Malta, Luxembourg, Denmark, Austria, Estonia, the Netherlands, Finland, and Sweden; the red cluster includes France, Germany, Spain, Italy, Hungary, Poland, Slovenia, Belgium, Croatia, Czechia, Slovakia, Latvia, Portugal, and Cyprus. The most performant countries in 2017 are those in the red cluster, while the least performant countries are included in the blue cluster.

Regarding the countries with similar levels of public sector performance for 2022 (Figure 5b), we identify two clusters: in the green cluster—France, Germany, Austria, Cyprus, Czechia, Slovenia, and Italy, Estonia and Malta, Cyprus and Greece, Poland and Hungary, Spain, Denmark, Netherlands, Finland, Sweden, Belgium, Hungary, Slovakia, Estonia, Latvia, Luxembourg, Malta, Greece, Hungary, Lithuania, and Portugal; and in the blue cluster—Greece, Bulgaria, and Romania.

The analysis reveals the highest performance levels are in developed countries with stable public administration [49,50,51]. In contrast, the lowest values are observed in developing countries characterized by instability in their state institutions. Therefore, the result underscores the importance of stable governance and robust public administration, which decision makers should firmly consider when evaluating and implementing policy measures that impact citizens [52,53].

The present country’s clustering analysis using the dendrogram reveals significant differences in the digital performance of the public sector. The trends in digitizing public services in recent years show a steady improvement, which can be attributed to the strategies and measures imposed by the European Union, such as the digital governance framework, structural reforms, and technical support [54]. These initiatives can contribute to efficiently allocating public funds and improving services offered to citizens [55]. At the same time, the European Union continuously encourages and supports member states in their efforts to strengthen this critical sector, which is essential for the proper functioning of society [56,57]. Thus, the recently EU-acceded countries such as Romania, Bulgaria, Croatia, and Slovakia have a public sector performance level that is close but far from the performance level of countries such as Denmark, Sweden, and Finland, which are included in the cluster considered the most performant [58,59]. The most performant countries at the level of 2022 are those in the red cluster, while the least performant countries are included in the blue cluster.

According to the objective of the group analysis, the countries included in the same group have similar levels of digital performance. The countries included on the same line of the dendrogram are the countries closest in the value of the indicators. Based on these results, our hypothesis 2, H2, is validated. Comparing the hierarchical clustering results for both years would provide additional insights into how countries are grouped based on digital performance and how these groupings evolve. This can help identify best practice benchmarks and inform policy recommendations.

Based on the dendrogram, we can assert that there is significant diversity in the digital performance of the public sector among EU member states. However, recent trends suggest a steady improvement in digital technology. This can be attributed to measures imposed by the EU, such as the economic governance framework, administrative reform, promotion of transparency and accountability, and social initiatives like the development of digital skills and literacy programs, as well as the exchange of best practices among member states [60]. These initiatives have contributed to increased efficiency and accountability in managing public funds and enhancing public services offered to citizens. The EU continues to encourage and support member states in their efforts to improve this vital sector for the proper functioning of society, aligning these efforts with the Sustainable Development Goals (SDGs), particularly Goal 9 (Industry, Innovation and Infrastructure) and Goal 16 (Peace, Justice and Strong Institutions). By promoting robust digital public services and strengthening institutional capacities, the EU directly contributes to the development of a resilient and inclusive infrastructure, each of which are considered essential aspects for sustainable development. The exchange of best practices between EU member states supports international collaboration and mutual exchange of knowledge, necessary for implementing SDGs on a large scale. This collaboration is necessary to identify best practice models required for disseminating effective strategies and specifically contributes significantly to accelerating progress towards sustainable development while reinforcing the digital and institutional pillars emphasized by Goals 9 and 16.

This study also highlights the transformative potential of digitalization in improving the performance of the public sector in the European Union by examining the interaction between e-government, digital infrastructure, the digital skills of the population, and various factors influencing the efficiency of public service delivery. Therefore, the research provides a comprehensive understanding of the challenges and opportunities associated with this complex process.

Although the findings prioritize the importance of multiple approaches to e-governance, which could integrate through the prism of its complex processes a wide range of determinants to reflect the current reality of the issue of public service delivery, the emphasis on digitalization in the provision of public services also directly contributes to the achievement of the Sustainable Development Goals. The correlation between the digital performance of the public sector in the EU and the SDGs, especially those related to innovation, governance, and quality public services, collectively favor stronger institutions and better governance. Moreover, the digitalization of processes can enhance the efficiency of the public sector, contributing to a stable economic environment through optimizing resource allocation and reducing administrative burden.

5. Conclusions

This study examined the digital performance of European Union regions, offering a perspective on the digitalization process and its effects on the European public system, with emphasis on opportunities and challenges in adopting digital technologies in public service delivery. Unlike studies focusing on specific aspects of public sector performance, this research integrates a broader range of determinants, providing a nuanced understanding of the multifaceted nature of e-governance.

The theoretical foundation of the current knowledge in this field has facilitated the identification of key instruments for analyzing public sector performance. Furthermore, the most significant methods and techniques for measuring public sector performance have been identified. Relevant studies highlight non-parametric and parametric analysis methods. Consequently, various indices for measuring performance and efficiency are based on opportunity indicators. Composite indicators are among the most widely used instruments in these types of analyses. The preference for composite indicators can be attributed to their multi-complex nature, which allows for a comprehensive analysis of public sector performance that extends beyond traditional economic, social, and financial indicators. Therefore, the need to improve the methods of measuring the performance of the public sector by incorporating new sub-sectors, such as digitalization and innovation, is evident.

Initially, this study analyzed the relationship between the public sector and digitalization within the European Union, employing Gaussian graphical models to evaluate the connections between these domains. The empirical results indicate that digitalization has both positive and negative effects on public services in the European Union. These findings provide valuable insights into the complex relationship between digitalization and e-government across the EU-27, enhancing the understanding of its multifaceted impact on public sector. The Gaussian graphical analyses effectively visualize the intricate connections between variables, indicating that recent improvements in digital infrastructure have facilitated a broader integration of digital networks within public administration. A comparison of the GGM results for EU member countries over the period 2017–2022 reveals that digitalization and e-government have directly influenced economic performance and that digitalization has a significant impact on e-government. As the centrality index analysis suggests the quality of regulations plays a crucial role in the integration of digitalization, a relationship highlighted by the strong interconnections between variables in the GGM.

In order to assess the level of digital performance of the EU regions, a Data Envelopment Analysis was performed. Ultimately, the member states of the European Union were classified based on their digital performance. The DEA results indicate that several countries, with constant efficiency, including Portugal, Lithuania, Croatia, Slovenia, the Czech Republic, Poland, Cyprus, Latvia, Slovakia, Hungary, Greece, Bulgaria, Italy, and Romania, demonstrate a need for improvements in digitalization. Furthermore, some of these nations exhibit lower investment in digital infrastructure and technology, thereby registering moderate digital efficiency. The expenditures related to digital technology are viewed as additional costs rather than a sustainable investment for the future, and these nations struggle to attract sources of funding. Conversely, countries such as Austria, Belgium, Germany, Sweden, and France could emulate best practices from highly digitalized nations to optimize their digitalization levels. The Netherlands, Finland, and Denmark emerge as the most digitally advanced within the analyzed group.

The cluster analysis, employing the K-means algorithm and Hierarchical K-means dendrogram, reveals significant diversity in public sector performance among EU regions. These trends suggest a general improvement in digital performance in recent years. While a potential factor in the increased use of digital technology may have been the COVID-19 pandemic, which prompted the integration of digital technologies, disparities remain. Specifically, Romania and Bulgaria continue to exhibit lower levels of digitization. This may relate to a perception of digital infrastructure development as a cost rather than a long-term investment.

The European Union’s policy has highlighted the importance of digital solutions. It has emphasized how digital innovations contribute to the optimal allocation of resources, obtaining a competitive advantage, and stimulating the public sector’s digital performance. However, it is necessary to evaluate the effects of digitalization, considering that not all forms of digitalization can lead to improved performance in the public sector.

This research highlights the critical role of digital infrastructure and citizens’ digital skills in the digitalization of public services. It emphasizes the need for effective analytical methods and further investigation into the unexpected negative relationships between e-government and digitalization variables in the European context.