Are ERDFs Devoted to Boosting ICTs in SMEs Inefficient? A Three-Stage SBM Approach

Abstract

We assessed the implementation of operational programs (OPs) aimed at boosting the deployment of information and communication technologies (ICTs) in small and medium-sized enterprises (SMEs). We performed a three-stage slack-based measure (SBM) data envelopment analysis (DEA) model combined with the stochastic frontier analysis (SFA), which considered data and contextual factors reported from the European Union (EU) to appraise 51 OPs from 16 countries. Overall, we discovered that by eliminating the contextual factors, almost 27% of the OPs (14) attained efficient procedural results. The OP “Multi-regional Spain—ERDF” is widely perceived as a benchmark, irrespective of its contextual factors, remaining robustly efficient for data perturbations ranging from 5% to 10%. The “Number of Operations Supported” is the indicator that requires attention, both with or without the removal of contextual factors. Our findings suggest that more developed regions, with a greater proportion of ICT professionals, are associated with a poor utilization and allocation of ERDF funds to promote ICT adoption in SMEs. This could be attributed to an inability of SMEs to handle the complex bureaucratic processes of submitting and executing European Regional Development Fund (ERDF) initiatives. Consequently, it is vital to provide additional assistance that streamlines the management formalities and satisfies the needs of SMEs.

1. Introduction

Information and communication technologies (ICTs) may help businesses adjust to world trade conditions by cutting border operational costs and boosting access to novel, valuable resources [1,2,3]. Furthermore, it is well-known that ICTs may have a significant beneficial influence on profitability [4,5,6]. The influence of ICTs on establishing connections, both internal and external, is also vital to the success of SMEs in their innovation activities [7,8]. It has been demonstrated that adopting broadband Internet also has a positive impact on the innovation capabilities of SMEs [9]. Some researchers have determined that ICTs may assist small businesses in improving their production, efficiency, and performance [10,11,12,13]. Furthermore, various studies have demonstrated that the beginning of the COVID-19 pandemic resulted in the rapidly increased use of digital technology, particularly for SMEs [14,15].

Despite the benefits and opportunities that ICTs and digital technologies may offer, as well as the fast development of their adoption in recent years, SMEs have not yet exploited their full capabilities [16,17]. New technologies, notably digital technologies and ICTs, continue to pose challenges for companies [18,19,20]. This may in part be due to the fact that SMEs have limited resources, technology, and expertise. In practice, there are various barriers that limit the use of ICTs by SMEs [21]: financial, as substantial expenditure is necessary and funding is difficult to receive; infrastructure, due to electricity prices, broadband, and reliable Internet access; organizational, mainly related to the dearth of qualified staff; and technical, as technological development requires suitable planning. Another problem hindering the use of ICTs is the widespread misunderstanding of the possibilities and consequences of digital transformation [22,23]. On the one hand, SMEs fear the loss of competitiveness, productivity, and profitability if they do not undergo digital transformation [24,25]. On the other hand, managers tend to dismiss digital initiatives, as they are unsure of how to incorporate them into the organizations they lead [22,26].

To promote ICT adoption by SMEs, governments must implement policies that close the digital divide (DD), provide free broadband Internet access, and support education on the relevant topics [22,23,26].

Several attempts have been made over the last 10 years to examine the importance of ICT use by SMEs, resulting in the production of a vast corpus of literature reviews. Taylor [27] investigated two conceptual models: the diffusion of innovation theory [28] and the technology, organization, and environment framework [29], to create a comprehensive theoretical foundation for ICT use by SMEs. This holistic model included an overarching ontology that identifies many of the most significant internal and external factors impacting ICT usage by SMEs. Similarly, [30] conducted a literature review on the relationship between ICTs, SMEs, and poverty reduction. This research highlighted the importance of ICT implementation by SMEs and investigated how SMEs could use ICTs to assist in poverty reduction. Differences in the accessibility of ICTs across enterprises were also explored in a survey by [31] on the DD between companies. This evaluation examined the geographical location, company type, and period of the study, as well as the effect and sources of the DD. Other research has examined the primary causes, consequences, and challenges to ICT usage by SMEs [21]. Tarutė and Gatautis [32] investigated the potential effects of ICTs on the profitability of SMEs. Lehner and Sundby [22] focused on the relevance of ICT skills for SMEs, considering several viewpoints. Oberländer et al. [20] recently evaluated digital competence in the workplace. Isensee et al. [33] created a theoretical framework to provide a holistic view of the organization behavior, sustainability, and digitization levels (including interactions between the three) of SMEs; they concluded that the most frequently researched cultural characteristics were strategies, organizational skill sets, administration, and attitudes. Another recent report [34] examined digital innovation in SMEs, observing that it was governed by prior experience, advanced through multiple phases of innovation, and resulted in a continual organizational and corporate improvement. In the same vein, [35] conducted a literature review to assist in the identification of the key challenges and opportunities for SMEs in the context of digitalization and ICT breakthroughs.

None of the prior reviews centered on the examination of ICT policy. As [36] noted, there has been a lack of academic emphasis on how to choose the optimal strategy for finance allocation. Furthermore, there have only been a few studies conducted to assess if the type of financing is distributed in accordance with the most urgent needs of every location [37,38]. Ex post evaluations are commonly used in studies, which appraise the European structural funds that are allocated to ICTs [37,38]. Some research has also provided ex ante assessments of the variables impacting funding allocation among various ICT initiatives [36]. Nevertheless, to date, no studies have been conducted that contrast the application of OPs related to ICT policies with their counterparts during programmatic timeframes, or that highlight the changes that must be implemented to make an inefficient OP more efficient.

As a result, the purpose of this study was to add to the literature by using a methodological approach that enables management authorities (MAs) to evaluate the execution of OPs dedicated to assisting the implementation of ICTs in SMEs using a non-parametric methodology, specifically, the SBM model in conjunction with the SFA model.

Basically, the primary research questions this study sought to answer were as follows:

RQ1: Which factors hinder the efficient use of ERDFs allotted to increase ICT usage in the EU SMEs?

RQ2: Which OPs were widely reported as benchmarks throughout the programming period evaluated?

RQ3: Which OPs show a more resilient performance in the face of variations in the utilized metrics?

RQ4: Which contextual variables have the highest influence on the inefficiency of the OPs?

RQ5: How does efficiency vary when contextual variables are introduced?

This paper has the following structure. Section 2 provides a literature review on ICT policies/strategies among the EU SMEs. Section 3 explains the basic assumptions underlying the suggested methodological approaches to assess the execution of the OPs being evaluated. Section 4 addresses the key rationale for the chosen input and output parameters and contextual factors utilized in our assessment, as well as some descriptive statistics on the data used in the SBM and SFA models. Section 5 discusses the primary findings in depth. Section 6 summarizes the significant findings, discusses the potential political consequences, highlights the key shortcomings, and proposes future research directions.

2. Literature Review

Successful strategies and suitable policies regarding the adoption of ICTs by SMEs require an understanding of the literature dedicated to this topic. In this context, starting with the Scopus and OpenAlex databases (using Harzing’s Publish or Perish software, version 8, released by David Adams [39]), we began our literature search by considering the combinations of title words, such as “ICT”/“digital transformation”, “SMEs”/“firms”, and “policy” for articles published in the years selected by default by the software. Subsequently, we refined our search by only considering publications that were available online, in English, and devoted to Europe or European countries. The findings of this search are given in Table 1.

From the obtained results, it can be concluded that the literature on this topic is scarce. Only a few studies have addressed the successful development and use of ICTs by SMEs through the establishment of programs and policies by the EU, national, and regional governments. In this context, [40] addressed several initiatives undertaken by the European Commission since the Lisbon summit of March 2000, concluding that the European regional policies have changed from simply getting SMEs connected to the Internet to the effective integration of ICTs into business processes. Recently, [41] also examined the lessons that the EU learned with respect to their experience in fostering digital adoption by SMEs, namely in terms of strengthening the ability for digital transformation, sustainable growth, lowering regulatory burdens, expanding market access, developing finance channels, and lowering the difficulty and cost of financing. Along the same lines but considering a national context, [42] studied Spain as a point of comparison. Later, [43] explored the first public initiatives in Spain to help SMEs and micro-SMEs use cloud computing services, which also aimed to support SMEs in their digital transformation, promote e-commerce, and boost competitiveness. Skoko et al. [44] proposed an ICT adoption model for Australian and Croatian SMEs, which was based on the concept that SMEs are the primary economic development force in all advanced economies. Similarly, [45] presented the findings of an ex post assessment of a national ICT program devoted to SMEs in the Netherlands that took place from 2002 to 2007. Later, [46] evaluated the government program to boost SMEs’ inter-organizational ICTs, proposing simple, awareness-focused policy programs. More recently, [47] studied the influence of digitization on demand-side policies that encourage SMEs in Wales to embrace broadband and digital technologies.

Alternatively, other published works addressed specific activity sectors. With respect to this framework, [48] reviewed and compiled an extremely diverse set of literature on the use of ICTs in rural SMEs, offering an overview of the generic policy concerns. In a different sector, [49] sought to examine if horizontal, general purpose direct support mechanisms at the national level and financial support measures at the local level allowed for the effective deployment of public funds; they had a particular focus on the Italian ICT services firms. More recently, [50] assessed legislative measures performed in Greece to facilitate the national digital transformation of Greek tourist SMEs.

Other studies addressed specific policy instruments, such as in [51], which aimed to provide national/regional decision-makers and/or existing digital innovation hub (DIH) managers with valuable and organized details regarding how to install a new DIH, or how to strengthen current ones while receiving funding, with a special focus on the ERDF of 2021–2027.

Finally, [52] discussed the main hurdles that governments face in encouraging SMEs to benefit from digital transformation, and put forth important policy recommendations, such as: (1) stimulating SMEs so that they adopt digital technologies, (2) assisting SMEs in training and developing the appropriate skills, (3) improving management skills in SMEs, and (4) utilizing financial technology (Fintech) and innovative sources of funding for SMEs.

From the literature review conducted, we established that the strategy of ICT adoption by businesses was influenced by and was a product of the regional economic dynamics. Therefore, the assessment of environmental factors through the SFA model proposed herein is timely and relevant. Additionally, it can be ascertained that the studies available usually conducted ex post assessments of ICT policy interventions, with a dearth of scholarly attention to the evaluation of the ICT policy during its midterm/terminal uptake. Hence, this study aimed to fill this gap in the literature by using an overarching methodological approach, one that allowed for the performance of a midterm/terminal assessment of the execution of OPs dedicated to assisting SMEs in the adoption of ICTs using a non-parametric methodology, specifically the SBM model, in conjunction with the SFA model. With this approach, it is possible to detect whether the inefficiency of these OPs is the result of management failures or contextual factors.

Table 1. Studies dedicated to study ICTs/digital policies in SMEs.

Authors	Main Topics Addressed
Cuadrado-Roura and Garcia-Tabuenca [42]	Analyzed EU programs and policies aimed at the successful use of ICT in SMEs, focusing on Spain
Santinha and Soares [40]	Analyzed several initiatives undertaken by the European Commission since the Lisbon summit of March 2000, particularly European regional policies related to the effective integration of ICTs into business processes
Galloway and Mochrie [48]	Aimed to compile the extremely diverse literature on the use of ICTs in rural SMEs to offer an overview of the generic policy concerns
Skoko et al. [44]	Suggested an ICT adoption model in Australian and Croatian SMEs
Colombo and Grilli [49]	Evaluated if both horizontally general purpose direct support mechanisms at the national and local levels allowed for the effective deployment of public funds, with a focus on the Italian ICT services industry
Plomp et al. [45]	Ex post assessment of the Netherlands ICT policy program for SMEs that took place between 2002 and 2007
Plomp et al. [46]	Proposed simple, awareness-focused policy programs rather than extensive, government-supported initiatives in the Netherlands
Calle et al. [43]	Assessment of the first public initiatives to help Spanish SMEs in their digital transformation
Kalpaka et al. [51]	Proposed digital innovation hubs as a policy instrument to boost the digitalization of SMEs, focusing on the ERDF of 2021–2027
Henderson [47]	Investigated the influence of digitization on demand-side policies that encourage SMEs to embrace broadband and digital technologies
Kergroach [52]	Appraisal of the government challenges in fostering digital transformation in OECD SMEs
Dionysopoulou and Tsakopoulou [50]	Looked at the ongoing policy initiatives in Greece to support the digital transformation of Greek tourism SMEs on a national level
Dong and Meng [41]	Observed the lessons learned from the EU’s experience in assisting the digital transformation of SMEs as a benchmark for Chinese SMEs

Table 1. Studies dedicated to study ICTs/digital policies in SMEs.

Authors	Main Topics Addressed
Cuadrado-Roura and Garcia-Tabuenca [42]	Analyzed EU programs and policies aimed at the successful use of ICT in SMEs, focusing on Spain
Santinha and Soares [40]	Analyzed several initiatives undertaken by the European Commission since the Lisbon summit of March 2000, particularly European regional policies related to the effective integration of ICTs into business processes
Galloway and Mochrie [48]	Aimed to compile the extremely diverse literature on the use of ICTs in rural SMEs to offer an overview of the generic policy concerns
Skoko et al. [44]	Suggested an ICT adoption model in Australian and Croatian SMEs
Colombo and Grilli [49]	Evaluated if both horizontally general purpose direct support mechanisms at the national and local levels allowed for the effective deployment of public funds, with a focus on the Italian ICT services industry
Plomp et al. [45]	Ex post assessment of the Netherlands ICT policy program for SMEs that took place between 2002 and 2007
Plomp et al. [46]	Proposed simple, awareness-focused policy programs rather than extensive, government-supported initiatives in the Netherlands
Calle et al. [43]	Assessment of the first public initiatives to help Spanish SMEs in their digital transformation
Kalpaka et al. [51]	Proposed digital innovation hubs as a policy instrument to boost the digitalization of SMEs, focusing on the ERDF of 2021–2027
Henderson [47]	Investigated the influence of digitization on demand-side policies that encourage SMEs to embrace broadband and digital technologies
Kergroach [52]	Appraisal of the government challenges in fostering digital transformation in OECD SMEs
Dionysopoulou and Tsakopoulou [50]	Looked at the ongoing policy initiatives in Greece to support the digital transformation of Greek tourism SMEs on a national level
Dong and Meng [41]	Observed the lessons learned from the EU’s experience in assisting the digital transformation of SMEs as a benchmark for Chinese SMEs

3. Methodology

Conventional DEA approaches, such as the Charnes–Cooper–Rhodes model, or CCR model [53], and the Banker–Charnes–Cooper model, or BCC [54], are radial, i.e., they only handle the proportionate changes in the inputs (resources) or outputs (outcomes) utilized in the efficiency evaluation. As a result, the CCR and BCC efficiency scores generated represent the greatest possible proportional input (or output) retraction (or growth) rate for all inputs (or outputs). In practice, however, this type of assumption is often not attainable due to factor substitution.

Therefore, in contrast to the CCR and BCC models, we used the SBM model [55], which allows for a more detailed examination of efficiency as it is non-radial (i.e., inputs and outputs can adjust non-proportionally) and non-oriented (i.e., it allows for simultaneous changes in the inputs and outputs).

Nevertheless, one of the disadvantages of the DEA technique is that it does not take into consideration the influence of contextual variables and random errors in the efficiency evaluation. As a result, [56] suggested a three-stage DEA model. First, the SBM model is used to generate the efficiency scores of each decision-making unit (DMU), i.e., the units under evaluation (the OPs in this study), as well as the required adjustments on the input and output factors to turn inefficient DMUs into efficient ones (i.e., the slacks). Second, the slacks are broken down into three categories: contextual factors, management inefficiency, and statistical noise. The slacks are the dependent variables, whereas the independent variables are the contextual factors. The goal is to eliminate the impact of contextual variables and random errors. Subsequently, the input and output factors are modified using the SFA model [57,58]. Finally, at the third stage, the efficiency scores are computed once more with the modified input and output factors.

3.1. First Stage: Computing Efficiencies for Every DMU with Original Inputs and Outputs through the SBM Model

Based on the three-stage method of [56], in the first stage, the slacks are computed through the SBM model. The SBM model is given by [55]:

$$\begin{gathered} \begin{array}{c}Min\\ \mathbf{\lambda },{\mathit{s}}^{-}, {\mathit{s}}^{+}\end{array} \mathsf{\rho }=\frac{1-\frac{1}{m}{\displaystyle {{\displaystyle \sum }}_{i=1}^m}{s}_i^{-}/x_{ik}}{1+\frac{1}{s}{\displaystyle {{\displaystyle \sum }}_{r=1}^s}{s}_r^{+}/y_{rk}} \\ \mathrm{s}.\mathrm{t}. \\ {x}_{ik}={{\displaystyle \sum }}_{j=1}^n{x}_{ij}{}_j+s_i^{-}, i=1, \dots , m \\ {y}_{rk}={{\displaystyle \sum }}_{j=1}^n{y}_{rj}{}_j-s_i^{+}, r=1, \dots , s \\ {\mathit{\lambda }}_j\ge 0, j=1, \dots , n, \\ s_i^{-}\ge 0, i=1, \dots , m, \\ s_i^{+}\ge 0, r=1, \dots , s \end{gathered}$$

(1)

where we consider a set of $n$ DMUs ($DM{U}_1,DM{U}_2,\dots , DM{U}_n$), with X = [x_ij, i = 1, 2, …, m, j = 1, 2, …, n] being the matrix of inputs (m × n), Y = [y_rj, r = 1, 2, …, s, j = 1, 2, …, n] being the vector of outputs (s × n), and the rows of the matrices for DMU_k are ${\mathit{x}}_k^T$ and ${\mathit{y}}_k^T$, respectively, with ^T representing the transpose of a vector.

The value of 0 < ρ < 1 can be seen as the ratio of average inefficiencies of inputs and outputs.

Model (1) can be transformed into Model (2), by using a positive scalar variable t:

$$\begin{gathered} \begin{array}{c}\mathit{min}\\ t, \mathbf{\lambda }, {\mathit{s}}^{-}, {\mathit{s}}^{+}\end{array}\tau =t-\frac{1}{m}{\displaystyle \sum }_{i=1}^{m}t{s}_i^{-}/x_{ik} \\ \mathrm{s}.\mathrm{t}. t+\frac{1}{s}{{\displaystyle \sum }}_{r=1}^{s}t{s}_r^{+}/y_{rk}=1, \\ {\mathit{x}}_k=X\mathit{\lambda } +{\mathit{s}}_{}^{-}, \\ {\mathit{y}}_k=Y\lambda -{\mathit{s}}_{}^{+}, \\ \mathbf{\lambda }\ge \mathbf{0}, {\mathit{s}}_{}^{-}\ge \mathbf{0}, {\mathit{s}}_{}^{+}\ge \mathbf{0}, t>0. \end{gathered}$$

(2)

Let ${\mathit{S}}_{}^{-}$= $t{\mathit{s}}_{}^{-}$, ${\mathit{S}}_{}^{+}$= $t{\mathit{s}}_{}^{+}$ and $\mathbf{Ʌ}$ = $t\mathbf{\lambda }$. Next, Model (2) turns into:

$$\begin{gathered} \begin{array}{c}Min\\ t,\mathbf{\lambda },{\mathit{s}}^{-}, {\mathit{s}}^{+}\end{array}\tau =t-\frac{1}{m}{{\displaystyle \sum }}_{i=1}^m{S}_i^{-}/x_{ik} \\ \mathrm{s}.\mathrm{t}. t+\frac{1}{s}{{\displaystyle \sum }}_{r=1}^s{S}_r^{+}/y_{rk}=1, \\ t{\mathit{x}}_k =X\mathbf{Ʌ} +{\mathit{S}}_{}^{-}, \\ t{\mathit{y}}_k=Y\mathbf{Ʌ} -{\mathit{S}}_{}^{+}, \\ \mathbf{Ʌ}\ge \mathbf{0}, {\mathit{s}}_{}^{-}\ge \mathbf{0}, {\mathit{s}}_{}^{+}\ge \mathbf{0}, t>0. \end{gathered}$$

(3)

The optimal solution corresponds to:

ρ* = τ*, $\lambda$* = $\mathbf{Ʌ}$*/t*, $s_{}^{-*}$ = $S_{}^{-}$/t*, $s_{}^{+*}$ = $S_{}^{+}$/t*.

Definition 1.

A DMU_k is efficient if${\rho }^{ *}=1$, meaning that${\mathit{s}}_{}^{-*}$ = 0 and${\mathit{s}}_{}^{+*} = \mathit{0}$.

Definition 2.

The set of benchmark DMUs for each inefficient DMU_k is E_k = {j:${\lambda }_j^{*}>0$, j = 1, …, n}.

Definition 3.

The reference point for each inefficient DMU_k is:

$$({\widehat{\mathit{x}}}_k, {\widehat{\mathit{y}}}_k)=({\mathit{x}}_k-{\mathit{s}}_{}^{-*}, {\mathit{y}}_k+{\mathit{s}}_{}^{+*})=({{\displaystyle \sum }}_{jϵE_k}^{}{\lambda }_j^{*}{\mathit{x}}_{\mathit{j}},{{\displaystyle \sum }}_{jϵE_k}^{}{\lambda }_j^{*}{\mathit{y}}_j).$$

(4)

We have also used the Super-SBM non-oriented model proposed by [59], which evaluates the efficiency of a DMU, considering the closest efficient point on the frontier excluding itself. The optimal value of the objective function of this latter model is always greater than or equal to 1. Nevertheless, since the super-efficiency for DMU_k can be 1, even if the DMU is inefficient, to check if a DMU_k is inefficient or not, both Model (1) and Tone’s model must be solved. If DMU_k is efficient in accordance with Model (1), Tone’s model should be applied to compute its Super-SBM non-oriented efficiency score.

3.2. Second Stage: Obtaining the Adjusted Input and Output Factors for Inefficient DMUs Using the SFA Model

In the second stage, the input and output variables of each DMU are modified according to the SFA results by eliminating the significant contextual effects and statistical noises.

Each input slack is obtained for j inefficient DMU ($j=1,\dots ,p$) through:

$$s_{ij}=f\left(Z_j, {\beta }^i\right)+v_{ij}+u_{ij} ,i=1,\dots ,m; j=1,\dots ,p$$

(5)

where $s_{ij}$ is the slack value of input i of DMUj, $f\left(Z_j, {\beta }^i\right)$ is the deterministic feasible slack frontier, and ${\beta }^i$ denotes the coefficients associated with the contextual factors. The term $v_{ij}+u_{ij}$ is the mixed error, $v_{ij}$ relates to the statistical noise, and $u_{ij}$ relates to the management inefficiency. Usually, it is assumed that $v_{ij}~N\left(0; {\sigma }_v^2\right)$ and $u_{ij}~N^{+}\left({\mu }^i;{\sigma }_u^2\right),$ with $v_{ij}$ and $u_{ij}$ corresponding to the independent variables.

Let $\gamma =\frac{{\sigma }_u^2}{{\sigma }_u^2+{\sigma }_v^2}$. If $\gamma$ is near 1, it means that the management factors are in a leading position, i.e., most of the change required to attain efficiency is linked to the management inefficiency. If $\gamma$is near 0, the random error is the predominant factor, i.e., most of the change required to attain efficiency is related to the statistical noise.

The adjusted input and output slacks are then computed by decomposing the mixed error. In line with [60], the management inefficiency is computed as follows:

$$E\left(u_{ij}|u_{ij}+v_{ij} \right)=\frac{\sigma \delta }{1+{\delta }^2}\left[\frac{\phi \left(\frac{{\epsilon }_j\delta }{\sigma }\right)}{\varnothing \left(\frac{{\epsilon }_j\delta }{\sigma }\right)}+\frac{{\epsilon }_j\delta }{\sigma }\right]$$

(6)

where $\delta =\frac{{\sigma }_u}{{\sigma }_v}, {\epsilon }_j=v_{ij}+u_{ij,}, {\sigma }^2={\sigma }_u^2+{\sigma }_v^2, \phi$and $\varnothing$ are, respectively, the density and distribution functions of the standard normal distribution. Therefore, the random error term can be obtained as:

$$E\left(v_{ij}|u_{ij}+v_{ij} \right)=s_{ij}-f\left(Z_j, {\beta }^i\right)-E\left(u_{ij}|u_{ij}+v_{ij} \right)$$

(7)

The input data are then modified by considering [56]:

$$x_{ij}^A=x_{ij}+\left[\underset{i}{\max }\left\{f\left(Z_j, {\beta }^i\right)\right\}-f\left(Z_j, {\beta }^i\right)\right]+\left[\underset{j}{\max }\left\{v_{ij}\right\}-v_{ij}\right]$$

(8)

By following an analogous process, the adjusted outputs are obtained as follows [61]:

$$y_{rj}^A=y_{rj}+\left[f\left(Z_j, {\beta }^r\right)-\underset{r}{\min }\left\{f\left(Z_j, {\beta }^r\right)\right\}\right]+\left[v_{rj}-\underset{r}{\min }\left\{v_{rj}\right\}\right]$$

(9)

3.3. Third Stage: Computing the Efficiencies of Every DMU with the Adjusted Inputs and Outputs through the SBM Model

Finally, in the third stage, the efficiency scores are computed through the SBM method, with the modified input and output factors as follows:

$$\begin{gathered} \begin{array}{c}Min\\ \mathbf{\lambda },{\mathit{s}}^{-}, {\mathit{s}}^{+}\end{array} \rho =\frac{1-\frac{1}{m}{{\displaystyle \sum }}_{i=1}^m{s}_i^{-}/x_{ik}^A}{1+\frac{1}{s}{{\displaystyle \sum }}_{r=1}^s{s}_r^{+}/y_{rk}^A} \\ \mathrm{s}.\mathrm{t}. \\ {x}_{ik}^A ={{\displaystyle \sum }}_{j=1}^n{x}_{ij}^A{\lambda }_j +s_i^{-}, i=1, \dots , m \\ {y}_{rk}^A ={{\displaystyle \sum }}_{j=1}^n{y}_{rj}^A{\lambda }_j -s_i^{+}, r=1, \dots , s \\ {\lambda }_j\ge 0, j=1, \dots , n, \\ s_i^{-}\ge 0, i=1, \dots , m, \\ s_i^{+}\ge 0, r=1, \dots , s \end{gathered}$$

(10)

where $x_{ij}^A$ = $x_{ij}$ if DMU_j (j = 1, …, n) is found to be efficient in the first stage and $x_{ij}^A=x_{ij}+\left[\underset{i}{\max }\left\{f\left(Z_j, {\beta }^i\right)\right\}-f\left(Z_j, {\beta }^i\right)\right]+\left[\underset{j}{\max }\left\{v_{ij}\right\}-v_{ij}\right]$ if otherwise. Similarly, $y_{rj}^A=y_{rj}$ if DMU_j (j = 1, …, n) is found to be efficient in the first stage and $y_{rj}^A=y_{rj}+\left[f\left(Z_j, {\beta }^r\right)-\underset{r}{\min }\left\{f\left(Z_j, {\beta }^r\right)\right\}\right]+\left[v_{rj}-\underset{r}{\min }\left\{v_{rj}\right\}\right]$ if otherwise.

4. Data

The input and output factors chosen for evaluating the execution efficiency of the ERDF engagement in SME ICT usage were drawn from an array of cross-cutting metrics formally required by the EU (Available online: https://cohesiondata.ec.europa.eu/2014-2020-Categorisation/ESIF-2014-2020-categorisation-ERDF-ESF-CF-planned-/3kkx-ekfq (accessed on 30 March 2022)).

Due to the lack of data on ICT usage at the corporate level from conventional datasets per NUTS2 region [36,62,63], we utilized indicators that were accessible from the Regional Innovation Scoreboard 2021 as the contextual factors [64]. These were selected according to the literature on the primary factors influencing ICT usage in EU SMEs. Other statistical data were obtained from OECD statistics (Available online: https://stats.oecd.org/Index.aspx?DataSetCode=REGION_ECONOM (accessed on 30 March 2022)).

4.1. Input and Output Factors

4.1.1. “Total Eligible Costs Decided” and “Total Eligible Spending”

“Total Eligible Spending” and “Total Eligible Costs Decided” were the metrics used to assess the capability of the OPs’ uptakes. The first is about the eligible expenses that have been recorded and validated by a decision body. As a result, this factor was used as an output, since the higher the value allocated to it, the higher the financial execution of each project. The second was considered an input because it pertains to the budget available to the projects chosen for financing, which must be kept to a minimum.

4.1.2. “Number of Operations Supported”

The “Number of Operations Supported” refers to the number of ERDF-funded initiatives. The greater the number of initiatives financed, the larger the possibility of increasing/improving enterprises’ usage of ICTs. As a result, this indicator was regarded as an output.

Table 2 provides the descriptive statistics of the inputs and outputs employed in the efficiency assessment of the OPs.

Table 2. The descriptive statistics of the inputs and outputs.

Statistics	Total Eligible Spending	Number of Operations	Total Eligible Costs Decided
Mean	15,861,300	409	28,169,468
Median	3,238,795	27	5,000,000
Standard Deviation	38,520,025	1068	63,497,428
Minimum	68,486	1	251,294
Maximum	237,904,467	5457	311,154,920
Count	51	51	51

Source: Authors’ own computation. Data available online at: https://cohesiondata.ec.europa.eu/2014-2020-Categorisation/ESIF-2014-2020-categorisation-ERDF-ESF-CF-planned-/3kkx-ekfq (accessed on 30 March 2022).

4.2. Contextual Variables

Regarding independent factors (see Table 2), we used the regional GDP at purchasing power parity per capita (GDPPPPpc) as a proxy for the economic growth [36,62,63]. According to [65], ICTs are more effectively deployed in richer regions.

We also included the percentage of the population aged 25–34 years who have finished university education as a contextual variable, as evidence shows a favorable relationship between education received and ICT use [62,63]. In the case of ICT use, certain explanations have been emphasized to justify this apparent favorable link. On the one hand, education is believed to provide the necessary skills for utilizing and benefiting from ICTs. Employees, on the other hand, have been assumed to be more adept in understanding how to use innovative technology, particularly ICTs [63].

Furthermore, because R&D expenditures promote ICT diffusion in European regions [62,63,66], we utilized R&D spending in the corporate sector as a percentage of GDP and SMEs bringing innovative products as a percentage of all SMEs, corresponding as recorded independent variables.

Furthermore, because a firm’s ICT skills are viewed as an essential technology-related aspect that influences user acceptability and ICT adoption [66], we included employees with above-average digital capabilities as a fraction of the total SME employees. Lastly, the proportion of ICT experts have been utilized as a fraction of total SMEs’ labor; in other words, these are employees whose major vocation is ICTs and can manage a wide range of tasks related to an enterprise’s computer systems [67].

The min-max approach was used to normalize the data, which meant that the minimum score for all locations throughout all the years was removed from the corresponding converted score, which was then divided by the difference between the maximum and minimum values recorded for all data (regions and years). Table 3 depicts the normalized scores that range between 0 and 1, apart from the GDPPPPpc variable, which was measured by an index number.

Table 3. The descriptive statistics of the contextual variables.

Environmental Factors	Mean	Standard Deviation	Min	Max
Population with Tertiary Education	0.5767	0.1916	0.1156	1
Digital Skills	0.5359	0.1949	0.2814	0.9318
R&D Expenditures in the Business Sector	0.3105	0.2101	0.0215	0.8024
ICT Specialists	0.4018	0.2527	0.0470	1
Product Process Innovators	0.5529	0.2511	0.1767	1
GDPPPPpc	87.72	23.9315	49.09	178.30

Source: Authors’ own computation. Data on GDPPPPpc available online at: https://stats.oecd.org/Index.aspx?DataSetCode=REGION_ECONOM (accessed on 30 March 2022). Data for the remaining variables available online at: https://ec.europa.eu/info/research-and-innovation/statistics/performance-indicators/european-innovation-scoreboard_pt (accessed on 30 March 2022).

5. Discussion of Results

The discussion of the results obtained with the three-stage SBM approach are listed below.

5.1. First Stage: Computing the Efficiencies for Every DMU with the Original Inputs and Outputs through the SBM Model

Results were obtained through the Max DEA software, version 8 Ultra (v8.22, R2022/3/21), and their descriptive statistics are depicted in Table 4.

Table 4. The descriptive statistics of the results obtained.

	Statistics	Efficiency	Total Eligible Spending	Number of Operations	Total Eligible Costs Decided
Efficient DMUs	Mean	1.20	46,026,233.00	1310.40	75,514,839.90
	Median	1.16	9,217,730.00	339.50	9,633,113.00
	Standard Deviation	0.23	74,818,282.11	2108.72	118,719,405.31
	Minimum	1.00	329,249.00	1.00	251,294.00
	Maximum	1.71	237,904,467.00	5457.00	311,154,920.00
	Count	10	10	10	10
Inefficient DMUs	Mean	0.20	8,503,999.66	189.54	16,621,815.98
	Median	0.06	1,963,414.00	14.00	4,901,930.00
	Standard Deviation	0.25	17,671,362.04	415.58	34,228,777.84
	Minimum	0.00	68,486.00	1.00	373,794.00
	Maximum	0.96	102,175,668.00	2184.00	202,847,237.00
	Count	41	41	41	41

Source: Authors’ own computation.

As seen from Table 4, the efficient OPs have greater mean scores than inefficient OPs (with efficiency values varying between 1.00 and 1.71 and at least 50% of the efficient OPs having efficiency values greater than 1.16). In addition, the inefficient OPs have a wide variety of efficiency ratings scores, ranging from 0 to 0.96 (with a standard deviation of 0.25 and at least 50% of the inefficient OPs with efficiency values lower than 0.06).

The four OPs chosen as the references of best practices are “Extremadura—ERDF” (27), “Pas Vasco—ERDF” (26), “Provence-Alpes-Côte d’Azur—ERDF/ESF/YEI” (20), and “Multi-regional Spain—ERDF” (14) (see Table 5). Surprisingly, all of these territories belong to countries that are regarded as the largest spenders on ICT assistance for SMEs [68]. Moreover, the results produced by the Spanish regional OP conform with the findings of [67].

Table 5. The characteristics of the efficient OPs.

MS (2 Digit ISO)	OP	Nº of Times as Benchmark	Rank	Total Eligible Spending	Number of Operations	Total Eligible Costs Decided
FR	Provence-Alpes-Côte d’Azur—ERDF/ESF/YEI	20	1	329,249	1	251,294
CZ	Enterprise and Innovation for Competitiveness—CZ—ERDF	1	2	237,904,467	451	311,154,920
ES	Multi-regional Spain—ERDF	14	3	58,864,158	5108	95,971,219
ES	País Vasco—ERDF	26	4	3,964,897	575	4,618,616
ES	Extremadura—ERDF	27	5	1,560,112	810	4,823,735
GR	Competitiveness Entrepreneurship and Innovation—GR—ERDF/ESF	0	6	100,667,978	5457	275,856,182
BG	Innovations and Competitiveness—BG—ERDF	4	7	33,942,154	228	38,612,352
LT	EU Structural Funds Investments—LT—ERDF/ESF/CF/YEI	0	8	7,607,793	210	7,786,656
GR	Epirus—ERDF/ESF	3	9	4,593,855	144	4,593,855
PL	Podkarpackie Voivodeship—ERDF/ESF	0	10	10,827,667	120	11,479,570

Source: Authors’ own computation.

According to these authors, Spanish regions (at the corporate level) have a moderate or higher level of digital development compared with other EU counterparts, as well as a lower DD between the firms (i.e., a lesser discrepancy in firm digitalization) than the other European MSs. In the study published by [67], Greece and Bulgaria were recognized as countries whose companies were positioned last in terms of digital acceptance (per the 2015 data), showcasing the steps taken by these countries in the integration of ICTs by SMEs in the most recent programmatic period.

The SBM model also provides an overview of the modifications that must be made to inputs and outputs to turn inefficient OPs into efficient ones (Table 6).

Table 6. Improvement potential for the OPs.

Factor	Average Original	Average Projection	Variation
Total Eligible Spending	8,504,000	8,716,600	3%
Number of Operations	189.54	783.72	313%
Total Eligible Costs Decided	16,621,816	13,041,314.01	−22%

Source: Authors’ own computation.

The “Number of Operations Supported” had the greatest enhancement potentiality (i.e., it must rise from 189.54 to 782.72 to promote efficiency), while “Eligible Costs Decided” required a moderate reduction (−22%); “Eligible Spending” (2%) only required minor modifications (see Table 6).

5.2. Second Stage: Obtaining the Adjusted Input and Output Factors for Inefficient DMUs Using the SFA Model

The slacks of the outputs derived from Model (2) were employed as the dependent variables, leading to two linear regressions. We evaluated the multicollinearity through the correlation coefficients and variance inflation factors (VIFs). From the correlation analysis, it is possible to conclude that ICT specialists and GDPPPPPC have a strong positive correlation (0.73). Meanwhile, all the remaining associations are moderate correlations, with coefficients ranging from 0.32 to 0.65.

Since multicollinearity occurs when two or more explanatory factors are strongly correlated, the VIF values were obtained as well. The VIF values begin at one and has no maximum bound. Values between one and five suggest a moderate correlation, although we deemed this range to not be significant enough to cause concern [69,70]. The highest VIF value was 3.2690, meaning that all variables could be used in the model. However, due to problems of misspecification, some of them might not appear in the model formulation.

To execute the SFA regression models, R software, version 4.0.5 [71] was used, particularly, the sfaR package version 0.1.1 [72]. The findings thus obtained are shown in Table 7.

Table 7. Results obtained with SFA.

Variables	Slacks
	Total Eligible Spending	Number of Operations
Constant	−242,050 ***	237.20 ***
Population with Tertiary Education	−890,650 ***	-
Digital Skills	−890,970 ***	195.99 ***
ICT Specialists	1,417,700 ***	135.17 ***
Product Process Innovators	-	−73.17 ***
GDPPPPpc	1286 ***	−3.39 ***
Sigma-squared	8.91 × 1011 ***	8.11 × 105 ***
Gamma	0.98 **	0.99 **
Log-likelihood Function	−593.83	−308.67

** and *** Significance at the 5% and 1% levels, respectively. Source: Authors’ own computation.

The values of γ were near one and were statistically significant (1%), showing that management problems were the primary cause of the technical efficiency achieved. To compute unbiased efficiency values, we used the SFA model to eliminate the impacts of contextual variables and random errors. The regression coefficients were all significant (1%), indicating that the specified contextual factors have a considerable influence on the required adjustments computed.

According to Table 6, a rise in both the percentage of ICT experts and GDPPPPpc contributed to a larger necessary increase of “Total Eligible Spending,” but the remaining variables have a detrimental effect on this particular slack. These results demonstrated that richer regions and a larger number of ICT specialists may not always imply a better rate of execution of ERDF funds targeted at strengthening the ICTs in SMEs. In line with these results, [73] recognized the underutilization of ERDFs by Croatian SMEs’ ICTs from 2014 to 2020. The authors of the study proposed that the complexities and time necessary to apply, create, and evaluate the projects were a possible explanation for these outcomes [73]. Furthermore, [74] reported that SMEs’ investments were lower than would be predicted by typical economic models, suggesting that these enterprises are especially sensitive to financing challenges. Another explanation for these results might be that these SMEs utilize other sources of funding [68].

Regarding the need for improving the “Number of Operations Supported”, the analysis revealed that this factor tended to grow as digital skills and ICT experts increased, but tended to decrease when the proportion of SMEs with product process innovations and GDPPPPpc increased. These findings indicate once more that a large percentage of ICT skills/specialists does not inevitably lead to an efficient “Number of Operations Supported”. Contrastingly, regions with a higher GDPPPP_PC and greater number of SMEs more susceptible to process innovation do not always have to apply for more ERDF-supported initiatives, since they show a higher efficiency in seeking funds.

5.3. Third Stage: Computing the Efficiencies of Every DMU with the Adjusted Inputs and Outputs through the SBM Model

As observed in Table 8, we established that the efficient OPs decreased their variability in terms of efficiency scores (the standard deviation was 0.15 compared with the previous 0.23). In addition, the efficiency scores were bounded within [1.00, 1.49], showing efficiency scores greater than 1.10 for more than 50% of the efficient OPs. Furthermore, the inefficient OPs also decreased the variability of their efficiency scores (with a standard deviation of 0.21 and more than 50% of inefficient OPs having efficiency values just under 0.41, compared with the previous 0.06) and considerably increased their efficiency (highlighting the relevance of the contextual variables).

Table 8. The descriptive statistics of the results obtained with adjusted factors.

	Statistics	Efficiency	Total Eligible Spending	Number of Operations	Total Eligible Costs Decided
Efficient DMUs	Mean	1.15	34,224,227.54	1190.82	56,264,030.64
	Median	1.10	2,016,502.16	323.85	3,015,425.50
	Standard Deviation	0.15	65,761,340.90	1832.82	104,511,357.35
	Minimum	1.00	329,249.00	1.00	251,294.00
	Maximum	1.49	237,904,467.00	5457.00	311,154,920.00
	Count	14	14	14	14
Inefficient DMUs	Mean	0.43	9,490,069.87	374.48	17,539,092.57
	Median	0.41	3,591,017.20	260.17	5,637,368.00
	Standard Deviation	0.21	17,915,300.53	291.66	35,107,221.84
	Minimum	0.01	657,276.80	2.00	671,429.00
	Maximum	0.88	103,201,263.85	1550.54	202,847,237.00
	Count	37	37	37	37

Source: Authors’ own computation.

Approximately 27% of the OPs achieved an efficient procedural performance, compared with the preceding 20%, i.e., 10 out of 51 (see Table 3 and Table 7).

Figure 1 shows the highest difference in the technical efficiency of the OPs with and without adjusted factors.

Figure 1. OPs that increased their technical efficiency with adjusted factors. Source: Authors’ own computation.

When compared with the first phase of the analysis, “Melilla—ERDF” and “Upper Norrland—ERDF,” showed the highest improvement on efficiency, with values ranging from 0.02 to 1.02 (4493% increase) and from 0.12 to 1.00 (763% increase), respectively (Figure 1). These findings imply that the former inefficiencies of the OPs depicted in Figure 1 were not exclusively due to their low technical level but were also related to their contextual factors.

As observed in Table 9, it is possible to conclude that despite the first three ranked OPs remaining the same irrespective of the adjustment of factors, the number of times that they achieved benchmark status was different.

Table 9. The characteristics of efficient OPs with adjusted factors.

OP	Nº of Times as Benchmark with Adjustment	Nº of Times as Benchmark without Adjustment	Rank with Adjustments	Rank without Adjustments
Provence-Alpes-Côte d’Azur—ERDF/ESF/YEI	0	20	1	1
Enterprise and Innovation for Competitiveness—CZ—ERDF	1	1	2	2
Multi-regional Spain—ERDF	9	14	3	3
Berlin—ERDF	18	0	4	22
Competitiveness Entrepreneurship and Innovation—GR—ERDF/ESF	0	0	5	6
Haute-Normandie—ERDF/ESF/YEI	4	0	6	27
Central Macedonia—ERDF/ESF	25	0	7	12
Extremadura—ERDF	2	27	8	5
Puglia—ERDF/ESF	10	0	9	17
Innovations and Competitiveness—BG—ERDF	7	4	10	7
Melilla—ERDF	0	0	11	43
Umbria—ERDF	4	0	12	13
Sachsen—ERDF	6	0	13	11
Upper Norrland—ERDF	0	0	14	28

Source: Authors’ own computation.

Now, “Central Macedonia—ERDF/ESF” (25), “Berlin—ERDF” (18), “Puglia—ERDF/ESF” (10), and “Multi-regional Spain—ERDF” (9) were the top 4 OPs selected more frequently as benchmarks (see Table 8). “Multi-regional Spain—ERDF” ranked among the top four efficient OPs, serving as a reference for the best practices, irrespective of the adjustments. Moreover, two of the OPs that remained efficient regardless of the adjustments of the factors belonged to Spain. It is important to mention that the MSs in Southern (such as Italy and Spain), Central, and Eastern EU were among the leading recipients of ICTs and digital economy assistance [68]. This is particularly true for nations with efficient OPs such as Spain, Greece, Bulgaria, and the Czech Republic [68]. Over half of the firms in countries with declining efficiency related to environmental factors, such as Poland (formerly ranked 10th and now ranked 22nd) and Bulgaria (formerly ranked 7th and now ranked 10th), had relatively low levels of digitization in 2016 [68]. Additionally, other OPs from Italy and Germany mainly became efficient because of environmental factors.

Finally, the required improvement on the “Number of Operations Supported” became significantly reduced, dropping from 313% to 111% (see Table 10).

Table 10. The improvement potential for the OPs with the adjusted factors.

Factor	Average Original	Average Projection	Variation
Total Eligible Spending	9,490,070	9,699,250	2%
Number of Operations	374	789	111%
Total Eligible Costs Decided	17,539,093	12,998,433	−26%

Source: Authors’ own computation.

5.4. Robustness Study and Sensitivity Analysis

Deterministic values are used for both inputs and outputs in conventional DEA approaches. However, the values considered to represent the input and output data are sometimes uncertain. These data might be expressed as intervals or so-called “fuzzy” numbers [75]. In this context, the Max DEA software transforms the “fuzzy” DEA model into two models to identify the lower and upper limits of the α-level associated with the membership functions of the efficiency values, thus allowing for the acquisition of the upper and lower bounds of the efficiency scores. The robustness assessment was conducted by employing data perturbations of 5% and 10%, either for the case of adjusted or non-adjusted factors, whereas the sensitivity analysis was performed by omitting a single factor of evaluation (input or output) at a time and subsequently evaluating the impacts thus obtained in the efficiency assessment [76].

5.4.1. Robustness Study

As per our results in Figure 2, just four efficient OPs remained efficient for both data perturbations without the consideration of the contextual factors: “Enterprise and Innovation for Competitiveness—CZ—ERDF”, “Provence-Alpes-Côte d’Azur—ERDF/ESF/YEI”, “Multi-regional Spain—ERDF”, and “Extremadura—ERDF”. In comparison, 19 OPs stayed potentially efficient for both data perturbations (i.e., 37% of the sample of OPs evaluated). Lastly, 28 OPs were robustly inefficient for the same data perturbations. As supported by Figure 3, we concluded that there were only three efficient OPs that remained efficient for the data disturbances of 5% and 10%, which were “Enterprise and Innovation for Competitiveness—CZ—ERDF”, “Provence-Alpes-Côte d’Azur—ERDF/ESF/YEI,” and “Multi-regional Spain—ERDF.” Without taking the adjusted factors into account, these OPs overlapped with three of the formerly most robust OPs.

Figure 2. Results of the robustness analysis per OP (non-adjusted factors). Source: Authors’ own computation.

Figure 3. Results of the robustness analysis per OP (adjusted factors). Source: Authors’ own computation.

Overall, these findings point to a poor use of ERDF money in supporting the ICT development of SMEs.

These outcomes corroborate the study by Pellegrin et al. (2018), where the EU appeared to lag behind its rivals (namely, the US, Japan, and South Korea), either in terms of ICT adoption or ICT infrastructures, particularly among SMEs.

5.4.2. Sensitivity Analysis

Figure 4 and Table 11 show the findings of the sensitivity analysis. The “Number of Operations” factor has the greatest influence on efficiency, as its omission from the analysis resulted in the highest value of |1-slope|, 0.4373, whereas the “Total Eligible Spending” factor has the lowest impact on efficiency. These results remained valid with the removal of contextual variables, that is, the factor that continued to show the greatest impact on efficiency is the “Number of Operations”.

Figure 4. The sensitivity analysis for the Total Eligible Spending (a) and Number of Operations (b) without adjustments and the sensitivity analysis for the for the Total Eligible Spending (c) and Number of Operations (d) with adjustments. The x-axis of each graph represents the original efficiency score, and the y-axis represents the recalculated efficiency by omitting one output factor at a time. The solid blue lines represent the lines of the best fit. Source: Authors’ own computation.

Table 11. Results of the sensitivity analysis.

Variables	Slope	|1-slope|	R2	Classification
Total Eligible Spending (non-adjusted)	0.6432	0.3568	0.556	Output
Number of Operations (non-adjusted)	0.5627	0.4373	0.421	Output
Total Eligible Spending (adjusted)	0.8105	0.1895	0.529	Output
Number of Operations (adjusted)	0.6524	0.3476	0.377	Output

Source: Authors’ own computation.

6. Conclusions and Further Research

The purpose of this study was to assess the procedural efficiency of the execution of 51 OPs that were linked to the support of ICTs in SMEs from 16 EU countries. We proposed a three-stage SBM modelling framework to achieve this. First, the SBM model was used to determine the efficiency scores of each OP. Relevant information was gathered regarding the changes that must be performed to mitigate the potential differences from the inefficient OPs against their benchmarks.

In contrast to the other frequently used approaches employed in similar contexts, namely in benchmark case studies, econometric and statistical approaches, and macroeconomic and microeconomic studies, the SBM model can be especially insightful for MAs, as it allows them to pinpoint the benchmarks and variations that should be applied to enhance the efficient execution of OPs. After eliminating the contextual effects and statistical noise, the second step involved applying the SFA model to the slacks of the inefficient OPs to modify the inputs and outputs. At this point, it was also possible to understand just how significantly the contextual factors may affect the efficiency of the application of ERDFs in various OPs committed to encouraging ICT usage in SMEs, and the relevance of management failures. Lastly, the formerly adjusted factors were used to generate new efficiency scores through the SBM model.

Our key findings are summarized below.

RQ1: Which factors hinder the efficient use of ERDF allotted to increase ICT usage in the EU SMEs?

Before adjusting the input and output factors, the “Number of Operations Supported” is the indicator that demands the most concern from MAs, while “Eligible Costs Decided” require a modest enhancement (i.e., a 22% reduction) and “Eligible Spending” demands a neglectable improvement (i.e., a 3% increase). Both with or without the adjusted factors (by removing the environmental effect and statistical noise), the “Number of Operations Supported” is the main factor that requires concern from MAs.

RQ2: Which OPs were widely reported as benchmarks throughout the programming period evaluated?

The OP that is most often elected as benchmark in this study, with or without the consideration of the environmental factors, is “Multi-regional Spain—ERDF,” which is ranked on the top four efficient OPs viewed as benchmarks in either adjustment situation. In this respect, it is important to note that the usage of “vouchers” in Spain (Pellegrini et al., 2018) appears to be an efficient way for contacting SMEs and offering them with assistance that is easy to manage and tailored to their needs.

RQ3: Which OPs show a more resilient performance in the face of variations in the metrics utilized?

The robustness assessment reveals that “Enterprise and Innovation for Competitiveness—CZ—ERDF”, “Provence-Alpes-Côte d’Azur—ERDF/ESF/YEI”, “Multi-regional Spain—ERDF”, and “Extremadura—ERDF” are the OPs that keep their efficiency for data variations of 5% and 10%, regardless of accounting for any adjustments of the factors. Moreover, without any adjustments, 55% of the OPs are robustly inefficient; with adjustments, 37% of the OPs are robustly inefficient. In any case, these data indicate that the management practices of these OPs were the primary cause for such results.

RQ4: Which contextual variables have the highest influence on the OPs’ inefficiency?

Our findings indicate that wealthier regions with a higher concentration of ICT professionals tend to underuse ERDF funds for boosting ICTs in SMEs. Additionally, a higher share of ICT skills/specialists corresponds to a lower “Number of Operations Supported”. Contrastingly, wealthier regions and a greater number of SMEs proposing product innovations appear to be more efficient in obtaining financial assistance. These findings may be linked to bureaucratic issues that act in compliance with EU processes, financial channels, and administrative legislation, particularly for SMEs.

RQ5: How does efficiency vary when contextual variables are introduced?

When the factors were adjusted, more than 27% of the OPs (14) attained technical efficiency compared with the prior 20% (10), highlighting the relevance of contextual variables in evaluating efficiency.

Overall, it can be concluded that SMEs’ access to ESIFs (especially ERDFs) remains limited, as they require the organizational capacity to deal with the many formalities for the application for and completion of ERDF projects. In comparison to “traditional” SME activities, this problem becomes more pressing when it comes to ICTs. In consequence, activities in a sector recognized for its fast change, such as ICTs, demand additional flexibility and skills. As a result, MAs should look for methods to provide additional assistance that streamline procedures and meet the requirements of SMEs.

Furthermore, our study emphasizes the scarcity of metrics available to measure the success of ESIF funds dedicated to ICT assistance in SMEs. Lastly, although this study provided novel insights and creative techniques for evaluating the efficiency of financing that is allocated to increasing the ICT usage in EU SMEs, future works should especially examine the economic consequences of these OPs, even though this examination continues to be a challenging endeavor.