Enhancing Knowledge on Energy Refurbishment of Buildings and Green Procurement through Living Labs

Abstract

Buildings account for a significant portion of energy consumption and greenhouse gas emissions, underscoring the urgent need for energy refurbishment and green procurement strategies. This study explores the potential of Living Labs, a collaborative and user-centered approach, to bridge knowledge gaps and foster innovation in these areas. The research employed a comprehensive methodology, including stakeholder surveys and statistical analyses, to evaluate knowledge enhancement in various domains such as green policies, mentoring, funding possibilities, and green public procurement. The results demonstrated statistically significant improvements in knowledge across different fields and stakeholder groups. This study contributes to the objectives of the European Green Deal by highlighting the role of Living Labs in advancing knowledge critical to achieving energy efficiency and sustainability targets. Based on these findings, several recommendations are proposed to further leverage the effectiveness of Living Labs in promoting the energy refurbishment of buildings and green public procurement.

1. Introduction

The growing severity of global warming and climate change highlights the importance of EU buildings in meeting the energy, emissions, and environmental standards outlined in the European Green Deal. By 2030, new buildings are required to be carbon-neutral, with the goal for all buildings to achieve this standard by 2050 [1]. Buildings in the EU account for the lion’s share of Europe’s energy consumption, consuming 40% of energy and producing 36% of greenhouse gas emissions. This significant impact is largely due to the prevalence of energy-inefficient buildings that rely heavily on fossil fuels [2]. A study conducted by the Buildings Performance Institute Europe (BPIE), encompassing data from 16 EU countries, reveals that a staggering 97% of the building stock requires energy refurbishment to realize the vision set for 2050. Moreover, addressing this concern is a strategic response to increasing energy prices: the EU’s least efficient buildings consume vastly more energy than newly constructed or adequately renovated ones [3]. Considering that the building stock in the EU accounts for more than one third of its total greenhouse gas (GHG) emissions, and with projections showing that 85% of these buildings will remain in use until 2050, there is an urgency to implement effective measures to reduce their environmental impact [2].

One of the most effective measures to achieve a significant reduction in greenhouse gas emissions from EU buildings is to prioritize energy efficiency improvements [4]. This can be accomplished by implementing policies and measures that focus on retrofitting and renovating existing buildings. By improving insulation, upgrading heating and cooling systems, and integrating renewable energy technologies, buildings can become more energy efficient and reduce their carbon footprint [5]. In addition to energy efficiency improvements, it is crucial to invest in sustainable building materials and construction practices. This includes using materials with lower embodied carbon, such as recycled or locally sourced materials, and implementing sustainable construction techniques that minimize waste and emissions during the building process [6]. Furthermore, promoting the use of smart technologies and digitalization can also contribute to reducing energy consumption and optimizing building performance [7]. These policies can include setting mandatory energy performance standards for buildings, providing financial incentives and subsidies for energy-efficient renovations, and promoting the use of renewable energy sources through feed-in tariffs or tax credits [8,9].

Additionally, the concept of green public procurement (GPP) plays a vital role in promoting sustainable practices in EU building stock. GPP involves the consideration of environmental criteria, such as energy efficiency and the use of sustainable materials, in the procurement of goods and services by public authorities [10]. By incorporating GPP into the procurement process, a significant impact on reducing greenhouse gas emissions and promoting sustainable practices in the construction industry can be achieved [11,12]. To effectively implement GPP, it is essential to establish clear guidelines and criteria for public authorities to follow when procuring goods and services for building projects. These guidelines should prioritize energy-efficient products and materials that have a lower environmental impact throughout their life cycle [13]. Furthermore, monitoring and evaluation mechanisms should be put in place to ensure that GPP criteria are being effectively implemented and to track the environmental performance of purchased goods and services [14].

To support the implementation of green public procurement in the EU, several policies and measures have been established. Italy, for example, has implemented a range of policies to promote green public procurement. These include setting specific environmental criteria for the procurement of goods and services, providing guidance and training to public authorities on implementing green procurement practices, and promoting the use of eco-labels and certifications in procurement processes [15]. Additionally, the EU Handbook on Green Public Procurement provides guidance and best practices for public authorities to effectively integrate environmental criteria into their procurement processes [16]. Furthermore, the European Union has also implemented the Circular Economy Action Plan, which recognizes the potential of green public procurement as a tool to promote sustainable practices and drive the transition to a circular economy. This includes promoting the use of environmentally friendly materials, encouraging energy efficiency measures in construction projects, and promoting the circular economy through waste reduction and recycling. In view of its tremendous economic proportions, public procurement is widely recognized as a potentially important tool for implementing the EU Circular Economy Action Plan [17]. It is, however, essential to ensure that the classic public procurement model is transformed into a green public procurement model. This can be achieved by incorporating environmental criteria and requirements into the procurement process and prioritizing goods, services, and works with reduced environmental impacts throughout their lifecycle when compared to alternatives [18].

While policies promote building refurbishment through strategies, legislations, and funding mechanisms, it is equally vital to consider softer measures [19]. As such, enhancing the knowledge of those who execute the investments is a pivotal way to boost the effectiveness of energy refurbishment. In a rapidly evolving world driven by technological advancements and environmental challenges, the knowledge surrounding energy refurbishment and green public procurement remains surprisingly insufficient [20]. The reasons include a lack of awareness and understanding of sustainable building practices, limited access to up-to-date information and resources, inadequate training and education programs for professionals in the building sector, and a lack of incentives and support for implementing sustainable construction practices [21].

To address these challenges, various measures can be implemented. Firstly, educational and training programs can be developed to provide professionals in the building sector with the necessary knowledge and skills regarding sustainable building practices, including energy efficiency measures and green procurement strategies [22]. Secondly, awareness campaigns can be conducted to educate stakeholders, including public authorities and private companies, about the benefits of sustainable building practices and green public procurement [23]. Thirdly, incentives and support mechanisms can be put in place to encourage the adoption of sustainable construction practices [24].

The urgency to enhance this knowledge is evident; however, mere conventional approaches may prove inadequate. The field of green energy and green growth is a vast area, and transfer and co-creation of new knowledge is a very challenging activity, due to time and human resource restriction [2]. What is required are innovative solutions that can bridge this knowledge gap effectively and efficiently. In this context, different approaches can and should be used to increase the knowledge, research, and innovation competences of various stakeholders, such as workshops and training sessions, case study analysis, mentorship programs, and study tours [25]. Especially effective are Living Labs (LLs), which stimulate stakeholders to be actively engaged in actions, allowing them to gain new knowledge, innovation, and research capacities, and to co-create common knowledge and solutions [26].

Living labs have proven to be one of the most promising approaches to engage and stimulate stakeholders to co-create innovation in favor of green energy and growth. Living Labs have gained prominence as dynamic research environments where real-world experimentation takes place [27,28,29]. These open innovation ecosystems have been utilized across various domains, including energy efficiency and sustainability [30,31,32,33,34,35]. Recent studies highlight their effectiveness in fostering collaboration among diverse stakeholders, facilitating user engagement, and driving innovation in sustainable practices [26,36,37,38]. In recent research, Living Labs (LLs) have increasingly been recognized as an effective method for knowledge transfer across various domains. For example, studies have highlighted the role of LLs in facilitating the exchange of knowledge in natural areas, sustainability practices, and technological innovation [39]. Additionally, some contemporary studies have developed assessment frameworks to evaluate how Living Labs contribute to sustainability transitions. However, these frameworks often emphasize the functioning of the LLs themselves, rather than focusing explicitly on the knowledge transfer process [40]. Moreover, other studies have employed methods such as Delphi studies to identify significant gaps in LLs’ performance and concluded that future research should focus on addressing knowledge gaps to enhance the effectiveness of LLs in various contexts [31]. While numerous studies address the effectiveness of Living Labs (LLs) and their role in knowledge acquisition, many fail to adopt research approaches that precisely measure the extent of knowledge improvement resulting from participation in LLs [41,42]. Our article makes a significant contribution by demonstrating, through rigorous evaluation, the substantial enhancement in specific knowledge areas, such as green policies, funding mechanisms, and public procurement, resulting from active involvement in Living Labs.

This article focuses on the pivotal question of whether Living Labs have genuinely enhanced the knowledge of their stakeholders in the areas of energy refurbishment and green public procurement. This study aims to explore the effectiveness of Living Labs in enhancing stakeholder knowledge in the areas of energy refurbishment of buildings and green public procurement. This research employs a robust methodological framework, including stakeholder surveys and a series of in-depth statistical analyses such as the Wilcoxon Signed-Rank Test and the Fisher–Freeman–Halton Test, to evaluate the impact of Living Labs on knowledge improvement. By analyzing data across different countries and organizational types, this study provides critical insights into how Living Labs contribute to the broader goals of the European Green Deal, particularly in fostering sustainable practices in the built environment. The findings reveal statistically significant improvements in knowledge, offering a detailed understanding of the extent to which participation in Living Labs enhances stakeholders’ expertise in these key areas.

The article in Section 2 offers an overview of the adopted methodology and data. Section 3 presents the results, followed by in-depth statistical analyses. Subsequent discussions on these results are elaborated upon in Section 4. Concluding remarks from this study are gathered in Section 5.

2. Evaluate Knowledge Enhancement of Living Labs: A Methodological Approach

The evaluation of Living Labs is crucial to assess their effectiveness in improving stakeholders’ knowledge. Evaluation methods and approaches play a vital role in capturing the impacts and benefits of Living Labs. One approach to evaluating the effectiveness of Living Labs is through networking and knowledge sharing among stakeholders [26]. Creating platforms and forums for stakeholders to share their experiences, challenges, and lessons learned enables the collection of valuable insights and the evaluation of how much Living Labs have enhanced stakeholders’ knowledge [43].

Another approach is to build upon the existing literature and research on Living Labs [44]. This can involve reviewing studies that explore the functioning and outcomes of Living Labs, such as their role in fostering innovation, the involvement of users and stakeholders, innovation methodologies and tools, and business model innovation. Furthermore, the use of action research can be employed to capture and address the experiences, feedback, and suggestions of participants in Living Labs. By using an action research approach, practitioners can optimize the operations of the Living Lab itself and design effective and sustainable user involvement processes [45]. The most promising approach, which is used also in this research, is stakeholder self-assessment [46]. The latter has been accomplished in the present research where surveys were conducted with various stakeholders to provide quantitative and qualitative data on their level of knowledge before and after participating in a Living Lab. This research focuses on the benefits of participating in Living Labs (LLs) in different countries, involving various stakeholders and organizational aspects. It specifically targets the domains of energy refurbishment and green public procurement within the context of Living Labs.

2.1. Participating Living Labs and Setting up Methodology

A total of seven Living Labs from six different countries—Greece, Italy, Slovenia, Spain, Cyprus, and Bosnia and Herzegovina—were selected for this study to ensure a diverse and comprehensive analysis of knowledge transfer processes across various cultural, economic, and regulatory environments. The primary objective behind establishing various Living Labs was to foster the development of innovative solutions and concepts within the realms of energy refurbishment of buildings and green public procurement. The LLs participants were provided with essential knowledge and tools related to the energy-efficient refurbishment of buildings and the implementation of green procurement practices.

The selection criteria for the Living Labs (LLs) focused on ensuring geographical and cultural diversity, with LLs chosen from six different countries to represent a broad range of cultural, economic, and regulatory environments across Europe. By including Living Labs from both Mediterranean and Central European countries, this study aims to capture a wide range of regional challenges and opportunities in green public procurement and energy refurbishment. The selected countries varied in economic development and regulatory frameworks, enabling this study to explore how these differences impact the adaptability of LL methodologies. Additionally, the LLs represented a mix of organizational types, including SMEs, public authorities, and educational institutions, to examine how different organizational characteristics influence knowledge transfer and innovation. The focus on energy-efficient refurbishment of buildings and green public procurement was a key criterion, aligning with this study’s objectives. LLs were also required to comply with a structured operational methodology involving six phases, ensuring consistency in implementation across diverse contexts. Furthermore, the ability of the LLs to effectively engage stakeholders and tailor educational activities to local needs was a crucial factor in their selection. Finally, the LLs were chosen based on their potential to evaluate implemented solutions and disseminate findings effectively to relevant audiences, ensuring a comprehensive analysis of knowledge transfer and innovation in energy and green procurement across Europe.

The working methodology or pathway for establishing all LLs consisted of six operational phases: Phase 1: Connect, Phase 2: Educate and Train, Phase 3: Implement, Phase 4: Improve, Phase 5: Evaluate, and Phase 6: Disseminate, as seen in Figure 1.

In the first phase “Connect”, the primary focus was on identifying and engaging relevant stakeholders, including SMEs, public authorities, and R&D institutions, to establish a network with clear leadership and defined roles. This network was built around a common problem or opportunity for collaboration. Following the connection phase, the “Educate and Train” phase offered workshops and training sessions on tools and methodologies for energy management, green procurement, and refurbishment. The goal of this phase was to equip stakeholders with the necessary knowledge and skills for the subsequent implementation phase. During the implementation phase, stakeholders were encouraged to apply and test the proposed solutions in real-world environments, providing valuable feedback for identifying areas of improvement. This feedback was then addressed in the “Improve” phase, where necessary adjustments and new solutions were developed. In the final stages, the process involves evaluating the effectiveness of the solutions and the overall Living Lab process, assessing outcomes, stakeholder experiences, and the overall impact. Based on this evaluation, guidelines and recommendations are developed. Following the evaluation, the results, knowledge, and best practices are shared with a broader audience, and findings are published along with recommendations for future Living Labs.

Through these six operational phases, we can notice some differences among LLs. The approach to connecting with stakeholders varied significantly. For example, some LLs (like those in Slovenia and Cyprus) had a strong focus on involving public authorities and SMEs early in the process, emphasizing green public procurement. In contrast, other Labs might have prioritized different types of stakeholders, such as research institutions or community organizations, depending on the local context and specific goals of the LLs. The educational activities and training sessions were tailored to the specific needs of the participants in each LL. For instance, in Slovenia, the focus was on educating stakeholders about the eGPP tool and green public procurement, while in other Labs, training might have been more oriented towards energy management or green refurbishment techniques. The content and depth of training were adjusted based on the existing knowledge levels of the stakeholders involved. The implementation phase differed based on the specific tools and strategies being tested. In Slovenia, the emphasis was on implementing the eGPP tool among SMEs and public authorities, with a focus on entering products and using green public procurement. Other LLs may have implemented different solutions or tools relevant to their specific focus, such as energy efficiency measures or green building practices. The improvement phase involved collecting feedback and adjusting the tools or processes used in each LL. The nature of these improvements varied depending on the initial implementation results. For example, in Slovenia, improvements were made to the eGPP tool based on user feedback, focusing on ease of use and functionality. In other LLs, improvements might have focused on different areas, such as the LCC tool, refining training materials or adjusting the focus of educational sessions. While the same evaluation methods and metrics were used when measuring general knowledge, when going into a specific scope and topics, the evaluation was adopted to reflect their different objectives and the tools they implemented. In some LLs, evaluation focused on the effectiveness of knowledge transfer in the field of green polices or funding, while others might have emphasized the impact of implemented solutions on energy efficiency or procurement processes. The dissemination activities also varied, with some LLs focusing on national or regional events to share their findings and best practices, while others aimed for broader, transnational dissemination through conferences, publications, and networking events. The specific audiences targeted during dissemination varied depending on the LLs’ focus and the stakeholders involved.

LLs were addressing a broad spectrum of challenges, ranging from green public procurement, green funding and mentoring, green policy, and electricity consumption, with the following focus relevant to their countries specific needs:

• The LLs in Slovenia and Bosnia and Herzegovina, focused on sharing and gaining expertise in tackling challenges associated with green public procurement;
• The LLs in Italy and Greece assumed a leading role in addressing funding and mentoring-related challenges;
• The LLs in Cyprus and Italy played a pivotal role in shaping green policy solutions;
• The LL in Spain focused on an aspect of electricity consumption and energy management solutions.

The diversity of fields and contexts presents an excellent opportunity for comparing evaluation results related to different cultural environments, specific area characteristics, and various topics. The Living Labs engaged a diverse range of stakeholders, including policymakers, energy efficiency experts, financing experts, public authorities (PAs), and small and medium enterprises (SME). Public authorities, such as universities, chambers of commerce, and development agencies, were responsible for energy refurbishment of buildings and green public procurement. Companies participated as potential product or service providers. Additionally, energy, finance, and mentoring experts contributed their expertise in funding and mentoring opportunities and in developing and using eco-innovative solutions for green public procurement and building refurbishment. Policy stakeholders, such as ministries for public works or infrastructure, were responsible for designing public policies, action plans for energy refurbishment of public buildings, and frameworks to support green energy policies.

2.2. Evaluation and Survey Methodology

The assessment methodology employed in this study was based on a comprehensive set of Key Performance Indicators (KPIs), as detailed in Appendix A,

Table A1. KPI for measuring the level of knowledge for all LLs.

Indicator Topic	Indicator Description
Green Policies: designed to promote environmental sustainability across varying administrative levels.	This KPI evaluates the depth of participants’ understanding of such policies that form the backbone of sustainable initiatives. Whether at the EU, national, or local level, these policies provide strategic guidelines, standards, and incentives. They steer organizations and governments toward environmentally conscious decisions, particularly in areas of green energy, eco-innovations, and building refurbishments.
Funding for Green Investments: financial resources allocated to environmentally friendly and sustainable projects or initiatives.	Through this KPI, participants’ familiarity with the diverse funding opportunities available for green endeavors is assessed. With the rising importance of green energy, pioneering eco-innovations, and sustainable building refurbishments, understanding the available financial avenues is paramount to foster and implement sustainable strategies.
Green Public Procurement: estimates the procurement of goods and services with minimal environmental impacts.	This KPI serves as a barometer for participants’ grasp of best practices in green procurement, especially within the realm of energy-centric building refurbishments. As public authorities prioritize sustainability in their procurement decisions, understanding these practices becomes essential in driving market demand for green products and fostering eco-innovation.
eGPP Tool is an electronic tool forged for green public procurement.	This KPI assesses participants’ adeptness with this tool, emphasizing its applicability in facilitating sustainable procurement decisions related to green energy, cutting-edge eco-innovations, and building refurbishment.
Mentoring Mechanisms: structured programs or processes where seasoned professionals guide, train, and support less experienced individuals or groups.	This KPI measures participants’ awareness and understanding of mentoring processes tailored specifically for the realm of green energy, eco-innovations, and building refurbishment. Such mechanisms are pivotal in capacity building, ensuring that knowledge gaps are bridged and best practices are disseminated, thereby accelerating the adoption of sustainable practices across sectors.
Best Practices: optimal procedures or techniques that have been validated through consistent results and research.	This KPI evaluates participants’ knowledge of such practices across various areas, from innovative funding strategies to energy conservation and advanced building refurbishments. Embracing these practices ensures efficiency, cost savings, and alignment with sustainability goals.
Pilot Actions in Green Energy: preliminary projects crafted to test and validate the feasibility of innovative concepts, strategies, or technologies.	This KPI evaluates participants’ knowledge regarding trailblazing initiatives in the verdant spheres of green energy, eco-innovations, and building refurbishment. Understanding these pilots is crucial as they provide insights into challenges, benefits, and areas of improvement, informing larger-scale implementations.
Living Lab Concept: an open innovation approach where users and stakeholders collaboratively develop solutions in real-world environments.	This KPI delves into participants’ understanding of this sophisticated framework, spotlighting their comprehension of its potential to foster co-creation, iterative testing, and feedback in the context of green energy and sustainability.

,

Table A2. KPI for measuring the level of knowledge in LLs focusing on green policy.

Indicator Topic	Indicator Description
Policy energy goals of the EU until 2020 and 2030	The indicator measures familiarity of LL stakeholders with policy energy goals of the EU until 2020 and 2030 after participating in Living Lab activities.
Knowledge of European green policies	The indicator measures the level of LL stakeholders’ knowledge regarding the existing European green policies for eco-innovations and energy refurbishment of buildings after participating in Living Lab activities.
Knowledge of national green policies	The indicator measures the level of LL stakeholders’ knowledge of existing national green policies for eco-innovations and energy refurbishment of buildings after participating in Living Lab activities.
Knowledge of green procurement policies	The indicator measures the level of LL stakeholders’ knowledge of the existing national green procurement policies after participating in Living Lab activities.
Innovative policies (EE and RES)	The indicator measures LL stakeholders’ knowledge of innovative polices on energy efficiency and energy sources after participating in Living Lab activities.
Knowledge of SEAP	The indicator measures the level of LL stakeholders’ knowledge regarding SEAP.

,

Table A3. KPI for measuring the level of knowledge in LLs focusing on funding.

Indicator Topic	Indicator Description
Knowledge of national funding mechanism	The indicator measures the level of knowledge of LL stakeholders on existing national funding mechanisms for eco-innovations and energy refurbishment of buildings after participating in Living Lab activities.
Knowledge of European funding mechanism	The indicator measures the level of knowledge of LL stakeholders on existing European funding mechanisms for eco-innovations and energy refurbishment of buildings after participating in Living Lab activities.
Identification of appropriate funding opportunities	The indicator measures how well the LL activities assist stakeholders in identifying appropriate funding opportunities according to their needs in the field of green energy.
Applying for funding	The indicator measures how well the LL activities assist stakeholders when applying and/or preparing documentation for funding opportunities in the field of green energy.
Up to date with funding opportunities in energy market	The indicator measures how much the LL activities affect stakeholders to be more up to date with funding opportunities in the energy market.
New financial mechanism for energy management	The indicator measures how much the LL activities affected stakeholders’ knowledge of new financial mechanisms for energy management.
New financial mechanism for green refurbishment of buildings	The indicator measures how much the LL activities affected stakeholders’ knowledge of new financial mechanisms for green refurbishment of buildings.
New types of financial partnership	The indicator measures how much the LL activities affect identifying or forming new kinds of partnership with shared financial schemes (such as public–private partnership, etc.)
Good practices on funding	The indicator measures how well the LL activities affect stakeholders’ awareness of existing good practices for funding green energy, eco-innovations, and energy refurbishment of buildings.

,

Table A4. KPI for measuring the level of knowledge in LLs focusing on mentoring.

Indicator Topic	Indicator Description
Knowledge of national mentoring mechanism	The indicator measures the level of knowledge of LL stakeholders on existing national mentoring mechanisms for eco-innovations, and energy refurbishment of buildings after participating in Living Lab activities.
Knowledge of European mentoring mechanism	The indicator measures the level of knowledge of LL stakeholders on existing European mentoring mechanisms for eco-innovations and energy refurbishment of buildings after participating in Living Lab activities.
Identification of appropriate mentoring schemes	The indicator measures how well the LL activities assist stakeholders in identifying appropriate mentoring opportunities according to their needs in field of green energy.
Engaging mentoring services	The indicator measures how well the LL activities assist stakeholders in engaging with mentoring in the field of green energy.

and

Table A5. KPI for measuring the level of knowledge in LLs focusing on GPP.

Indicator Topic	Indicator Description
Knowledge on electronic green public procurement	The indicator measures the level of knowledge on electronic green procurement gained throughout the use of the eGPP tool and online platform.
Knowledge on green public procurement criteria	The indicator measures the level of knowledge on green public procurement criteria after use of the eGPP tool and platform.
Use of green public procurement criteria	The indicator measures how the use of green public procurement criteria changed after using the eGPP tool and platform.
Knowledge on green best practices	The indicator measures how use of the eGPP tool and platform affected knowledge of green best practices.
Knowledge on environmental questions	The indicator measures the level of environmental answers gained throughout the use of the eGPP tool.
Knowledge on tender implementation	The indicator measures how much the level of knowledge on implementing the tenders for energy refurbishment of public buildings improved after using tools provided in the LLs.

. These indicators were translated into a survey, which is explained in more detail in the following section, using a Likert scale to effectively gather stakeholder feedback.

During the Connect phase of the Living Labs (LLs), relevant organizations were identified for each LL. In response to their common challenges and knowledge gaps in the energy refurbishment of buildings, the curriculum for the education and training sessions was adapted accordingly. Outcome measures were also defined for each LL, with necessary actions and strategies put in place to achieve these goals, as seen in

Table 1. LLs’ actions in relation to outcome measures and KPIs.

LL Scope (Country)	Specific Actions/Strategies	Outcome Measure
All LLs	Delivering series of education sessions to provide a comprehensive understanding and awareness of green policies, funding opportunities, green public procurement, mentoring practices, and best practices across various areas. This includes innovative funding strategies, energy conservation, advanced building refurbishments, and the Living Labs (LLs) approach, highlighting its potential to foster co-creation, iterative testing, and feedback in the context of the energy refurbishment of buildings.	Increased understanding and awareness of green policies, funding opportunities, green public procurement, mentoring practices, and the Living Labs (LLs) approach, particularly as applied in the field of energy refurbishment of buildings (measured via KPIs: Appendix A: Table A1).
Green policy (Cyprus, Italy)	Delivering education and training sessions on the green policy goals of national and European policies, especially innovative policies related to energy efficiency (EE), renewable energy sources (RES), and SEAP. During the implementation and improvement phases of the Living Labs sessions, necessary improvements to energy-related national policies were proposed.	Enhanced knowledge of national and European energy policy goals, green policies, and innovative approaches to energy efficiency and renewable energy sources, specifically in relation to the energy refurbishment of buildings (measured via KPIs: Appendix A: Table A2).
Green funding and mentoring (Greece and Italy)	Delivering education and training sessions on financing energy refurbishment of public buildings, with a focus on relevant legislation, application procedures, current funding opportunities, innovative financial mechanisms, best practice examples, and available mentoring support. During the implementation and improvement phases of the Living Labs sessions, specific funding and mentoring mechanisms/services were utilized, and customized funding options were proposed.	Identification of funding and mentoring opportunities for green growth, ensuring tailored mentoring or financing to SMEs and PAs for eco-innovation actions (measured via KPIs: Appendix A: Table A3 and Table A4)
eGPP (Slovenia and Bosnia and Herzegovina)	Providing education and training on green public procurement legislation, criteria, tender procedures, environmental concerns, and best practice cases, along with the use of the eGPP tool, equipping participants with the skills needed to implement sustainable procurement practices. Through implementation and improvement sessions, participants are empowered to effectively use the eGPP tool and contribute to its ongoing enhancement.	Increased knowledge in green public procurement and energy-efficient refurbishment practices and expanded use of the eGPP tool. (measured via KPIs: Appendix A: Table A5)

. Based on this, KPIs (as mentioned before) were established to track progress in the improvement of participant knowledge.

The training and education sessions were structured into several phases. The first phase offered a series of workshops that provided general information on the Living Lab concept, relevant policies, mentoring, and available funding opportunities, alongside green public procurement process. In subsequent phases, additional training sessions were executed which allowed participants to deepen their knowledge in specialized sub-fields such as green policy, funding, mentoring, and green public procurement. The final phase provided interactive sessions that showcased real-world examples of best practices for the energy refurbishment of buildings. In some LLs, such as in the case of Slovenia, this phase also featured the use of specific tools, such as the eGPP tool for the public procurement process. The training was highly interactive, allowing participants to be actively engaged in the learning process and facilitating the exchange of ideas and opinions. Consequently, one set of KPIs measured the overall improvement in knowledge provided to all LLs (KPIs detailed in Appendix A: Table A1), while an additional set of KPIs specifically measured knowledge relevant to individual LLs, as these specialized topics were not covered by all Living Labs (KPIs in Appendix A: Table A2, Table A3, Table A4 and Table A5). These KPIs were directly linked to the actions taken within each LL. The connection between LLs’ scope, actions, strategies, outcomes, and KPIs can be seen in Table 1. For further evaluation of LLs specialized in a specific topic, we have considered 6 LLs as detailed in Table 1, while the LL established in Spain was not part of further analysis due to small sample size.

The survey, provided in the Supplementary Materials, was conducted through an online questionnaire, completed in one session, that gathered information on participants’ perceived knowledge both before and after participating in the LLs. Simultaneous assessment of before- and after-training knowledge/skills is recommended to provide immediate insight into learning effectiveness while reducing response-shift bias, especially when participants may not fully recognize their initial knowledge gaps. Additionally, it is more feasible than a pre–post design, reducing participant burden and ensuring higher response rates and reliable data [47,48].

The survey consists of 20 questions that ranged from fundamental demographic information, including the country of operation, the scope of the Living Lab, and the type of participating organization, to more focused areas such as participants’ satisfaction with their Living Lab experience, the benefits they perceived from their involvement, and any enhancements in their knowledge. This research aims to examine the development of participants’ knowledge because of their engagement in Living Labs (LLs). The focus was on evaluating the enhancement of knowledge across all LLs and differences among LLs that focused on specific topics such as green policy, funding, mentoring, and green public procurement. Responses to these inquiries on knowledge levels were gauged using a five-point Likert scale, with ratings as follows: 1—‘Poor’, 2—‘Fair’, 3—‘Good’, 4—‘Very Good’, and 5—‘Excellent’. This facilitated understanding of participants’ experiences and outcomes from their engagement in Living Labs.

The survey was divided into two parts. The initial set of questions was directed at all Living Lab stakeholders, irrespective of the specialization in specific areas mentioned before (in Section 2.1). Altogether eight KPIs were established to measure improvement comparing the level of knowledge before and after participating in an LL, covering areas such as green policies, funding, green public procurement, mentoring, best practices pilot actions, and the LL concept as presented in Appendix A: Table A1. The results of this part are presented in Section 3.1. The first set of questions allows us to compare the improvement in perceived general knowledge, regardless of LL specialization, since all LLs received basic training on these topics through various workshops and educational sessions. The second part of the survey questions was tailored to reflect the specific focus area of each Living Lab. This means that, for example, in the case of LLs focusing on green public procurement, the set of questions aims to measure more in-depth knowledge on the topic, rather than just general understanding of what green public procurement is. Accordingly, the evaluation of knowledge enhancement varied depending on the Living Lab’s area of specialization. For Living Labs concentrated on green policy, knowledge improvement was measured using six Key Performance Indicators (KPIs) detailed in Appendix A, Table A2. Living Labs with a focus on funding were assessed using a set of nine distinct KPIs, outlined in Appendix A, Table A3. Living Labs dedicated to mentoring topics underwent evaluation based on four KPIs (Tabel A3), and those specializing in the field of green public procurement were assessed using six KPIs, as detailed in Appendix A, Table A4. This customized approach ensured that the assessment was relevant and accurately reflected the specific objectives and outcomes of each Living Lab’s area of expertise. The results of this part are presented in Section 3.2, Section 3.3, Section 3.4, Section 3.5. The Living Lab specialized in the aspect of electricity consumption and energy management solutions was excluded from more detailed analysis due to the small responsive sample.

To graphically present the average level of knowledge, we calculated percentages of participants who rated their level of knowledge as Poor, Fair, Good, Very Good, and Excellent (see graphical presentation of results in the following section). To quantitatively evaluate the level of knowledge, a weighted averages method was applied. This method calculates the average of a set of values $x$ (in our case percentages of participants assigned to a particular level of knowledge), each assigned a weight that reflects its relative importance. In our analysis, the weights ($w)$ for level for knowledge were assigned as follows: Poor = 1, Fair = 2, Good = 3, Very Good = 4, and Excellent = 5. Mathematically, the weighted average is calculated using the following formula:

$${\overline{x}}_w={\displaystyle \frac{\sum _{1=1}^n{w}_i{x}_i}{\sum _{1=1}^n{w}_i}}$$

(1)

In this formula, $x_i$ represents the individual data points (percentages), $w_i$ denotes the weights assigned to these data points, and $n$ is the number of data points. Weighted average is calculated for knowledge before $\left({\overline{x}}_B\right)$ and after $\left({\overline{x}}_A\right)$ participating in the Living Lab. Based on that, relative improvement of knowledge is calculated ($∆{\overline{x}}_{A-B})$, as presented in the following section. The weighted averages were applied, because they enable more accurate representation of final averages especially in the case of having a data set where some groups are over-represented or under-represented.

3. Results

The sample of 86 survey participants from seven Living Labs revealed that 44% associated their scope with green policy, 26% with green funding or mentoring, 25% with green public procurement, and 5% with electricity consumption, with respondents predominantly from Italy (30%), Greece (24%), Cyprus (21%), Bosnia and Herzegovina (13%), and Slovenia (12%); 61% were from public authorities (PAs), 21% from small and medium enterprises (SMEs), and the remainder from higher education, research, and business support organizations.

3.1. Level of General Knowledge Gain through Participation in Living Labs

In our research, we explored the level of general knowledge, before and after participating in LL. As seen from Figure 2, eight general topics were addressed regarding the knowledge on green policies, mentoring, funding, procurement process, best practices, LL concept, pilot actions, and the eGPP tool related to green procurement and the energy refurbishment of buildings. Level of knowledge has been examined in all LLs, regardless of their specialization. The survey results indicate a significant enhancement in knowledge across diverse areas of green policies and practices. This improvement is characterized by a substantial decrease in the percentage of stakeholders having lower knowledge levels (‘Poor’ and ‘Fair’) and an increase in higher levels (‘Good’, ‘Very good’, and ‘Excellent’) after participating in LLs as seen in the left part of Figure 2.

As illustrated in Figure 1 (the right part indicating average values (${\overline{x}}_B,{\overline{x}}_A)$ and differences ($∆{\overline{x}}_{A-B}),$the lowest level of knowledge before participating in LLs was observed for the topic of the Living Lab concept $({\overline{x}}_B=2.03)$ and eGPP tools $({\overline{x}}_B=2.08)$, while the highest level of knowledge was observed for the topic of green policies $({\overline{x}}_B=2.62)$. This indicated that stakeholders had, on average, fair knowledge of LLs and eGPP tools and quite good knowledge of green policies. After participating in LLs, the highest level of knowledge was observed in the field of pilot actions $({\overline{x}}_A=3.75)$. In relative terms, knowledge level improved in all areas, with the highest relative improvement being observed in the field of eGPP tools ($∆{\overline{x}}_{A-B}=1.45$), the Living Lab concept ($∆{\overline{x}}_{A-B}=1.44)$, and knowledge related to pilot actions ($∆{\overline{x}}_{A-B}=1.42)$. Besides assessing the improvement in general knowledge from participating in Living Labs (LLs), the aim of this research was also to evaluate the enhancement of in-depth knowledge specific to the LLs’ areas of specialization. Thus, the subsequent section provides insights into the level of expertise developed in specialized fields such as green policies, funding, mentoring, and green public procurement among LLs focused on these topics.

3.2. Level of Knowledge Gain in LLs Specialized in Green Policy

The survey results of LLs specialized in Green Policy areas indicate a notable enhancement in knowledge after participation. As seen from the left part of Figure 3, the improvement is marked by a substantial decrease in lower knowledge levels (‘Poor’ and ‘Fair’) and an increase in higher levels (‘Good’, ‘Very good’, and ‘Excellent’).

As illustrated in Figure 3 (the right part indicating average values (${\overline{x}}_B,{\overline{x}}_A)$ and differences ($∆{\overline{x}}_{A-B})$), the lowest level of knowledge before participating in the LLs was observed for the topic of “European green policies for eco-innovations and energy refurbishment of buildings”$({\overline{x}}_B=2.28)$, while the highest level of knowledge was noted for “Knowledge of Sustainable Energy Action Plan (SEAP)” $({\overline{x}}_B=2.66)$. This indicates that stakeholders generally had a lower initial understanding of European green policies compared to their knowledge of the SEAP. After participating in LLs, the highest level of knowledge was observed for the topic of “Green procurement policies” (${\overline{x}}_A=3.40)$. In relative terms, knowledge levels improved across all areas, with the highest relative improvement being observed in the field of “National green policies for eco-innovations and energy refurbishment of buildings” ($∆{\overline{x}}_{A-B}=0.87)$ followed by “European green policies for eco-innovations and energy refurbishment of buildings” ($∆{\overline{x}}_{A-B}=0.86)$ and “Policy energy goals of the EU until 2020 and 2030” ($∆{\overline{x}}_{A-B}=0.86)$. This demonstrates a significant increase in stakeholders’ knowledge after participating in LLs, particularly in understanding national green policies and the EU’s energy goals for the coming decades.

3.3. Level of Knowledge Gain in LLs Specialized in Funding

As seen from the left part of Figure 4, for LLs specialized in funding areas, a significant enhancement of knowledge after participating in LLs can also be observed.

As illustrated in Figure 4 (the right part indicating average values (${\overline{x}}_B,{\overline{x}}_A)$ and differences ($∆{\overline{x}}_{A-B})$), the lowest level of knowledge before participating in the LLs was observed for the topic of “New financial mechanism for energy management” $({\overline{x}}_B=1.78)$, while the highest level of knowledge was noted for “Funding opportunities corresponding to my needs” $({\overline{x}}_B=2.49).$ This indicates that stakeholders generally had a lower initial understanding of new financial mechanisms specifically tailored for energy management compared to general funding opportunities that align with their needs. After participating in LLs, the highest level of knowledge was observed for the topic of “Good practices on funding for eco-innovations and energy refurbishment of buildings” (${\overline{x}}_A=3.55)$. In relative terms, knowledge levels improved significantly across all areas, with the highest relative improvement being observed for the field of “Good practices on funding for eco-innovations and energy refurbishment of buildings”($∆{\overline{x}}_{A-B}=1.39)$, followed by “New financial mechanism for energy management” ($∆{\overline{x}}_{A-B}=1.37)$ and “New types of financial partnership” ($∆{\overline{x}}_{A-B}=1.36)$. This demonstrates a substantial increase in stakeholders’ knowledge after participating in LLs, particularly in understanding new financial mechanisms and best practices for funding eco-innovations and energy refurbishment projects.

3.4. Level of Knowledge Gain in LLs Specialized in Mentoring

As seen from the left part of Figure 5, LLs specialized in mentoring knowledge areas demonstrate a significant increase in understanding post-engagement.

As illustrated in Figure 5 (the right part indicating average values (${\overline{x}}_B,{\overline{x}}_A)$ and differences ($∆{\overline{x}}_{A-B})$), the lowest level of knowledge before participating in the LLs was observed for the topic of “Engaging mentoring services for eco-innovations and energy refurbishment of buildings” $({\overline{x}}_B=1.92)$, while the highest level of knowledge was noted for “National mentoring mechanism for eco-innovations and energy refurbishment of buildings” $({\overline{x}}_B=2.21)$. This indicates that stakeholders generally had a lower initial understanding of how to engage mentoring services specifically tailored for eco-innovations and energy refurbishment compared to national mechanisms for mentoring. After participating in LLs, the highest level of knowledge was observed for the topic of “Engaging mentoring services for eco-innovations and energy refurbishment of buildings” (${\overline{x}}_A=3.22)$. In relative terms, knowledge levels improved significantly across all areas, with the highest relative improvement being observed for the field of “Engaging mentoring services for eco-innovations and energy refurbishment of buildings” ($∆{\overline{x}}_{A-B}=1.30)$, followed by “European mentoring mechanism for eco-innovations and energy refurbishment of buildings” $(∆{\overline{x}}_{A-B}=1.07)$ and “Application requirements and procedures for mentoring services on eco-innovations and energy refurbishment of buildings” $(∆{\overline{x}}_{A-B}=1.07)$. This demonstrates a substantial increase in stakeholders’ knowledge after participating in LLs, particularly in understanding how to engage mentoring services for eco-innovations and energy refurbishment projects.

3.5. Level of Knowledge Gain in LLs Specialized in Green Public Procurement

As seen from the left part of Figure 6, LLs focusing on electronic green public procurement (eGPP) highlight a pronounced improvement in knowledge following their participation.

As illustrated in Figure 6 (the right part indicating average values (${\overline{x}}_B,{\overline{x}}_A)$ and differences ($∆{\overline{x}}_{A-B})$), the lowest level of knowledge before participating in the LLs was observed for the topic of “Implementation of energy refurbishment of public buildings (tenders’ implementation) $({\overline{x}}_B=1.78)$, while the highest level of knowledge was noted for “Environmental questions considering green procurement and energy refurbishment of public buildings” $({\overline{x}}_B=2.51$). This indicates that stakeholders generally had a lower initial knowledge regarding the practical implementation of energy refurbishment projects using specific tools, compared to environmental considerations. After participating in LLs, the highest level of knowledge was observed for the topic of “Green public procurement criteria for energy refurbishment of public buildings” $({\overline{x}}_A=3.89)$. In relative terms, knowledge levels improved significantly across all areas, with the highest relative improvement being observed for the field of “Implementation of energy refurbishment of public buildings with eGPP tool” ($∆{\overline{x}}_{A-B}=1.82)$, followed by “Use of green public procurement criteria for energy refurbishment of public buildings” ($∆{\overline{x}}_{A-B}=1.66)$ and “Green public procurement criteria for energy refurbishment of public buildings” ($∆{\overline{x}}_{A-B}=1.62)$. This demonstrates a substantial increase in stakeholders’ knowledge after participating in LLs, particularly in the practical aspects of green public procurement and energy refurbishment projects.

3.6. Statistical Relevance of the Results

To determine the relevance and significance of stakeholder’s knowledge improvement, we utilized the Wilcoxon Signed-Rank Test for paired samples. This test was employed to compare knowledge levels on eight topics before and after engagement in all LLs.

A p-value of <0.001 indicates that there is a statistically significant difference between the “Before” and “After” level of knowledge for all eight topics (as seen form

Table 2. Wilcoxon Signed-Rank Test for level of knowledge before and after.

	Green Polices	Mentoring	Funding	Procurement	BP	LL	Pilots	eGPP
z	−7.267 *	−6.927 *	−7.069 *	−6.536 *	−7.596 *	−7.531 *	−7.263 *	−6.756 *
Asymp.Sig (2-tailed)				p < 0.001

* This Wilcoxon Signed-Rank Test was selected as the most appropriate because the data, captured using a Likert scale, were considered ordinal. It is important to mention that, in addition to the ordinal data consideration, the paired t-test was also excluded because it violated the assumption of normality.

). The negative Z-values indicates that, on average, “Before” values are lower than “After values”, which confirms that knowledge has statistically significantly improved (as explained in Section 3.1).

Further research was undertaken to discern if there were significant variations in knowledge levels among eight topics depending on the specialization of the Living Lab and the type of organizations (e.g., public authorities, small–medium enterprises, higher education and research organization, business support organization, and other).

We applied the Fisher exact test, chosen for its suitability with small sample sizes and analyzing contingency tables with more than 2 × 2 dimensions, to assess whether level of knowledge after participating in LLs varies among Living Labs (LLs) with different specializations. We have considered LLs specialized in green public procurement, green funding, and/or mentoring and green policy.

As seen in the

Table 3. Fisher–Freeman–Halton Tests (topics and scope of LLs).

Topics	Pearson Chi-Square	Likelihood Ratio	Fisher–Freeman–Halton Exact	p
Green Policies	25.434 *	29.772	26.261	0.001
Mentoring	29.314 *	31.264	24.845	0.004
Funding	16.929 *	17.550	16.476	0.120
Procurement	28.169 *	29.791	23.586	0.005
Best Practices	34.320 *	38.953	32.326	<0.001
LL concept	19.767 *	21.756	18.656	0.042
Pilots	19.498 *	19.928	18.226	0.057
eGPP	28.895 *	29.365	24.127	0.005

* 20 cells (80.0%) have expected count less than 5. The minimum expected count is 0.07.

, the test revealed that the level of knowledge on most topics (six out of eight) exhibited statistically significant differences (p < 0.05). No statistically significant difference was observed for topics related to funding and pilot actions. Living Labs (LLs) specializing in green public procurement demonstrated a significant enhancement in their knowledge in comparison to green funding and/or mentoring and green policy-oriented LLs. This is shown by the highest share of stakeholders claiming to reach excellent knowledge. The excellent level on the relevant topics was also achieved by stakeholders participating in LLs with green financing/mentoring, although the proportion of stakeholders is much lower than in green public procurement.

On the other hand, participants in LLs on green policy reported, for most topics (with the exception of the LL concept and eGPP), that they did not reach an excellent level of knowledge. The results also emphasize some logical correlations between the scope of the LL and the topics covered.

The eGPP achieves the highest proportion of stakeholders who are very knowledgeable about the eGPP (36.36%) compared to LLs focusing on funding/mentoring and green policy (22.73%/23.68%). Green funding/mentoring achieves the highest proportion for mentoring topics (36.36%) compared to eGPP (27.27%) and green policy LLs (31.58%). The LLs for green policy achieve the highest share for topics related to green policy (42.11%) compared to eGPP (36.36%) and funding/mentoring LLs (31.82%).

To summarize, the eGPP area leads the way when it comes to achieving the highest levels of knowledge, particularly on the topics of eGPP, “green policy”, and “best practice”. However, each area has its strengths and areas for improvement, reflecting the different focus and outcomes of the LLs’ learning initiatives. The results are presented in

Table 4. Distribution across topics and scopes of LLs.

Scope	Topics	Poor (%)	Fair (%)	Good (%)	Very Good (%)	Excellent (%)
eGPP	Green Policies	4.55%	4.55%	18.18%	36.36%	36.36%
	Mentoring	0.00%	27.27%	18.18%	27.27%	27.27%
	Procurement	0.00%	4.55%	22.73%	40.91%	31.82%
	Best Practice	0.00%	9.09%	13.64%	50.00%	27.27%
	LL Concept	0.00%	9.09%	27.27%	45.45%	18.18%
	eGPP	0.00%	13.64%	18.18%	36.36%	31.82%
Green funding and/or mentoring	Green Policies	4.55%	9.09%	45.45%	31.82%	9.09%
	Mentoring	9.09%	9.09%	36.36%	36.36%	9.09%
	Procurement	9.09%	9.09%	36.36%	36.36%	9.09%
	Best Practice	4.55%	0.00%	31.82%	54.55%	9.09%
	LL Concept	4.55%	18.18%	18.18%	45.45%	13.64%
	eGPP	13.64%	9.09%	31.82%	22.73%	22.73%
Green Policy	Green Policies	0.00%	5.26%	52.63%	42.11%	0.00%
	Mentoring	2.63%	13.16%	52.63%	31.58%	0.00%
	Procurement	0.00%	7.89%	50.00%	42.11%	0.00%
	Best Practice	0.00%	15.79%	57.89%	26.32%	0.00%
	LL Concept	2.63%	10.53%	55.26%	28.95%	2.63%
	eGPP	2.63%	10.53%	57.89%	23.68%	5.26%

.

To examine how the knowledge was improved after participating in LLs among different types of organizations, Fisher’s exact test was deployed. A statistical relevant difference in relation to type of organization was found between the level of knowledge on funding topics (p = 0.017) and LL concept (p = 0.034). The results are presented in

Table 5. Fisher–Freeman–Halton Tests (topics and organization type).

Topics	Pearson Chi-Square	Likelihood Ratio	Fisher–Freeman–Halton Exact	p
Green Policies	17.196 *	18.683	18.214	0.226
Mentoring	22.291 *	25.125	20.031	0.103
Funding	30.657 *	29.660	26.136	0.017
Procurement	13.930 *	14.371	14.405	0.545
Best Practices	23.223 *	20.440	20.901	0.138
LL concept	30.892 *	22.624	23.561	0.034
Pilots	20.298 *	16.801	18.003	0.608
eGPP	15.817	19.008	16.017	0.309

* 20 cells (80.0%) have expected count less than 5. The minimum expected count is 0.07.

.

The analysis, of which the results are presented in

Table 6. Distribution across topics and organization type.

Topic	Organization Type	Poor (%)	Fair (%)	Good (%)	Very Good (%)	Excellent (%)
Funding	Public authority	0.0%	13.2%	41.5%	28.3%	17.0%
	SME	0.0%	0.0%	22.2%	50.0%	27.8%
	Higher education and research	0.0%	0.0%	33.3%	66.7%	0.0%
	Business support organization	0.0%	0.0%	33.3%	66.7%	0.0%
	Other	0.0%	50.0%	0.0%	50.0%	0.0%
LL concept	Public authority	1.9%	9.4%	45.3%	35.8%	7.5%
	SME	0.0%	5.6%	22.2%	66.7%	5.6%
	Higher education and research	0.0%	0.0%	0.0%	33.3%	66.7%
	Business support organization	0.0%	50.0%	25.0%	25.0%	0.0%
	Other	12.5%	25.0%	25.0%	25.0%	12.5%
	eGPP	1.9%	9.4%	45.3%	35.8%	7.5%

, highlights that stakeholders from higher education and research organizations have demonstrated a notably high proficiency in understanding the LL concept, with 66.7% achieving an excellent level of knowledge, significantly outperforming other types of organizations. This contrast becomes even more apparent when examining their proficiency in funding topics, where the majority (66.7%) attained very good knowledge. In comparison, SMEs led the chart in reporting excellent knowledge in funding topics, with 27.8% of them achieving these levels. Most stakeholders from public authorities reported their knowledge on funding and LL concepts as good (41.5%/45.3%). On the other hand, business support organizations have shown limitations in reaching the highest level of knowledge in both funding and LL concept topics. Their knowledge spectrum is predominantly bifurcated, with an even distribution between fair and very good levels of knowledge in the funding domain. This pattern persists in their assessment of the LL concept, where a notable 50% perceived their knowledge level as fair.

4. Discussion and Implications

Overall assessment shows that in all eight topics, the level of knowledge has significantly increased after participating in LLs, which provides evidence that participating in LLs leads to a consistent improvement in knowledge levels. We observed a more pronounced enhancement in knowledge levels after participating in an LL for topics such as understanding the Living Lab (LL) concept, best practices, green policies, pilot actions, and funding. The increase in level of knowledge in other topics was only slightly less pronounced. Although, in general, across all these categories, there has been a positive change in ratings, moving from the ‘Fair’ range closer to or into the ‘Good’ range, with some topics even reaching the ‘Excellent’ category. This suggests that participating in an LL had enhanced the level of knowledge in energy refurbishment and green procurement.

The potential reasons for this may lie in the collaborative learning environment. LLs typically foster a collaborative environment where participants can learn from each other. This exchange of ideas and experiences can enhance the learning process, leading to better retention and understanding of concepts. Also, in the case of LLs, learning regarding the energy refurbishment of buildings is put in the context of real-world applications (through pilot actions) that can make abstract concepts more tangible and easier to grasp. LLs often involve experts in the field who provide guidance, insights, and feedback to participants. Learning about energy from financial experts can significantly boost the understanding on carrying out energy refurbishment of buildings and green public procurement. Additionally, LLs most often address current trends and topics like green policies, pilot actions, and funding in the field of energy refurbishment, which are pertinent in today’s world. When participants see the immediate relevance and applicability of what they are learning, they are more likely to be engaged and retain information. The learning process is also distinguishable from “traditional teaching methods”, and the process is supported with diverse learning materials and approaches, such as case studies, pilot actions, eGPP and LCC tools, and real-world examples. Such an approach can cater to different learning styles and enhance the overall learning experience. Additionally, LLs might incorporate repetitive exercises or tasks that reinforce certain concepts. Repetition is a proven method to enhance memory and understanding.

Across many topics, participants from LLs specialized in green public procurement generally exhibited higher confidence in their knowledge after participating, especially in the “very good” and “excellent” ratings. A high level of knowledge was also achieved in LLs specialized in funding/mentoring. The potential reasons for that may be several. One of the reasons for achieving a greater level of knowledge may lie in the relevance of content provided through Living Lab sessions. Green procurement, funding, and mentoring topics might be more immediately applicable to stakeholders when performing energy refurbishment of buildings, more than, for example, green polices. The majority of stakeholders from these LLs were involved in the energy refurbishment of public buildings either as owners/leasers of public buildings or contractors. Additionally, it should be highlighted that the representation of SMEs in the Living Labs was higher compared to other Living Labs. The presence of industry (SMEs) actors provides participants with deeper insights, leading to increased confidence in their level of knowledge. The teaching methodology in these LLs leans heavily on interactive sessions, discussions, case studies, or debates that challenge participants and deepen their understanding. What is being noticed is that LLs in green funding and/or mentoring have carried out more pilot actions for energy refurbishment of public buildings compared to other LLs.

Recognizing that participants from Living Labs specializing in green procurement, funding, and mentoring have shown greater confidence in their knowledge, it is advisable to develop tailored content for LL sessions that emphasizes these topics, aligning closely with stakeholders’ professional needs. It is crucial to engage industry experts and SMEs with expertise in green funding and mentoring as leaders or contributors to these sessions. Efforts should be made to specifically attract individuals who have a pronounced interest in these areas. Employing interactive educational methods, such as pilot projects, case studies, and debates, can enhance the learning experience. Additionally, establishing robust, interactive collaboration among participants through regular meetings can deepen understanding, promote cooperation, and inspire participants to apply their new knowledge in practical energy refurbishment projects.

The analysis also showed divergence in knowledge acquisition among different organizational types, with educational and research institutions excelling in complex conceptual understanding, while SMEs show strengths in practical funding knowledge. On the other hand, public authorities and business support organizations exhibit a more moderate distribution in knowledge levels, indicating potential areas for targeted improvement and capacity-building efforts.

Reasons for different levels of knowledge regarding funding among groups may be related to the nature of each type of organization. SMEs, especially in the startup phase, often directly engage with funding mechanisms to sustain and grow their business. Therefore, SMEs are more familiar with this topic and can usually obtain and understand new funding mechanisms in this area. SMEs often need to implement best practices in energy refurbishment directly to remain competitive, efficient, and profitable via searching for possible funding mechanisms. As a result, their stakeholders might have a hands-on understanding and experience of these funding opportunities and practices, leading to higher self-assessed knowledge levels. Additionally, due to their smaller size compared to larger enterprises or large public authorities, SMEs often have the flexibility to adapt quickly to new funding possibilities and rules. While SMEs are primarily focused on operational excellence, profitability, and growth, PAs and higher education and research institutions might prioritize policy formulation, governance, research, and education over the direct implementation of business best practices. The notable difference between educational and research organizations and public authorities (PAs) on funding knowledge can be attributed to the distinct objectives and operational focuses of these organizations. The educational and research organizations are deeply invested in conducting comprehensive research and providing education, which necessitates a thorough understanding of funding mechanisms. Their academic and research endeavors require them to explore funding opportunities in depth, analyze various funding models, and understand the implications of these mechanisms on their projects and fields of study. This in-depth engagement is not merely about securing funds; it is about understanding the broader economic, social, and policy landscapes that shape funding opportunities and constraints. This contrasts with the approach of PAs, which may be more focused on policymaking, governance, and the administration of funding rather than its study and analysis. While PAs certainly engage with funding, their perspective is often more oriented towards distribution and regulation, which might not require as deep an understanding of how to use the funding mechanisms for the actual energy refurbishment of buildings.

The varying levels of knowledge about the Living Lab concept among different groups can be largely explained by the directness of engagement with Living Labs, the operational objectives of each group, and how central the Living Lab concept is to their core activities. Educational and research organizations, with their focus on in-depth study and practical innovation within Living Labs, naturally report the highest levels of knowledge. SMEs, benefiting from the practical application of Living Labs, also report high knowledge levels but with a more operational perspective. Public authorities and business support organizations, with more indirect roles in governance, policy, and support, understandably have a lower level of detailed knowledge about Living Labs.

From the acknowledgment that there are differences in knowledge levels among various stakeholders, several implications can arise. Different stakeholder groups, such as PAs, SMEs, and higher education and research organizations, have varied initial knowledge levels, indicating that the curriculum should be tailored to the specific needs and baseline knowledge levels of each stakeholder group to ensure maximum relevance and engagement. Some groups, especially public authorities and higher education entities, may have more theoretical knowledge indicating that more practical workshops and sessions should be introduced to bridge the gap between theoretical knowledge and real-world application. SMEs displayed a higher baseline understanding in certain areas. This signifies readiness for more advanced content. Therefore, offering specialized advanced sessions or modules for SME stakeholders would allow them to delve deeper into familiar topics. PAs might benefit from seeing the practical application of concepts. Organizing field trips, analyzing case studies, or inviting industry experts for guest sessions can effectively demonstrate the real-world application of LL concepts.

Several limitations were encountered during this research. One significant limitation of this study is that it evaluates knowledge improvement based on participants’ self-perception rather than through objective assessments that directly measure how much their knowledge has increased. Additionally, this study primarily focused on assessing knowledge improvement immediately following participation in the Living Labs, without conducting mid-term or long-term evaluations to determine how well this knowledge was retained and applied in practice. Moreover, while the evaluation methodology captured immediate knowledge gains, it lacked a comprehensive framework to assess the quality and depth of knowledge transfer. As a result, there may be gaps in understanding the extent to which the acquired knowledge is actionable or deeply internalized by the participants. It should also be mentioned that the methodology is designed specifically for energy-related topics and may not be relevant for a comprehensive evaluation of all aspects of the Living Labs.

Building on the findings of this article, several promising avenues for future research have been identified. A particularly intriguing area involves the development and refinement of evaluation methodologies to assess knowledge enhancement in Living Labs, by involving the experts through the Delphi method and also obtaining information on their professional skills and years of experience. In such methods, it is crucial to also consider the professional background and years of experience of the participants in Living Labs, as these factors can significantly influence the effectiveness of knowledge transfer. Additionally, a focus on diverse teaching and knowledge transfer techniques, such as case studies and pilot projects, should be prioritized to cater to different learning styles and enhance the overall impact of the Living Labs. This aspect received limited attention in the current research. Additionally, there is potential for research into how customizing approaches to suit the unique needs and characteristics of different organizational types (e.g., SMEs, public authorities, and educational institutions) can impact the knowledge outcomes for Living Lab stakeholders. The present approach in Living Labs tends to apply a one-size-fits-all strategy, disregarding the organizational diversity of stakeholders. Further research could also beneficially explore a mid-term review of the evolution of participants’ knowledge over time and assess the lasting influence of Living Labs on their professional practices and decision-making processes.

5. Conclusions

This research, reinforced by survey data, underscores that participation in Living Labs significantly improves stakeholders’ knowledge on the energy refurbishment of buildings, demonstrating a leap from basic to more advanced understanding across different sectors. Particularly, Living Labs focusing on green public procurement, financial, and mentoring aspects made notable improvement, attributing their success to an emphasis on the financial elements of energy refurbishment and green public procurement. The involvement of experts and small- and medium-sized enterprises (SMEs) further enriched these Living Labs, with different organizational types showing varied levels of knowledge gains, highlighting the importance of tailoring Living Lab approaches to the specific needs and characteristics of each organization. Despite varying levels of knowledge enhancement among Living Labs, overall, the impact on stakeholder knowledge in green procurement and energy refurbishment is significant. Future research should focus on tracking the evolution of participant knowledge over time and identifying effective self-evaluation and feedback mechanisms, ensuring the continuous improvement and adaptability of Living Labs to the needs of their stakeholders.