The Use of Energy Models in Local Heating Transition Decision Making: Insights from Ten Municipalities in The Netherlands

Abstract

In 2018, the Dutch national government announced its decision to end natural gas extraction. This decision posed a challenge for local governments (municipalities); they have to organise a heat supply that is natural gas-free. Energy models can decrease the complexity of this challenge, but some challenges hinder their effective use in decision-making. The main research question of this paper is: What are the perceived advantages and limitations of energy models used by municipalities within their data-driven decision-making process concerning the natural-gas free heating transition? To answer this question, literature on energy models, data-driven policy design and modelling practices were reviewed, and based on this, nine propositions were formulated. The propositions were tested by reflecting on data from case studies of ten municipalities, including 21 experts interviews. Results show that all municipalities investigated, use or are planning to use modelling studies to develop planning documents of their own, and that more than half of the municipalities use modelling studies at some point in their local heating projects. Perceived advantages of using energy models were that the modelling process provides perspective for action, financial and socio-economic insights, transparency and legitimacy and means to start useful discussions. Perceived limitations include that models and modelling results were considered too abstract for analysis of local circumstances, not user-friendly and highly complex. All municipalities using modelling studies were found to hire external expertise, indicating that the knowledge and skill level that municipal officials have is insufficient to model independently.

1. Introduction

1.1. The Dutch Heating Transition

In 2016, the heating and cooling sector accounted for half of the EU’s energy consumption [1]. In The Netherlands, 53% of the national heat supply is provided by natural gas [2]. In March 2018, the Dutch national government announced its decision to end natural gas extraction from the Groningen gas field by 2030 [1] to help reach the climate goals of the Paris Agreement and to reduce the negative impact of natural gas extraction in the province of Groningen [2]. This is also referred to as the so-called ‘heating transition’ in The Netherlands and was later defined by the RVO (The Netherlands Enterprise Agency) as removing natural gas from industry, the built environment and the agricultural sector [2], and replacing it by (sustainable) heating alternatives. According to the Climate Agreement, the main climate policy program in The Netherlands, a sufficient level of sustainable heating must be made available to replace the natural gas supply and to meet the climate change mitigation target of reducing CO₂ emissions by 3.4 megatons in the built environment. To reach this goal, 1.5 million existing residential homes have to be supplied with sustainable heating by 2030 [3].

However, this is challenging because decision-making and policymaking in this transition are far from simple, as actors, technology and institutions interact in a complex manner [1]. The heating transition requires a change of the supply of renewable energy, the infrastructure, residential heating systems and of thermal insulation in residential houses, which all raise questions about the division of costs and the freedom of choice [4]. Next to these dependencies, the heating transition poses significant financial challenges. Natural gas is currently cheaper than sustainable alternatives and residents do not always have sufficient funds available to provide the needed investments or to deal with increased living expenses [5].

To organise this complex transition, every municipality is expected to formulate a “Transition Vision Heat” (See

Table A1. Glossary and definitions of Dutch (policy) concepts and abbreviations used in the present study.

Concept	Abbreviation	Definition Used
Startanalyse	SA	The ‘Startanalyse’ (Start Analysis in English) is a national modelling study conducted with the Vesta MAIS model by PBL. The Startanalyse is presented together with guidelines for local analysis (Handreiking) in a guidebook (Leidraad) for Dutch municipalities.
Openingsbod	OB	The ‘Openingsbod’ (opening offer in English) is a modelling study initiated by Stedin, a Dutch network operator. The study was developed as a tool to quicken decision making in the Dutch heating transition. The study compares the modelling approach and the results of three different energy models.
Programme for Natural Gas-Free Districts	PAW	A joint programme of the Ministry of the Interior and Kingdom Relations, the Ministry of Economic Affairs and Climate Policy, the Association of Netherlands Municipalities and the Association of Regional Water Authorities that, among others, provides subsidies and requirements for the Test Beds for Natural Gas-Free Districts (pilot projects) [3].
Regional Energy Strategy	RES	“Within the RES, public authorities work alongside social partners, network managers the business community and, where possible, residents to develop regionally supported choices. The RES aims to realise the generation of renewable electricity (35 TWh), to realise the heating transition in the built environment (from fossil to sustainable sources) and to realise the necessary storage and energy infrastructure” [3].
Transition Vision Heat	TVW	The TVW is a policy document in which a municipal council has to establish a realistic schedule within which to transition away from natural gas [3]. The focus of the first TVW is on the period until 2030 and every municipality has to show which building will become natural gas free or insulated, with which electrical infrastructure and when [65].
Neighbourhood Implementation Plans	WUP	A WUP is the follow up of the TVW and indicates how a municipality will make a specific neighbourhood natural gas-free by transitioning to sustainable heating and cooking systems.
Basic registration of addresses and buildings	BAG	The BAG-dataset is a national dataset. Municipalities are responsible for providing data for the BAG-dataset, the dataset is maintained by the Dutch Cadastre, Land Registry and Mapping Agency

, Appendix A, Glossary) and an implementation plan in their local government plans, to show how they will organise a heat supply that is natural gas-free and affordable, according to the Environment and Planning Act. This means that municipalities are expected (by the national government) to take a leading role in the heating transition. This is new for municipalities and requires them to collect new knowledge, expertise and competences. To this end, the national government has set up Test Beds for Natural Gas-Free districts (i.e., pilot projects) and a knowledge and learning programme to learn and experiment [3] within the National Programme for Natural Gas-Free Districts. The latter has a 120 million euro budget.

1.2. The Use of Energy Models in Data-Driven Policymaking

To enable the heating transition, municipalities need to answer questions such as, which heating source would lead to low end-user costs, low societal costs and low CO₂ emissions? To evaluate the effect and impact of potential policy measure or decisions on, for example, a preferred technology for natural gas-free heating in city districts, evidence-based policymaking entails the derivation of fact-based knowledge to support the decision making by policymakers. One way to approach evidence-based policymaking is with data-driven policies. A data-driven policy uses data and tools for processing and analysing data to design policies and to facilitate collaboration with citizens to co-create [6]. Currently, municipalities make limited use of data and data processing and analysis tools for decision-making support. This is partly due to a lack of guidelines. New guidelines are to be developed that can make use of new data sources and tools [6]. Historically, the first decision-making support tool developed for environmental planning was the multi-criteria decision aid (MCDA). The MCDA is considered a qualitative decision support tool [7]. One drawback of MCDA tools is that they do not allow for analysis to compare whether doing an action is better than doing nothing [8]. In the last years, the number of quantitative tools to support decision-makers has been growing, which include energy models. The advantage of energy models, compared to more qualitative tools such as MCDA, include a higher degree of traceability, easier implementation in computing environments and better opportunities for ex-ante analysis [8]. Dutch municipalities are increasingly trying to include energy models when designing policy for the heating transition are energy models. In the present study, an energy model is defined as a computer model of an energy system that introduces a structured way of thinking about the implications of changing parts of the system [9]. Energy models may help analysts and policymakers to better understand the increasingly complex energy sector. However, clear guidelines on how to use these models while designing policies are still lacking.

Next to a lack of guidelines on how to integrate energy models, practitioners, such as policymakers, also experience challenges with energy models themselves. This hinders the use of energy models for policy design and decision-making [10]. When interpreting modelling results, caution is needed, because when modelling, it is unavoidable to make use of assumptions and estimates, which may not be valid under all circumstances [11,12]. According to a recently published research report, in The Netherlands [10] no less than six different models focusing on the heating transition sometimes provide different results for the same research question, due to differences in approach, assumptions and input data. This makes it difficult for policymakers to interpret, understand and trust modelling results.

Another significant challenge of current energy models is that they fail to take into account social aspects. This is problematic since the heating transition is highly dependent on humans and their intentions. Social aspects, such as behaviour and attitude of the public, affect proposed or implemented policies and should, therefore, not be ignored [13]. At present, building owners (either citizens/homeowners, institutional investors, private landlords or housing associations) have the right and responsibility to make investment decisions about the heating supply of their buildings [14]. In other words, they need to be incentivised to change their current gas-based heat supply. For this reason, building owners and local communities form an essential part of the heating system and their contribution to the heating transition, by deciding to adopt sustainable heating technologies and/or thermal insulation for their homes, is key in making the transition happen.

1.3. Research Focus

The present study focuses on the use of energy models in local heating transition projects to assess to what extent energy models are used in the decision-making process, how, and which advantages and limitations this has. The present paper aims to provide insight into the practice of energy modelling and insight into the needs and challenges of practitioners when using energy models in the heating transition. Thus far, no academic studies have addressed these issues. Insights therein can provide a starting point for more structured guidelines of effective energy modelling. The research question of this study was, therefore, as follows: What are the perceived advantages and limitations of using energy models for municipalities within their data-driven decision-making process concerning the natural-gas free heating transition? To answer the research question, a review of the literature and multiple embedded case studies was conducted in which different heating transition projects in ten Dutch municipalities were investigated. The scope was limited to energy models used by practitioners in the Dutch heating transition, as further explained in Section 3.

The paper is structured as follows. In Section 2, a literature review is presented on the use of energy models in heating transition projects, as well as on data-driven policy design and good modelling practices. Section 2 concludes with a set of theoretical propositions. In Section 3, research design and methodology are presented. In Section 4, the results of the analysis are presented. This includes testing of key propositions regarding the use of energy modelling. In Section 5, the results are discussed, and the academic merit of the present study is presented. The paper ends with a conclusion, the limitations of the study and suggestions for future research.

2. Literature Review

2.1. Data-Driven Policymaking

To plan for a transition to sustainable heating in the built environment, municipalities need data and evidence to support their decision-making processes [10]. One way to approach this is by formulating data-driven policies. Multiple studies agree that using a data-driven approach using new data sources and tools, such as energy models, can improve policymaking practices [6,13,15,16,17,18], but a systematic approach to do so is still missing [6,13]. Moreover, various studies express concerns about the capabilities of policymakers and stakeholders to deal with new data sources and technologies [16,18]. Thus far, no academic studies have been conducted addressing how the use of energy models affects practitioners within the heating transition, indicating a research gap. In addition, multiple studies call for more clear guidelines for the use of new data and tools by governmental institutions [6,13,19]. Argyrous [20] offers some guidelines on ensuring transparency and accountability, but only Koussouris et al. [17] offer concrete suggestions for practitioners besides ensuring the governmental organisation has the right expertise.

2.2. Challenges of Using Energy Models in Heating Transition Policymaking

Considering the academic literature regarding the use of energy models to support policymaking in the heating transition one thing becomes clear: there is a large variety of models and tools being used to support decision making within the energy transition, and few comparisons are being made between these models and tools. An overview of the literature found describing different modelling methods used for a sustainable heating transition is shown in

Table A2. Overview of the literature found describing different modelling methods used for sustainable heating transition projects.

Model Type	Studies	Relevant Findings for This Study
Agent-based model	[1,22,23]	These studies emphasise the importance of trying to incorporate social factors within modelling.
TIMES (The Integrated MARKAL-EFOM System) energy model (linear optimisation)	[24,25,26]	The maximum surplus assumption used in the model is often challenged.
Simulation model (using Long-rangeEnergy Alternatives Planning (LEAP software)	[27]	All stakeholders involved could run and modify the model themselves and even modify it according to their needs. The model has a large sensitivity for a multitude of assumptions
METIS simulation model	[28]	A current understanding of quantitative tools by policymakers is often missing
HOMER optimisation	[29]	Most models and tools currently used do not provide both economic and environmental analysis of energy systems, which can lead to the design of sub-optimal systems
Housing Stock Energy Model (HSEM)	[30]	Many HSEMs are lacking in transparency and modularity and that they are often limited in scope and limited in their utility. Behavioural responses are blurred in HSEMs.
Optimisation model	[31,32,33,34,35]	The results of the model will always depend on the focus of the optimisation and that there are not many models yet that can incorporate economic, environmental and social factors at the same time.
Dynamic system modelling	[36]	Analyses behaviour over time by identifying elements within the system and their mutual correlations.
PRIMES (Price-Induced Market Equilibrium System) model	[37]	The study does not seem to involve social or behavioural aspects. Analysis of energy systems is based on the inputs from GIS mapping
Eco-district heat kit optimisation model	[21]	This study attempted to make the model usable for non-experts.
Socio-Technical Energy Transition (STET) models	[38,39]	(Optimisation) Models tend to simplify their depiction of societal and political factors. STET models try to integrate both quantitative modelling and conceptual socio-technical transitions
Area-based model	[40]	Emphasises the importance of modelling at the sub-city scale as this enables, among others, more accurate quantification of demand increases.
Econometric model	[41]	Suggests that we should combine spatial attributes with econometric models.

, Appendix B (relevant findings for the present study). Reviewing this sample [1,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41] shows that although modelling approaches have the potential to reduce the uncertainty of complex social issues, there is currently no systematic approach on how to apply models to make policy decisions and how to consider not only objective facts but also social and socio-economic factors. As the complexity of heating transition projects is partly due to the dependency on social factors such as human behaviour, models which consider not only objective techno-economic factors but also social and socio-economic factors, could increase the value of modelling approaches in heating transition projects [13,22,35,38,39].

Furthermore, the literature shows a large variety of models that are currently used, based on different theories and mathematical principles. A few common challenges can be recognised among this variety. First, the correctness and sensitivity of assumptions. Second, the transparency and usability for practitioners. Third, the need to integrate both economic, environmental and social factors. Another interesting aspect concerns the lack of energy modelling research, particularly in the heating transition of The Netherlands. Thus far, in this country, only one academic study was conducted addressing a model focused on the heating transition [1].

Although there is limited academic literature available, grey literature is abundant. A whitepaper by Nikolic et al. [19] offers general principles for good modelling practice and red flags that indicate inadequate modelling practices. It concludes that there is a need for modelling guidelines that are more practical and easier to communicate, and that there is a need for more interaction between academia and practitioners. Both Nikolic et al. [19] and De Ridder et al. [42] suggest that municipalities need to develop more internal knowledge to understand and make use of models. Diran et al. [43,44] claim that better access to data regarding buildings, infrastructure and energy production is needed to utilise current energy models, especially within the utility sector. Figure 1 presents an overview of the energy models and tools regarding the heating transition as used in The Netherlands.

A study by Brouwer et al. [10] compares six models that are often used by municipalities, i.e., the Vesta MAIS (Multi Actor Impact Simulation) model, the CEGOIA model, the Energy Transition Model (ETM), a DWA model and the Caldomus model. The characteristics of these models are discussed in

Table 1. Overview of the six energy models often used by municipalities for heating transition policymaking.

Model a	Developer b	Type of Model c	Format	Availability	Geographical Scope
Vesta MAIS	PBL (English: Dutch Planning Bureau for the living environment)	Techno-economic optimisation	C++ (GeoDMS software)	Open access	National, regional, city, neighbourhood
CEGOIA	CE Delft	Techno-economic optimisation	Excel model	Model owned by CE Delft	National, regional, city, neighbourhood
Energietransitie model (ETM) (English: Energy Transition Model)	Quintel	Techno-economic simulation	Website	Open access	International, national, regional, city
Warmtetransitie model (WTM) (English: Heating Transition Model)	Over Morgen	Techno-economic optimisation	Unknown	Model owned by Over Morgen	Unknown
Integraal kostenmodel (IKM) (English: Integral cost model)	DWA (A Dutch engineering consultancy)	Techno-economic optimisation	Excel model	Model owned by DWA	Regional, city
Wijkwarmtemodel (WWM) (English: District heating model)	DWA	Techno-economic optimisation	Excel model	Model owned by DWA	Neighbourhood
Caldomus	Innoforte	Techno-economic optimisation	Excel model	Model owned by Innoforte	Regional, city, neighbourhood

^a English translation provided by the authors. ^b English translation provided by the authors. ^c Optimisation models find the optimal solution for a chosen criterion and constraints, whereas simulation models merely allow the end-user to explore how a system responds to different inputs.

. The study [10] reveals that these models provide significantly different results for the same research question due to differences in assumptions and modelling approach. Differences identified [10] include differences in building types and geographical borders, differences in renovations to improve surpassing energy label ‘B’; differences in costs of all-electric networks; differences in the order of steps within the approach; different assumptions regarding the scarcity of heat sources and different assumptions regarding learning curves; different heating technologies included; and differences in optimisation research questions.

2.3. Propositions on the Use of Energy Models by Municipalities

Based on the literature it can be deduced that clear guidelines for the use of energy models are missing and that there are serious concerns about the lack of expertise regarding energy models and data management at public organisations. Among energy models used for energy policy design, there are challenges regarding the correctness and sensitivity of assumptions, regarding the transparency and usability for practitioners (such as policymakers) and regarding the need to integrate more social factors. Moreover, although there is grey literature available, there is a lack of academic research about the use of energy models by municipalities. Based on the literature reviewed, propositions were formulated regarding current practices, advantages and limitations of municipalities using energy models in the heating transition.

Table 2. Overview of the theoretical propositions and their respective justifications.

Proposition	Justification
1. Different municipalities use different energy models (if any) with different aims.	Due to the large share of energy models available that use different approaches and assumptions and that have a different focus [10], it is expected that different municipalities will use different energy models with different aims.
2. If energy models are complex to use, then practitioners will make limited use of them while planning for the heating transition.	Current energy models are not usable for non-experts, such as practitioners [21,28]. It is therefore expected that practitioners make limited use of energy models due to the complexity of energy models.
3. If energy models do not integrate social or socio-economic factors, then practitioners will make limited use of them while planning for the heating transition.	The complexity of heating projects is partly due to the dependency on social factors such as human behaviour and that models which consider not only-objective but also social and socio-economic factors could increase the value of modelling approaches in heating transition projects [13,22,31,32,33,34,35,38,39]. Therefore, it is expected that practitioners currently make limited use of energy models because energy models currently used do not include social factors.
4. If assumptions within energy models are uncertain, than this will decrease the trust within energy models for practitioners.	The correctness and sensitivity of assumptions influence trust and willingness of practitioners to use energy models in their heating transition projects [10].
5. If data is uncertain or unavailable, then this will decrease the trust within energy models for decision making among practitioners.	More data is needed about buildings, infrastructure and energy production to utilise current energy models [44].
6. Practitioners seek the help of external parties to use and interpret energy models.	Current energy models are often not usable for non-experts, such as practitioners [21,28]. Therefore, it is expected that practitioners seek external expertise when using an energy model.
7. External parties have commercial reasons to not be transparent about their energy model design.	According to the data-driven approach and good modelling practices discussed in the literature review, models and modelling studies require a high degree of transparency [19,20]. Since many energy model developers are commercial parties, it is expected that external parties sometimes have commercial reasons to not be fully transparent.
8. Practitioners need new (in-house) expertise to effectively use energy models	Municipalities need to develop more internal knowledge and expertise to understand and make use of models [10,18,19,20,42].
9. Interactive visualisation and different interfaces for different stakeholders improve the usability of energy models.	Interactive visualisation can help in making models and their results more understandable for non-experts [17].

presents these propositions with argumentative justifications provided for each of them. Note that some of these propositions were formulated in an if-then structure to improve readability. However, this structure only has conversational implication and is not in line with formal logical implication, i.e., if X then Y” is only false in case “X” is true and “Y” is false.

3. Research Design

3.1. Embedded Case Study Research Design

To answer the research question, multiple embedded case studies were conducted. Based on the embedded case study design of Yin [46], the nine propositions formulated based on the literature review, guided design, data collection and analysis will be reflected upon [46]. In the present study, multiple cases represented a variety of heating transition projects. Key actors involved included heating transition practitioners and energy model developers. Practitioners, such as policymakers and project managers, are closely involved in the heating transition project of the municipality and/or in the development of the local heating vision document. Energy model developers are involved in the developing models that are used by municipalities.

3.2. Case Selection

The first generation of pilot projects from the National Programme for Natural Gas-Free Districts (see Table A1, Appendix A), consisting of 27 municipalities, served as an initial source of case study selection. It was predicted that these cases would produce similar results or contrasting results for anticipatable reasons. All of these projects started at a similar time in 2018, received government funding and had a similar manner of publicly documenting their progress. Differences in results between these projects are expected to be based on the size of the municipality, based on specific neighbourhood characteristics of the pilot projects and on different energy models that are used. In total, ten municipalities participated in the present study. This entailed a sample of three large municipalities (>100,000 residents), five medium-sized municipalities (>30,000 residents) and two small municipalities (<30,000 residents), across ten provinces (out of twelve provinces in the country; showing high geographical variation), with ten different approaches to natural gas alternatives analysis, and a variety of different selected heating alternatives.

Table 3. Overview of the ten case studies, presenting the size and the proposed alternative heating technology options of each of the municipalities analysed.

#	Municipality	Number of Residents (2019)	The Technological Heating Alternative Proposed for the Local Project
1	Loppersum	9614 [48]	Heating network, heat pumps and thermal energy storage [49]
2	Tytsjerksteradiel	31,780 [48]	Individual heat pumps [50]
3	Assen	67,963 [48]	Unknown [51]
4	Noordoostpolder	46,849 [48]	Heat network [52]
5	Katwijk	65,302 [48]	Aquathermic solution, Medium-Temperature Heat Network [53]
6	Rotterdam	644,618 [48]	High-temperature Heat Network (possibly later Medium-Temperature) [54]
7	Utrecht	352,866 [48]	High-Temperature heat network and heat pumps [55]
8	Eindhoven	231,642 [48]	Heat Network [56]
9	Brunssum	28,103 [48]	Low-Temperature Heat network [57]
10	Middelburg	48,544 [48]	High-Temperature Heat network [58]

presents an overview of the ten municipalities that participated and the potential alternatives for natural gas for their respective pilot projects, based on the information that was published in the project implementation reports of 2018.

3.3. Pattern Matching

To enable reflection from the empirical study to the theoretical propositions, the “pattern matching” technique was used. According to Yin [46], pattern matching is one of the most desirable techniques used in case study analysis. Pattern matching entails comparing empirically-based patterns with the predicted patterns made before collecting data, e.g., the theoretical propositions. The ATLAS.ti 8 [47] software was used to support the process of pattern matching. As there is a risk of collecting too little data with this approach [46] data were also collected on emerging themes that were present in the academic and grey literature but that were not captured in the propositions. After finalising the empirical study, each of the nine propositions will be reviewed separately and will be either confirmed or rejected based on confirmatory evidence that follows from the empirical analysis, as described in Section 3.4.

3.4. Data Collection, Treatment and Analysis

The types of data per case study that were used concerned: (1) governmental reports (for example heating transition implementation plans); (2) in-depth interviews with practitioners from municipalities and; (3) in-depth interviews with model developers. The information of these three sources was converged in a triangulating fashion. The documents (such as project implementation plans and model guidelines) provided secondary data that were used to structure the interviews. Only publicly available documents were used. Twenty-one in-depth, (expert) interviews provided primary data of the case studies.

All twenty-one interviewees were provided with informed consent forms and all interviewees provided, among others, permission for the use of their statements for the present study. An anonymised overview of respondents is shown in

Table A3. An overview of the respondents from municipal heating transition projects.

Municipality	Interviewee	Interviewee Function
1	01	Part-time project leader pilot project
2	02	Project leader pilot project
2	03	Project manager TVW
3	04	Environment manager pilot project
4	05	Project leader pilot project
5	06	Project manager TVW
5	07	Project leader pilot project
6	08	Project manager pilot project
6	09	Project manager TVW
7	10	Project leader pilot project
8	11	Process director pilot project
9	12	Project leader pilot project
9	13	Project manager TVW
10	14	Project leader pilot project

and

Table A4. An overview of the respondents from mode development firms involved in municipal heating transition projects.

Model Development Firm	Interviewee	Interviewee Function
1	15	Partner and modeller
2	16	Consultant natural resources
3	17	Senior Consultant
4	18	Consultant and technical expert model
5	19	Researcher climate, air and energy
6	20	Consultant heating transition
7	21	Director and modeller

, Appendix C. All interviews were conducted via video call or telephone, and audio was recorded. Interviews with both practitioners (14) and model developers (7) were fully transcribed. Transcripts were provided to the interviewees after the interviews and interviewees were given ample opportunity to read and alter the transcript. All interviews were conducted between the first of May and the first of September of 2020. The average duration of individual interviews was 55 min.

The interviews were semi-structured with open-ended questions to allow for in-depth analysis. Although a set of pre-defined questions was used, interviewees were also given the opportunity to explore questions in greater depth and to introduce new topics. This type of in-depth interviews, according to Roller [45], increases the credibility of the data by reducing response bias (distortion due to the tendency of interviewees to provide answers that are considered socially accessible) and by reducing satisficing (providing an easy ‘I do not know’ answer). The data collection process, including the informed consent forms, was approved by the Ethical Committee of the Technology, Policy and Management faculty at Delft University of Technology.

Analysis of the interview transcripts was completed by thematic coding. Atlas.ti 8 [47] (computer-aided qualitative data analysis software) was used to perform the coding process and to create coding reports. A semantic analysis was conducted, meaning that data was coded at face value, i.e., at the explicit meaning. Thematic coding is viewed as a relatively simple qualitative method that offers a high level of flexibility. Quotations were created based on the theoretical propositions and the research questions, and a code was assigned to each quotation. As proposed in standards for theoretical thematic analysis [59], an initial set of codes was set-up to guide analysis of the transcripts. The coding frame, as expected, did not fully cover all aspects related to the topic and was adapted and supplemented where needed with codes such as ‘motivation residents’ and ‘not familiar with energy models’. These adaptations were made rather inductively, meaning that the ‘open coding’ function of Atlas.ti 8 [47] was used to add codes during the first round of coding. After this first round of coding, all codes and their frequency were assessed to see whether splitting or merging of codes was necessary. To transform the raw data into meaningful information, all quotes were given an English title; code groups were created to show the relation between several codes and so-called network figures were created to show the focus of different quotes within one code. Moreover, code-occurrence tables (see

Table A5. The code-occurrence table shows an overview of the 37 thematic codes, the respective code frequencies and the number of transcripts that quotes were identified in.

Code	Code Frequency	# Transcripts
Approach	149	11
Third-party expertise	79	13
Coupling opportunities	70	13
Model/modelling study used	65	13
Motivation residents	52	12
Analysis tools used	51	12
Project progress	51	13
Data	43	9
Collaboration	42	13
Participation activities	42	12
Information Respondent	37	13
(Envisioned) natural gas alternative	35	10
Incentivizing Residents	30	5
Financial arrangement residents	27	9
Added value pilot project	25	1
Limitations approach:	22	9
Future approach	21	8
Limitations model: Vesta	19	5
Responsibility municipality	17	7
Added value model: CEGOIA	13	3
Limitations model: CEGOIA	11	4
Limitations models in general	11	5
Other reasons to opt for a heating alternative	8	3
Added value analysis tool: Resident questionnaire	7	1
Added value modelling study: Openingsbod	6	2
Limitations model: DWA	4	1
Added value model: Caldomus	4	2
Added value model: DWA	3	2
Added value modelling study: Startanalyse	3	3
Limitations model: Caldomus	3	1
Added value analysis tool: Greenvis	2	1
Not familiar with energy models	2	2
Added value analysis tool: Resident meetings	1	1
Limitations analysis tool: Resident game	1	1
Added value analysis tool: Susteen	1	1

and

Table A6. The code occurrence table shows an overview of the 53 thematic codes, their respective code frequency and the number of transcripts that quotes were identified in and the code group.

Code	Code Frequency	# Transcripts	Code Group
Modelling and Consultancy approach	100	7	Consultancy and modelling approach
Collaboration	68	7	Collaboration and Competition
data	46	7
Challenges modelling approach	45	6	Consultancy and modelling approach
Inclusion of socio-economic factors	43	7	Consultancy and modelling approach
Information about assumptions	37	7	Consultancy and modelling approach
Information about parameter sensitivity	34	7	Consultancy and modelling approach
Information about input data	32	7	Consultancy and modelling approach
Feedback channels	30	6	Consultancy and modelling approach
Result interpretation	28	6	Consultancy and modelling approach
Coupling opportunities	22	5	Consultancy and modelling approach
Users and Uses of Vesta MAIS	19	2	Uses and Users models
Limitations Vesta MAIS model	18	5	Limitations model
Information respondent	17	5
Advantages modelling approach	17	6	Consultancy and modelling approach
Information Caldomus	15	2	General information model/modelling study
Users and uses ETM	13	2	Uses and Users models
Information model ETM	13	1	General information model/modelling study
Limitations Startanalyse	13	4	Limitations modelling stud
Advantage Vesta MAIS model	13	2	Advantages model
Influence of municipality size	12	4
Limitations CEGOIA model	12	4	Limitations model
Information DWA Model(s)	11	1	General information model/modelling study
Planned changes Caldomus model	11	1	Limitations model
Connection RES, TVW, WUP	12	3	Consultancy and modelling approach
Competition	9	2	Collaboration and competition
Information model: CEGOIA	9	2	General information model/modelling study
Information Startanalyse	7	3	General information model/modelling study
Limitations Caldomus model	6	1	Limitations model
Planned changes Startanalyse	6	2	Limitations modelling stud
Advantage ETM	6	1	Advantages model
Information WTM	6	1	General information model/modelling study
Advantage Startanalyse	6	2	Advantages modelling study
Information Vesta Mais	5	2	General information model/modelling study
Limitations ETM	5	1	Limitations model
Users and uses of CEGOIA	5	1	Uses and Users models
Uses and users Startanalyse	5	2	Uses and users modelling studies
Advantage Openingsbod	4	2	Advantages modelling study
Users and uses WTM	4	1	Uses and Users models
Advantage CEGOIA model	3	1	Advantages model
Planned changes ETM	3	1	Limitations model
Planned changes CEGOIA model	3	1	Limitations model
Users and uses of the Caldomus model	3	1	Uses and Users models
Advantage WTM	3	1	Advantages model
Users and Uses DWA model(s)	3	1	Uses and Users models
Information Openingsbod	2	1	General information model/modelling study
Limitations DWA model(s)	2	1	Limitations model
Advantage Caldomus model	2	1	Advantages model
Uses and Users Openingsbod	2	1	Uses and users modelling studies
Limitations WTM	1	1	Limitations model
Information modelling study: Openingsbod	1	1	General information model/modelling study
Advantage DWA model(s)	1	1	Advantages model

in Appendix D and Appendix E) were made to quantify the findings, which reduced the subjectivity of result interpretation.

4. Results

The interviews conducted with practitioners yielded 820 quotes divided over 36 thematic codes. Seven interviews were conducted with model developers. These interviews yielded 561 quotes divided over 53 thematic codes (See for an overview of codes and code-occurrence Appendix B and Appendix C). The results of the case studies were used to either validate or reject the propositions (Section 2.3; Table 2). The findings regarding the testing of the propositions are presented in

Table 4. An overview of the findings that confirm or reject the propositions made.

Proposition	Confirmed/Rejected
1. Different municipalities use different energy models (if any) with different aims.	confirmed
2. If energy models are complex to use, then practitioners will make limited use of them while planning for the heating transition.	confirmed
3. If energy models do not integrate social or socio-economic factors, then practitioners will make limited use of them while planning for the heating transition.	rejected
4. If assumptions within energy models are uncertain, then this will decrease the trust within energy models for practitioners.	unclear
5. If data is uncertain or unavailable, then this will decrease the trust within energy models for heating transition decision making of practitioners.	unclear
6. Practitioners seek the help of external parties to use and interpret energy models.	confirmed
7. External parties have commercial reasons to not be transparent about their energy model design.	unclear
8. Practitioners need new (in-house) expertise to effectively use energy models.	confirmed
9. Interactive visualisation and different interfaces for different stakeholders improve the usability of energy models.	confirmed

. The findings will be discussed in more detail in Section 4.1, Section 4.2, Section 4.3, Section 4.4, Section 4.5, Section 4.6, Section 4.7 and Section 4.8 below.

4.1. Different Municipalities Use Different Energy Models with Different Aims

The proposition ‘Different municipalities use different energy models (if any) with different aims’ was confirmed based on the case studies. Six different energy models were used by the ten municipalities studied to support decision-making for heating transition pilot projects or the design of the Transition Vision Heat: the Vesta MAIS model, the CEGOIA model, the Caldomus model, DWA models (the IKM and the WWM), the ETM and the WTM). This is in line with [10] which mentioned these six models as the most used models for the Dutch heating transition. Moreover, two national modelling studies based on one or more of these energy models were used, the ‘Startanalyse’ (Start Analysis in English; translation by the authors) and the ‘Openingsbod’ (Opening Offer in English; translation by the authors) (see Table A1, Appendix A). In the case studies, these models were only seldom used by practitioners, with the only exception in this sample pertaining the municipality of Utrecht, where a modelling team was deployed to use the Vesta MAIS model to develop heat scenarios. More in general, municipalities were found to use models and modelling studies to support the decision-making process, to provide more legitimacy towards residents or as a basis for more detailed heating transition business cases. No socio-technical energy transition modelling methodologies or agent-based modelling methodologies were found though, indicating that these were not considered important in the current planning and implementation of the heating transition at the local level. All models, except for the ETM, were optimisation models. The ETM did not offer an automated optimisation function. All models, except for the ETM, aimed to find the heating alternative with the lowest societal costs.

4.2. Complexity and User-Friendliness of Energy Models

The propositions ‘If energy models are complex to use, then practitioners will make limited use of them while planning for the heating transition’, and ‘Practitioners seek the help of external parties to use and interpret energy models’ were confirmed based on the case studies analysed. The results showed that the use of energy models was not necessarily limited, seven out of the ten heating transition pilot projects investigated used an energy model in their decision-making process and seven out of seven Transition Vision Heat projects used or were planning on using an energy model. However, six interviewees mentioned there were issues regarding the complexity and user-friendliness of energy models that hindered effective usage in heating transition projects. Four out of seven model developers claimed that practitioners often did not have the right background or the time to master these complex tools independently. According to the same four interviewees, large-sized municipalities usually had more time and resources to learn how to use a model than their small-sized peers. If a third party conducted the modelling process, large-sized municipalities were therefore generally better able to critically reflect on the results. All seven municipalities from this sample that used energy models in their heating transition projects used third parties at some point during their heating transition projects to conduct modelling studies. Third-party expertise was used at all scope levels, Regional Energy Strategy development (see Table A1, Appendix A), Transition Vision Heat development and pilot projects. Municipalities were found to hire external parties to provide modelling calculations, home inspections, modelling result interpretation or to provide studies, for example into available heat sources. These findings confirm that there are indeed challenges with the complexity and user-friendliness of energy models and that these are usually overcome by seeking help from external parties.

4.3. Integration of Social or Socio-Economic Factors into Energy Models

The proposition ‘If energy models do not integrate social or socio-economic factors, then practitioners will make limited use of them while planning for the heating transition’ was rejected based on the case study analysis. All fourteen practitioners interviewees agreed that social and socio-economic factors are important and influence the success of heating transition projects. Three municipalities were found to use social or socio-economic data or information or were planning to use this to identify coupling opportunities (opportunities to combine activities for the heating transition with other improvement opportunities in a neighbourhood, such as sewer system updates, building renovations or traffic alterations), and two municipalities used or were planning to use social or socio-economic information to determine the prioritisation of neighbourhoods for heating transition activities. On the other hand, none of the practitioners or model developers interviewed claimed that social or socio-economic factors influenced the choice of heating alternatives, which is the focus of the six energy models municipalities of the sample used. The choice of heating alternative was based on the lowest societal costs in all municipal heating transition projects within the present study. All seven energy model developers agreed that that social, political and psychological aspects influence heating transition projects. However, all claimed that these factors should not and/or could not be included in their respective models and that it would be better to consider these factors alongside the techno-economic modelling results in energy modelling studies.

4.4. Unavailable Data and Uncertain Assumptions

The proposition ‘If assumptions within energy models are uncertain than this will decrease the trust within energy models for practitioners’ could neither be confirmed nor rejected based on the empirical results. Energy model developers were found to use different assumptions, and two energy model developers claimed that these are usually the reason why results between different energy models differ. Practitioners offered critiques of assumptions of models or modelling studies, in particular about assumptions regarding energy labels and the use of renewable gas. However, the impact this had on trust in energy models did not become clear in the interviews. The interviews showed that if practitioners did not agree with assumptions used in models or modelling studies that they requested model developers to change said assumptions or that they opted for a different model that used different assumptions. All seven model developers stated that they tried to be transparent about the assumptions they used and that, in collaboration with the practitioners, assumptions can be altered during the modelling process.

The proposition ‘If data is uncertain or unavailable, then this will decrease the trust within energy models for heating transition decision making of practitioners’ could not be confirmed nor be rejected. Data played an important role for municipalities and model developers in developing heating transition plans, and even though data was sometimes unavailable, this study offered no proof that this decreased the trust of practitioners in energy models. If municipalities decided to use a model, this energy model proved to be more useful if it was fed with local data. Unavailable data that could be useful according to practitioners and model developers is data about energy use per connection, data about the willingness to pay of residents and data about the potential impacts on the electricity grid. One energy model developer mentioned that the data collection process at public organisations was too time-consuming and two energy model developers mentioned that they ran into issues with the energy use data available from Statistics Netherlands (‘CBS’ in Dutch). These data were aggregated due to privacy laws and was often deemed too inaccurate to use for heating transition projects. Similarly, two energy model developers and one practitioner stated that the data from the Basic registration of addresses and buildings (BAG) (See Table A1, Appendix A) regarding energy labels provided too little insight into the level of thermal insulation present at residential houses. One of the most uncertain data sets used for heating transition projects was data about available heat sources. All model developers agreed that the datasets for heat source data were uncertain and that extra research was always needed to assess the local situation. However, whereas four energy models used the availability of heat sources as a determining factor for the choice of a natural gas alternative, two models did not use heat source availability as a determining factor.

4.5. The Use of Third Party Modelling Expertise

The proposition ‘Practitioners need new (in-house) expertise to effectively use energy models’ was confirmed based on the case studies. Only one municipality was yet capable of modelling scenarios individually. Others relied on the modelling expertise of third parties. Even if a municipality outsourced the modelling process, a minimum knowledge level was required to correctly interpret and critically reflect on results. According to energy model developers, practitioners, with only a few exceptions, did not meet this minimum condition. This also caused practitioners to propose incorrect or unsuitable research questions to model developers.

The proposition ‘interactive visualisation and different interfaces for different stakeholders could improve the usability of energy models’ was also confirmed based on the case studies. Three energy model developers had developed interactive models, maps or tools that, according to them, helped clients such as practitioners to better understand and interpret the modelling results. No statements from practitioners were gathered on the advantages of interactive models.

The proposition ‘External parties have commercial reasons to not be transparent about their energy model design’ could neither be confirmed nor be rejected. Two energy model developers stated that it was not always possible to gain access to underlying assumptions, data and parameters of models from other commercial agencies. However, all six models in this study were compared to each other in the benchmark study [10], indicating that model developers were at least willing to be transparent towards independent researchers. Moreover, one national modelling study compared the results and underlying assumptions, datasets and parameter sensitivities of multiple models (of which two were commercial). Besides, transparency was only mentioned as a limiting factor by one practitioner. Hence, one could state that even though transparency, especially at commercial model developers, could be improved, it did not seem to be a limiting factor for municipalities to use energy models.

4.6. Advantages and Limitations of Using Energy Models

According to the academic literature, energy modelling can aid in decision making and policymaking because it introduces a structured way of thinking about the implications of changing parts of the system [9]. The case studies provided more concrete benefits and limitations of using energy models for decision making in the Dutch heating transition. Practitioners stated that the use of energy models within heating transition projects provided perspective for action, financial insight, transparency and legitimacy, concrete propositions to residents and sparked useful discussions. Besides, one practitioner stated that nationally available modelling studies provided validation and robustness of (other) modelling results. Most of these advantages are related to creating public support for policy. Practitioners also mentioned limitations of using energy models. Interviewees argued that energy modelling results were considered too abstract, too general or too simplified for local analysis. In addition, models were considered not user-friendly and complex. Practitioners mentioned that modelling results provided no insight into available heat sources, limited insight into the impact of nearby heat networks and no or limited insight into end-user costs. Another challenge mentioned was that the Statistics Netherlands (‘CBS’ in Dutch) neighbourhood definitions do not provide a logical division of the city, which, among others, created the need to conduct a reality check after modelling to filter out odd results, especially for the utility sector.

4.7. Collaboration with Housing Associations, Network Operators and Citizen-Led Energy Cooperatives

Moreover, from the case studies, insights were gathered that suggest that collaboration with housing associations and network operators is important during heating transition projects to prepare implementation plans and to find coupling opportunities. Housing associations were considered important as they often have property within the municipality and because they have renovation plans that may or may not align with the municipal heating transition plans. Network operators were considered important because they are responsible for underground infrastructure and network reinforcements. Therefore they have to be made aware of the municipal heating transition plans, and they have to provide input about the current limitations of the infrastructure for specific heating options. Moreover, citizen-led energy cooperatives play an important role in heating transition pilot projects. In five out of thirteen interviews, it was mentioned that collaboration with citizen-led energy cooperatives is considered important. In one small and one medium-sized municipality, energy cooperatives even provided project leaders for heating transition pilot projects. For Transition Vision Heat development at larger municipalities citizen-led energy cooperatives were found to exercise less influence. Close collaboration with energy model developers happened only in municipalities that have established modelling teams that model energy systems independently; for this sample, those included the two largest municipalities (>300,000 residents).

4.8. The Use of Comparative Analysis and Multi-Model Ecologies

As mentioned, different models sometimes result in different outcomes, which can create confusion and uncertainty at practitioners. One practitioner interviewed explicitly mentioned experiencing such confusion. Three model developers of this sample actively used comparative analysis to reduce this issue, and one national energy modelling study, the ‘Openingsbod’, also offered comparative analysis. In such an analysis, differences in methodology, assumptions, data and results of different energy models or modelling studies are compared to one another. This indicated where result differences originate from and provided an overview of the robustness of results across models. One practitioner claimed that the latter helped in determining a priority of neighbourhoods to start with heating transition projects.

Finally, three practitioners mentioned the challenge of matching up heating transition plans at different levels of abstraction, which were found to influence each other and that were sometimes developed simultaneously and with different energy models. To decrease this challenge, one energy model developer tried to position his model in such a manner that he could assess how plans would fit together. This energy model developer envisioned a multi-model ecology in which their model provided a broad energy perspective and where other energy models would offer more detailed calculations on, for example, heating transition visions, heating transition business cases and the effects on power networks.

5. Discussion

5.1. Reflection vis-à-vis the Academic Literature

The present study has provided a more concrete image of the role of energy models in data-driven policymaking and decision-making in the heating transition. The literature review showed that modelling approaches have the potential to reduce the uncertainty and complexity of heating transition projects. The present study provided a concrete overview of the advantages of using energy models in heating transition decision making as experienced by practitioners and model developers. The advantages found seem to indicate that although energy models do not necessarily make a heating transition project less complex, they at least offer means to make legitimate choices. The advantages identified are in line with the advantages of data-driven policy design mentioned by Koussouris et al. [17] who stated that tools such as energy models, simplify decision-making processes, even under complicated conditions, by facilitating the opportunity to model complex processes and the opportunity to collaborate with different actors involved, and those mentioned by Adam et al. [15] who stated that providing evidence for the effectiveness of policy choices is one of the cornerstones of legitimate policymaking.

The results of the present study could provide a starting point for recommendations targeting policymakers and model developers to facilitate more effective use of energy models in heating transition decision-making. Such targeted recommendations were not found in the literature and could help towards designing a systematic approach for integrating energy models in data-driven policymaking, which is needed and currently lacking [6,13].

Moreover, the results of this study suggest that offering comparative model analysis would help practitioners to deal with the myriad of sometimes contrasting models, modelling studies and modelling results available and that setting up a multi-model ecology might decrease the challenges of aligning heating transition projects at different abstraction levels. This is in line with Manfren et al. [60] who state that multi-model ecologies could help in creating the integration between top-down and bottom-up modelling perspectives. Furthermore, it aligns with Nikolic et al. [61] who state that multi-model ecologies help get a more coherent and less biased understanding of the ”right thing” to do in energy transition decision making as using multiple models allows multiple perspectives to be explored and be brought together.

Although this study confirmed certain advantages of using energy models it also shed light on the limitations of using energy models for decision-making. Designing modelling scenarios is considered a time-consuming and costly task. Modelling results are not absolute truths but rather results subject to calculation rules and assumptions, and if a model or its outcomes are incorrect, one might be worse off than when not using a model to begin with [19]. According to energy model developers interviewed in the present study, not all practitioners understood the limitations of energy models and interpreted modelling results as absolute truths.

Finally, the literature review suggested that it is problematic that current heating transition models do not include social and/or socio-economic factors, as the transition is highly dependent on humans and their behaviour [13]. However, the present study showed that practitioners were not always sure how social or socio-economic data should influence the choice of a heating alternative or the prioritisation of neighbourhoods. Moreover, accessing these data was sometimes difficult due to privacy restrictions. Model developers did not see added value in including social or socio-economic factors within their heating transition models, which all had a techno-economic focus. Their models were focused on finding the lowest societal and/or end-user costs for different heating alternatives and did not include social factors, as affordability for residents is seen as one of the main challenges of the Dutch heating transition [5]. The costs of a heating alternative are, as far as known, not only depending on social or socio-economic factors. Something that could be depending on such factors, for example, concerns the degree of participation and technology adoption rates.

In the present study, not one municipality was found using model methodologies focused on assessing social interactions, such as Agent-Based Modelling, System Dynamic Modelling or Socio-Technical Energy Transition Modelling. Instead, municipalities used models with a mere techno-economic focus and assessed social and socio-economic data alongside the results of these modelling efforts to identify coupling opportunities and/or to determine prioritisation of neighbourhoods.

5.2. The Influence of National Agreements and Municipality Size

All municipalities that provided information about their Transition Vision Heat planning design in the present study used or were planning to use models/modelling studies. This was expected as it was agreed in the national Climate Agreement of 2019 [3,62] that municipalities would use the ‘Startanalyse’ and its guidelines [63] to design their Transition Vision Heat. According to the Climate Agreement, this would provide all stakeholders with a “uniform frame of reference regarding the impact of the various natural gas alternatives in a district” [3]. This agreement might have incentivised municipalities to use energy models when designing their Transition Vision heat. However, three pilot projects did not use energy models to choose a natural gas alternative. The pilot projects analysed, all started before this statement was made in the climate agreement and before the ‘Startanalyse’ and its guidelines [63] were published. Therefore, practitioners in pilot projects might have been less familiar with available models and modelling studies, might have had less access to models and modelling studies and/or might have been less incentivised to use available models or modelling studies.

Secondly, pilot projects that did not use an energy model to choose a heating alternative had a few things in common. All three pilot projects were located in villages with less than 2000 residents. All of them had active citizen-led energy cooperatives, two pilots were organised by the local energy cooperative, two pilot project leaders were not familiar with energy models, and two pilot projects entailed only or mostly detached houses, from before 1940 with poor thermal insulation levels. Two practitioners claimed that they did not feel that they needed an energy model because the choice for a heating alternative could be made with common sense and information about the residential characteristics. This indicates that an energy model might not always be considered necessary or desirable for heating transition decision-making and that it is important to consider when the use of an energy model would be beneficial and when other sources of evidence might be sufficient to support decision-making.

6. Conclusions

6.1. Answering the Research Question

This study aimed to answer the research question ‘What are the perceived advantages and limitations of using energy models for municipalities within their data-driven decision-making process concerning the natural-gas free heating transition?’. To answer this question, a literature review and embedded multiple case study research were conducted, which included different heating transition projects in ten Dutch municipalities.

Results inter alia show that energy models observed in the present study were mostly initiated and used by consultancy agencies to support Dutch municipalities in designing heating transition plans. Over half of the municipalities analysed were found to use models or modelling studies at some point during their respective heating transition pilot projects. All cases that provided information about local Transition Vision Heat development were using or planning to use models or modelling studies for the design of their vision document.

Models that were used pertained to the CEGOIA model, the Vesta MAIS model, DWA models, the ETM and the WTM. Modelling studies that were used concerned the ‘Openingsbod’ and the ‘Startanalyse’. Municipalities that did not utilise models or modelling studies for their pilot projects belonged to the four smallest municipalities analysed in the present study, indicating a negative relation between municipality size and model usage. All municipalities that used models or modelling studies requested external expertise at some point during the modelling process, indicating that the knowledge and skill level at municipalities was not sufficient to do this independently. This was confirmed by model developers who also stated that the knowledge level of practitioners is often insufficient to interpret results of modelling studies conducted by third parties.

Advantages of using models in heating transition projects mentioned in the interviews were that the modelling process and its results provided perspective for action, financial and socio-economic insights, transparency and legitimacy towards residents, concrete propositions for residents and means to start useful discussions. However, interviewees also mentioned several limitations. First, models and modelling results were found too abstract, too general or too simplified for local analysis, not user-friendly and were considered complex. Results were difficult to interpret for non-experts such as practitioners, and interactive models could provide practitioners with a better understanding of the answer and help with getting a feeling for parameter sensitivity. Second, modelling results provided too little insight into end-user costs and the effects on the electricity grid. Third, data sets regarding energy use, thermal insulation levels and heat sources proved to be insufficient for local analysis, and there was no consensus between model developers and practitioners about the different assumptions regarding green gas availability and energy labels used in different models.

This study also showed that model developers deemed it unpractical to integrate social and socio-economic factors in the energy models discussed, but agreed that this data should be incorporated in modelling studies/reports. Model developers usually did this by collecting social or socio-economic data and by presenting this data next to the modelling results to provide context for further decision-making.

Finally, the results suggest that offering comparative model analysis would help practitioners to deal with the myriad of sometimes contrasting models, modelling studies and modelling results available and that setting up a multi-model ecology might decrease the challenges of aligning heating transition projects at different abstraction levels.

6.2. Limitations

The external validity of the empirical results is limited by the context in which the present study was conducted, in selected municipalities in The Netherlands. This was a scoping choice motivated by the case study design and time constraints of the present study. The representativeness of these results to other geographical, political and cultural contexts might therefore be fairly limited. It is expected that representativeness will particularly be limited for countries where the heating transition is not organised in a decentral manner or where there are not multiple (national) energy models available to analyse the costs of this transition.

Limited access to background information on some commercial energy models limited the reflection on technical aspects of the models reviewed in the case studies. In the present study, the capabilities, limitations, underlying assumptions of models were only compared at the surface level, based on publicly available reports and the challenges and advantages mentioned by interviewees. This limited access to background information limited the potential for in-depth model comparison. On the other hand, the time constraints of this research and the focus on user experiences and the modelling process rather than the actual energy models also limited this potential. This choice was made because limited access to the background information of (commercial) models was foreseen and because there are already other studies, such as [10], that focus on in-depth model comparison.

The data collection tools chosen, interviews and thematic coding, also have their respective limitations. Interviews and thematic coding are research tools that require a high degree of interpretation from the researcher. During the coding process, quotes had to be translated and interpreted. The literal transcripts, the coding process and the coding reports ensured quotes were methodologically analysed and that it was possible to review the original quotes.

The present study used multiple sources of evidence in a triangulating fashion to decrease the subjectivity of the answers and to check their consistency over time. A remarkable observation was that within the pilot projects observed the views and plans of interviewees did not always align with the views as exhibited in the implementation plans of the pilot project, due to advancing insights.

6.3. Recommendations for Future Research

The present study did not provide an answer as to when heating transition projects should and when they should not use energy models to guide their heating transition decision-making process. The discussion offered some criteria that might indicate projects that do not need energy models such as municipality size, residential housing characteristics and the presence of an energy cooperative. It is therefore recommended to conduct more research into which criteria could indicate that projects would have an advantage of using an energy model. It is recommended to conduct more case studies, with different types of heating transition projects, to explore this topic. In addition, it is suggested to also include case studies that utilise other decision support tools, such as MCDA tools, in order to assess the relative advantages and limitations of energy models when compared to other tools.

Furthermore, it is recommended to further study the impact of social and socio-economic factors. The literature review revealed that that social and socio-economic factors are highly important for heating transition decision-making processes, but currently, the impact of social and socio-economic data within Dutch heating transition projects is limited and at best influences the prioritisation of neighbourhoods. More research into certain factors, for example, income or the presence of energy cooperatives, could provide insight into the correlation of these factors with heating transition project progress and into the potential value of models that include such factors. Such insights would not only benefit the Dutch heating sector but might also benefit a range of international energy transition projects. On the one hand, this might entail desk research into socio-technical transitions and models (such as Socio-Technical Energy Transition, System Dynamics or Agent-Based models). On the other hand, it might address practical case studies that test socio-technical transition theories and models within heat or energy transition projects. Ideally, such case studies are not restricted to The Netherlands but also include projects in countries with significantly different heating systems, energy markets, institutions, social and socio-economic values to compare and corroborate results.

Finally, it is recommended to conduct more research into the field of multi-model ecologies (e.g., systems of interacting models). The present study has shown the need for comparative analysis, for modelling at different abstraction levels and for assessing the impact of choices regarding the heating transition in other disciplines, such as electrical infrastructure and social welfare. More research into multi-model ecologies can benefit both the Dutch and the international academic modelling field as it offers the opportunity to add value to existing models, for example by making them more interactive with other national or international models. Nikolic et al. [61] and Manfren et al. [60] offer the first set of principles, challenges and guidelines that provide a conceptual basis for multi-model ecologies. Currently, the ‘Mondaine Suite’ project [64] is one of the first projects that is aiming to realise a multi-model ecology by developing a coupling mechanism for different (Dutch) energy models. However, this project does not yet couple Socio-Technical Energy Transition models, System Dynamics or Agent-Based models, which might offer an interesting opportunity for future research to include more social and behavioural components into multi-modal ecologies.