An Assessment of the Key Performance Indicators (KPIs) of Energy Efficient Retrofits to Existing Residential Buildings

Abstract

Quantifying the wider benefits of energy efficient building retrofits is crucial to incentivise householder retrofit investments. This research recognises the value of key performance indicators (KPIs) for assessing and demonstrating retrofitting benefits and provides an assessment of KPIs for evaluating retrofits. An integrated framework for evaluating retrofits using a set of economic, social, and environmental KPIs is proposed. This KPI framework is then applied in a pre- and post-retrofit assessment of five case study dwellings located in Ireland, revealing its usefulness in demonstrating the wider benefits of retrofitting to householders, with a view to driving retrofit investment. Three of these case study dwellings had state-of-the-art retrofit technologies installed as part of the works, including heat pumps and solar PV systems. In addition to demonstrating the wider benefits of retrofitting, the framework allowed for the identification of potential causes for differences in performance of these technologies across households, as well as patterns of underperformance. Such insights are useful for the future design of these technologies and retrofit packages, as well as policy measures, which support householders in the adoption and use of these measures. The results demonstrate that householders experience various benefits from retrofitting. Showcasing the different benefits that householders receive from retrofitting, and their satisfaction with the retrofit works, can serve to de-risk retrofit investments, and inspire others to seek similar benefits through retrofitting. Applying the developed framework to a larger, comparable sample size, can distinguish the retrofit packages, which perform best across the KPIs and various household profiles. Furthermore, the application of the developed framework can serve as an evidence base for retrofit designers, contractors, and policy makers in the design of retrofit packages and policy measures that will maximise the benefit for householders.

1. Introduction

1.1. Background and Challenges to Retrofitting Buildings in Ireland

Improving the energy efficiency of building stock in the European Union (EU) will play a key role in achieving the “carbon neutrality by 2050” target set by the European Green Deal [1]. To achieve this objective, the energy retrofit rate of the building stock in Europe will need to—at least—double the 2020 retrofit rate [1]. This is a significant challenge facing many countries, including Ireland. Ireland’s retrofit activity during most of the last decade was low, averaging at approximately 23,000 (primarily shallow) retrofits per annum from 2013 to 2019 [2]. This is a rate of 1.2% of the existing residential building stock per annum. In recognition of the pressing need for increased investment in this area, the Irish government’s Climate Action Plan 2021 has set ambitious targets for the achievement of 500,000 residential energy efficiency retrofits to a building energy rating (BER) [3,4] of B2 or better by 2030, at an average rate of 50,000 per year [2]. However, only 2600 residential buildings retrofitted in 2019 achieved a B2-BER (energy performance of at least 125 kWh/m²/year) or better [5].

Meeting the proposed targets will require convincing homeowners to invest in retrofit measures. Ultimately, however, a homeowner’s decision-making process to invest in an energy efficiency retrofit is highly complex, prevented by many barriers, and enabled by various personal and contextual influences, such as a householder’s socioeconomic characteristics and dwelling specific characteristics [6,7]. Persistent barriers include the high upfront costs of measures [8] and uncertainty regarding the payback period [9] of retrofit investments. This is further complicated when householders lack the financial resources to cover the cost [8] and, thus, only invest in retrofit if financial subsidies are available [10], with funding structures often being highly complex [7]. Moreover, the disruption and stress often associated with retrofitting also contribute to low retrofit uptake rates [7], while others argue that the limitations of the traditional retrofit delivery model majorly contributes to the slow diffusion of comprehensive energy retrofit [11]. Additionally, behavioural, rational, and cognitive limitations often inhibit householder understanding of retrofitting benefits; including the lack of an energy efficiency culture, lack of information, and lack of awareness [7,12,13,14]

1.2. Retrofit Benefits and the Use of KPIs

Despite the challenges of retrofitting buildings, there are also multiple benefits [15]. These are not limited to energy efficiency alone [16], but include economic, environmental, and social benefits [17,18]. In the economic sense, these include the effect of retrofitting on the building’s market value and reductions in operating costs [17,18,19,20]. Environmental benefits include energy consumption savings and emission reductions [18,21,22]. From the householder perspective, social benefits can include improved occupant satisfaction, comfort, and health [15,17,18]. Furthermore, a more energy efficient domestic sector has the potential to resolve a variety of social policy issues, such as fuel poverty and deprivation, public health, and high rates of winter mortality [18,23].

The literature has demonstrated that several efforts have been made to evaluate the impacts of retrofits, although these efforts are severely limited by their over-reliance on a narrow set of economic, techno-centric, and environmental indicators, with much focus placed on optimizing the technical and economic performance of dwellings [24]. The importance of economic influences, such as cost and profitability, on a householder’s decision to invest in a building retrofit is demonstrated across the literature, such as in [25]. However, not all homeowners decide to retrofit based on economic considerations alone. Recent research found that the majority of Irish homeowners consider the comfort impacts of energy efficiency investments, with many prioritising comfort as the main driver for taking action in the Irish context [26]. Wider literature on homeowners’ energy efficient investment decisions across Europe echoes such findings that homeowners retrofit choices are influenced by economic benefits, as well as a range of other benefits that may potentially arise, such as comfort improvements and environmental benefits [27].

Ultimately, the net benefit obtained from retrofitting is the main determinant of the householder’s decision to engage in retrofit works [13]. In other words, the householder’s decision to retrofit is in itself a trade-off between the benefits obtained, and the costs involved. Creating an evidence base of these benefits is important, to inform retrofit policy decisions, de-risk retrofit investment from a householder perspective and motivate the uptake of deeper retrofits by demonstrating the tangible benefits of such [28]. Irish policy has stressed the importance of effective communication of the multiple benefits of retrofitting in incentivising householder investment in retrofitting [29]. However, as pointed out by Wilson et al. [30], the non-monetary influences on a householder’s retrofit investment decision are systematically understudied.

Therefore, the aim of this paper is to develop an integrated framework of key performance indicators (KPIs) which are to be used to demonstrate the wider benefits of retrofitting to householders, with a view to encouraging homeowners to invest in retrofitting. KPIs are used to assess the performance of the critical project goals and are considered especially useful for dealing with complex contexts such as the sustainability assessment of construction projects [31,32]. Moreover, given its applicability for the performance assessment of retrofitted buildings, the KPI approach is fast becoming one of the most valuable tools for the quantification of the benefits of retrofit implementation [31,33]. This paper, firstly, provides an assessment of the KPIs used for evaluating retrofits, based on a review of the existing literature. Based on the critique of existing studies and models, this research develops an integrated framework of KPIs for evaluating the economic, social, and environmental benefits of building retrofits that should be included in retrofit evaluations at a minimum. This integrated framework is then applied to five case study dwellings in the west of Ireland, to reveal its usefulness in demonstrating the wider benefits of retrofitting to householders, with a view to encouraging homeowners to invest in retrofitting. Finally, the implications of applying the proposed framework for policy and practice are presented.

2. Literature Review: Key Performance Indicators (KPIs) in Retrofit Projects

This section presents a discussion of the economic, social, and environmental KPIs prevalent in literature, regarding energy efficient retrofit projects. The literature review focused on studies that (i) specifically focus on developing, classifying, and prioritising the KPIs of retrofit projects from various stakeholder perspectives; (ii) incorporated and used KPIs in the development of retrofit design decision-making frameworks; and (iii) assessed the performance and, subsequently, the benefits received from retrofits, either in post-occupancy evaluation (POE) studies [34] or in pre- and post-retrofit evaluation studies. Such studies referred to in (iii) may not specifically refer to performance measures as KPIs; however, they are regarded as KPIs in this study. The KPIs identified for demonstrating the multiple benefits of retrofitting to householders are categorised as shown in

Table 1. KPI categories identified in the existing literature for demonstrating the multiple benefits of retrofitting to householders.

	Economic		Social				Environmental
Category Ref	Cost	Market Value	Indoor Environmental Quality (IEQ)	Health and Well-Being	Fuel Poverty	Satisfaction with Retrofit	Energy	Carbon	Energy Performance Certificate (EPC)
[17]	●
[18]					●	●	●
[22]	●						●	●
[31]	●					●	●	●
[33]	●						●	●
[35]	●						●	●
[36]	●						●
[37]	●						●
[38]	●						●
[39]	●						●
[40]	●				●		●	●
[41]	●				●		●	●
[42]	●		●				●
[43]	●		●					●
[44]		●
[45]			●				●
[46]			●
[47]			●	●	●		●
[48]			●	●	●
[49]			●				●
[50]			●				●	●
[51]			●				●	●
[52]			●			●
[53]			●
[54]			●
[55]				●		●
[56]				●
[57]				●
[58]				●
[59]				●
[60]						●
[61]							●	●

. The categories identified in Table 1 are considered the core categories that should be included at a minimum for demonstrating the wider benefits of building retrofits to householders. The KPIs identified for each category are those that are common among the existing literature. While health and wellbeing are considered a core category, no specific health and wellbeing KPIs have been included, given the lack of a standardised methodology for their assessment.

The justification for including various KPIs based on the reviewed literature are discussed in the following sections, while a summary of the recommended KPI categories to use in evaluating energy efficient retrofits to existing residential buildings from a householder’s perspective are summarised in

Table 2. Recommended key performance indicators (KPIs) of energy efficient retrofits to existing residential buildings.

	KPI Category
Economic	Investment cost
	Life cycle costs (LCC)
	Payback period
	Change in market value
Social	Change in Indoor environmental quality (IEQ)
	Change in health & well-being
	Change in fuel poverty
	Satisfaction with retrofit
Environmental	Energy consumption savings
	CO2e emissions savings
	Change in Energy Performance Certificate (EPC)

.

2.1. Economic KPI Categories

A significant number of studies have emphasized the prevalence of economic influences on a householder’s decision to retrofit, with many of the structural barriers and drivers of retrofit investment being related to the cost structure [8]. This section discusses the various economic KPIs, which are prevalent in the existing literature, and those included in the KPI framework.

2.1.1. Cost

Firstly, the investment cost was identified as a KPI across various studies [17,22,31,33,38]. Typically, the minimisation of investment cost is an objective function in the design of retrofit solutions [17,22,31,33,38] and in pre- and post-retrofit evaluation studies [39,40,41]. From the householder perspective, the investment cost is considered an important KPI, given that a householder’s willingness to invest in retrofit often depends on the level of benefit they receive relative to the investment required [43]. Additionally, the high investment costs associated with energy efficiency upgrades and a householder lacking the financial resources to undertake a retrofit are major obstacles [16,17,62,63,64,65]. Therefore, the investment cost is included as an economic KPI, as it provides evidence of the various types of benefits householders could receive, depending on the level of investment they are willing to commit to the retrofit. Indirect costs, such as stable labour costs, static materials costs, and minimum variations costs have been included as economic KPIs in other studies [31,33], but are excluded as part of the investment cost KPI used in the framework.

Life cycle cost (LCC) is frequently included in the economic assessment of retrofits [22,31,33,35,42] and is commonly used to compare the investment cost of a retrofit with the economic benefit of alternative retrofit design solutions [17]. Moreover, LCC analysis is mandatory for the achievement of cost-optimal levels of energy performance, as required under the energy performance of buildings directive (EPBD) [66]. From a householder perspective, the LCC KPI is considered useful, as it indicates the economic benefit a householder will receive over a certain time period following a retrofit [67], thereby making it possible to identify the most cost-effective retrofit strategy [42]. Given its usefulness in this regard from a householder’s perspective, the LCC is included as an economic KPI in the cost category in the KPI framework.

Other economic KPIs prevalent in the literature include the payback period and net present value (or net present cost) [21,31,33,35,43]. For the KPI framework, the net present value KPI is not included as a distinct KPI as it is accounted for in the LCC analysis. The payback period measures the length of time necessary to recover any investments made [68]. This is often considered a crucial factor to speculative householders, who may prioritise a fast recovery of their retrofit investment [67]. A reasonable payback time is often considered a motivator for retrofitting [68,69], while on the other hand, both the perceived lengthy payback periods and lack of knowledge with regards to payback periods associated with retrofits are some of the frequently cited barriers to householder investment in retrofit [16,17,70]. Therefore, from a householder perspective, the payback period is considered a useful economic KPI [22], and is generally considered an accessible KPI that can be easily understood by householders. However, while the payback period is often used as an additional KPI [21,35,36,37], its inherent limitations do not make it appropriate for use as a distinct KPI, as it does not take into consideration any benefits or costs that occur after the payback period [67]. Therefore, it is better used in conjunction with more accurate indicators, such as the net present value or in this case the LCC, which consider any benefits that occur after the payback period and provide a better measure of the investments profitability [66,67]. Additionally, care should be taken in handling the payback period, as it may result in an over-emphasis on how quickly householder investments can be recovered, meaning that householders may favour less efficient retrofit investments, with short-term paybacks, as opposed to more efficient retrofit investments, with long-term paybacks [67]. Nonetheless, its usefulness is recognised, and it is therefore included as part of the KPI framework.

2.1.2. Market Value

A frequently discussed benefit of retrofitting and, subsequently, an important driver of householder retrofit investments, is the resulting increase in the market value of property [44,69]. Economic theory suggests that householders will place increased value on energy efficiency, and will pay a price premium for more energy efficient homes, anticipating lower energy bills and increased comfort [67]. However, despite its importance as a driver for retrofit investment, and as an important economic benefit to the householder, it is frequently excluded from economic evaluation methods [67]. While Reuter et al. [71] suggest that a market value multiple benefit indicator is more applicable to commercial buildings rather than residential buildings, other existing research such as those papers reviewed by [44] has suggested otherwise. Laquatra and Carswell [67] believe that market value should be included in economic evaluations to allow householders to make optimal decisions in energy efficiency investments and is included as part of the KPI framework.

Overall, the four economic KPIs selected are investment cost, LCC, payback period and market value. While it is recognised that these indicators do not present a great deal of novelty, the importance of accessibility and ease of householder understanding justify their selection.

2.2. Social KPI Categories

The measurement of social benefits of retrofitting is less developed than the economic and environmental benefits [55]. Those that are prevalent in existing literature and included as part of the KPI framework are discussed in the following sections.

2.2.1. Health and Well-Being

One of the most important social benefits of retrofitting is the positive health impacts associated with it [55]. In fact, the relationship between energy efficiency and health and well-being is one of increasing research interest. The general focus of such research is on the effects of improved indoor environmental quality, and in particular indoor air quality on occupant health, such as sick building syndrome, asthma, allergies, and other respiratory conditions [55]. Several studies reviewed sought to quantify these physical and mental health impacts of building energy efficiency [23,47,48,56,57,58,59].

Coyne et al. [18] point to the difficulty associated with evaluating the health impacts of retrofitting, in that these health effects often take a longer time to emerge in a measurable way as compared with the other benefits of retrofit, suggesting that reliable measurement would require a significant period of time of data collection. No standard approach or KPI exists to evaluate health and well-being, as this depends on the physical or mental health condition under consideration. Therefore, a KPI is not included as part of the presented KPI framework, and has not been included in the evaluation results presented in Section 5 due to data limitations. However, other studies, where possible, should demonstrate the health and well-being benefits of retrofitting to householders.

2.2.2. Indoor Environmental Quality

The social impacts of retrofitting include those that are experienced or felt in a perceptual (cognitive) or corporeal (physical, bodily) sense [55]. In this regard, improved indoor environmental quality (IEQ) is considered a social benefit of retrofitting. IEQ KPIs have been used in pre- and post-retrofit evaluations and post-occupancy evaluations in the literature [46,49,51,52,53,54]. In fact, most of the post-occupancy evaluation protocols reviewed in [34] included IEQ measurements. These IEQ KPIs include indoor air quality (IAQ), thermal comfort, visual comfort, and acoustic comfort.

Improved occupant comfort has been cited as a social benefit of retrofitting buildings [17]. Many studies have assessed thermal comfort improvements resulting from retrofitting buildings, in both pre- and post-retrofit evaluations, and post-occupancy evaluation studies [45,46,47,49,51]. Such studies assessed thermal comfort using either measured temperature and relative humidity (RH) sensor data, occupant surveys or a combination of both. Moreover, thermal comfort parameters were frequently included in retrofit design decision-making models [22]. It is also regarded as an accessible KPI that can be easily understood by householders. Improved comfort has also been identified as one of the factors for Irish householder’s willingness to invest in retrofit measures [25]. Therefore, thermal comfort is considered a KPI in the IEQ category.

The World Health Organisation has issued guidelines for pollutants to monitor to assess IAQ [72]. Retrofitting can have both positive and negative consequences on the indoor environment. Thus, evaluating IAQ KPIs, including increased concentrations of indoor air pollutants as a result of increased air tightness post-retrofit, is important and incorporated as part of the KPI framework [73,74,75,76]. Lighting and acoustics have also been evaluated in both pre- and post-retrofit evaluations [51,52] and are also included as part of the KPI framework. However, IAQ, lighting and acoustics are not evaluated in the case study presented in this paper due to a lack of data.

2.2.3. Occupant Satisfaction

Perceptual measures, such as stakeholder satisfaction, are often included as social KPIs [31]. At the building level, stakeholder satisfaction relates to occupant satisfaction. It is widely accepted that failing to address and incorporate occupant needs and expectations into building retrofits, has severe negative effects on the outcomes of the retrofitting efforts and occupant satisfaction [60]. Additionally, identifying and assessing the sources of occupant dissatisfaction is equally important to determine priority focus areas in retrofit design phases [60]. Effective occupant satisfaction measurement tools, therefore, address all factors related to the building occupants’ needs, activities, and goals [77].

Research has stressed the importance of positive feedback loops on stimulating further or new adoption of retrofit measures, while implying the potential impacts of negative feedback in creating barriers to such [78]. In other words, householders who have had previous positive renovation experiences (or heard accounts of positive renovation experiences), will be more likely to invest in retrofitting, while the opposite happens for those who have had previous negative renovation experiences (or heard accounts of negative renovation experiences) [25,78].

Various aspects of the retrofit process and the experience of the retrofit such as the efficiency and flexibility of the process, the level of control householders had in the process, or the level of inconvenience caused throughout the retrofit have been evaluated in the literature [18,25,78]. The level of information provided to the householders throughout the retrofit process, the satisfaction with the contractors involved [25], and the householder’s satisfaction with the implemented retrofits and the perceived benefits of such [18,25,54], have also been evaluated.

Therefore, KPIs assessing householder satisfaction regarding (i) the benefits received as a result of a retrofit; (ii) the retrofit process, quality of the works, and level of disruption (if any); (iii) the level of engagement with contractors in both the retrofit design and the handover process; and (iv) the retrofit technologies implemented are included in the core category of satisfaction in the KPI framework.

2.2.4. Fuel Poverty

Fuel poverty, also referred to as energy poverty, is increasingly being incorporated into pre- and post-retrofit evaluations of retrofit projects [18,40,41,47], with alleviation of fuel poverty being an important additional benefit of building energy efficiency [71]. Fuel poverty refers to the inability of householders to adequately heat or power a home and afford the energy services that ensure basic living standards [79,80,81]. This is considered the most significant negative social impact of domestic energy inefficiency [81]. In fact, the Building Performance Institute of Europe found that, as of 2015, between 50 and 125 million people in the EU experienced fuel poverty, and were unable to afford proper indoor thermal comfort [82]. The European Green Deal highlights the issue of fuel poverty by identifying it as an issue to be addressed as part of a ‘renovation wave’ [79]. Given the prevalence of fuel poverty, and its centrality in energy efficiency policy actions in Ireland and beyond, it has been included as part of the KPI framework.

2.3. Environmental KPI Categories

The most common environmental KPIs throughout the literature were energy and carbon emissions related KPIs [31].

2.3.1. Energy and Carbon Emissions KPIs

Almost all studies reviewed incorporated energy related KPIs into their environmental assessments [18,22,33,34,40,41,47,49,50]. Energy performance calculations are essential for the calculation of environmental impact and emissions [66]. Carbon emission related KPIs have also been included in retrofit design decision-making frameworks [22,31,33], and retrofit performance evaluations [35,40,41,43,50,51].

While energy and carbon emissions savings are a well-discussed benefit of retrofitting, the influence of environmental benefits as a motive for householders’ decisions to undertake retrofit projects is questionable, with conflicting findings being present in the literature. For example, some research has found that often, it is the non-energy related benefits that motivate householders to invest in retrofitting [26,83], with environmental benefits being of little concern or irrelevant in the decision [25,84,85]. On the other hand, however, some research has found evidence that environmental benefits and concerns are relevant in a householder’s decision to invest in retrofitting [9,12,27,86]. As such, this would suggest that demonstrating the energy consumption and emissions savings resulting from retrofit implementation may result in increased engagement by environment and climate motivated householders. Increasing energy efficiency and targeting emissions on a global scale is crucial. Moreover, energy models in the design phase must be validated [50], and the potential rebound effects of retrofitting, which form key barriers to residential energy efficiency must be examined [67]. Thus, energy and carbon emission related KPIs have been incorporated as core KPI categories in the developed framework.

2.3.2. Energy Performance Certificate

Energy performance certificates (EPCs) are seen as tools for providing clear and reliable information to homeowners and tenants to compare and assess the energy performance of buildings [87], encourage owners to invest in improving the energy efficiency of the building through the provision of cost-effective retrofit measures [87], and assist governments in developing policies to achieve national energy reduction targets in the building sector [88]. Countries use various scales and labels to represent the energy performance of dwellings to building occupants [89]. As some homeowners may struggle to grasp the energy demand savings level based on numerical energy demand values, the energy savings KPI should be complemented using an established certification to demonstrate the energy savings in a simpler way to a homeowner. It can also serve to highlight to homeowners interested in retrofitting their homes the potential impact of various retrofit measures and/or encourage homeowners to invest in additional retrofit measures. Therefore, an energy performance certificate rating is considered a core category of the KPI framework.

2.4. Summary of Literature Review Findings

As is evident from the literature review presented in Section 2.1,Section 2.2,Section 2.3, there are a lack of studies, which evaluate energy retrofit projects using economic, social, and environmental KPIs. Typically, existing studies evaluate economic and environmental KPIs, while neglecting social KPIs. Existing studies also tend to focus on evaluating one or two categories of KPIs, but very rarely, evaluate several categories of KPIs, within an integrated framework. Thus, an integrated framework for evaluating the wider benefits of dwelling retrofits, consisting of economic, social, and environmental KPIs (Table 2), is proposed and assessed in this study. The methods used to evaluate these KPIs are detailed in Section 3. This framework is applied in a pre-and post-retrofit assessment of five case study dwellings located in Ireland, to reveal and validate its usefulness in demonstrating the wider benefits of retrofitting to householders. Further detail of these case study dwellings is provided in Section 4.

3. Methodology

3.1. Data Collection

Quantitative and qualitative data, obtained from pre- and post-retrofit monitoring of case study dwellings, were used to evaluate the KPIs. There were four main forms of data collection during the pre-retrofit and post-retrofit monitoring of the case study dwellings, including: (i) surveys on the physical characteristics of the buildings; (ii) pre-retrofit and post-retrofit participant surveys; (iii) installation of temperature, relative humidity (RH), and electricity consumption data-logging instrumentation; and (iv) monthly readings of electricity and oil energy usage. At least four temperature and RH data loggers were installed in each case study dwelling. Details of the procedure followed to recruit potential research participants is given in Appendix A.

Pre- and post-retrofit, face-to-face semi-structured surveys were conducted with an adult (aged 18 years or older) in each household. Information was gathered on the householders’ demographic profiles, their attitudes towards energy use and conservation, quality of life and the environment, which items they viewed to be necessities or luxuries, their energy-related practices, and their thermal satisfaction within their homes. The level of householder satisfaction with various aspects of the retrofit was also gathered through these surveys. It should be noted, however, that while a large data set was obtained through these surveys, only data obtained on (i) householder demographics and socioeconomics; (ii) householder’s thermal satisfaction within their homes; (iii) household self-reported energy use; and (iv) satisfaction with various aspects of the retrofit are used in the evaluation of the proposed KPIs. Analysis of the remaining data were outside of the scope of this study. Further details on the other type of data collected as part of the semi-structured surveys are given in with the data used to examine the impact retrofitting has on the energy cultures and the sustainable-related outcomes among inhabitants of social housing units in Ireland [90].

3.2. Methods for Assessing Economic Indicators

3.2.1. Investment Cost

The investment cost for each retrofit was calculated by obtaining from each household the total cost borne by the household and the level of grant (if any) which householders received from granting authorities. From this information, the total investment cost for each retrofit design was calculated.

3.2.2. Life Cycle Cost (LCC)

The LCC was determined based on the methodology for the ‘calculation of the global cost in terms of net present value for each reference building’ [91]. A discount rate of 4% was used and a building life span of 30 years was assumed. The variables calculated as part of the LCC method included the investment cost and the annual operational energy consumption costs. Maintenance, repair, and replacement costs were excluded from the analysis. The annual operational energy costs were calculated based on the estimated annual pre- and post-retrofit energy consumption combined with the price per kWh for each fuel type. The life cycle energy consumption costs account for future fuel prices. Future fuel prices were estimated using an average annual percentage increase of fuel price per kWh.

3.2.3. Payback Period

The discounted payback period was calculated using a discounted payback method [92] and a discount rate of 4%.

3.2.4. Market Value

Price premiums for dwellings with better energy performance certificates (EPCs) have been noted in various countries (for example, Wales [93], Italy [94], Northern Ireland [95], and Ireland [44]). There are standard assessment procedures for the calculation of EPCs for member states across Europe. Market value improvements are estimated by calculating the improvement of EPC rating in each dwelling, and determining the resulting improvement in market value, using secondary data on the price premiums in particular housing markets, for each improvement in EPC-ratings.

In Ireland, a dwelling’s EPC is referred to as its building energy rating (BER). In the present study, the BER scale for residential buildings in Ireland has 15 ratings ranging from A1-G. Lyons et al. [44] found that each rating decline along the BER scale has a reduction of 1.3% in the sale value of property in Ireland. Assuming each rating increase in the BER scale results in a 1.3% value increase in residential buildings following a retrofit, the increase in market value is determined. In this case, secondary data specific to Ireland was used. While this was useful in this instance, it is acknowledged that such an approach may not be applicable to other countries.

3.3. Methods for Assessing Social Indicators

3.3.1. Indoor Environmental Quality

While many studies have assessed thermal comfort improvements as a result of retrofit projects, the quantification of comfort is often challenging due to the inherent subjectivity of its nature [23]. Therefore, a combination of quantitative temperature and RH data from data loggers and qualitative data from occupant surveys have been used in this study.

Two types of indicators were assessed with the quantitative data. The first compared the amount of time the average temperature and RH fell outside of recommended temperature and RH values in each of the residential buildings. The recommended values for temperature and RH for residential buildings used in this assessment, as specified in [96], ranged from 18 °C to 26 °C and 30–60% RH, respectively. The percentage of temperature and RH readings recorded as either below and above the lower and upper temperature and RH limits were determined and compared pre- and post-retrofit.

The second type of quantitative indicator compared the average temperature and RH in each dwelling during pre- and post-retrofit heating seasons. The average temperature and RH were taken as the average temperature and RH recorded by the data loggers in each dwelling. In addition to this quantitative data, qualitative data on thermal comfort were obtained by asking householders to rate their satisfaction with the thermal environment, as part of pre- and post-retrofit semi-structured surveys, using a seven-point scale from ‘very satisfied’ to ‘very dissatisfied’.

3.3.2. Fuel Poverty Alleviations

Whether the householders were in fuel poverty was determined using the ‘expenditure method’ [97], whereby householders were considered as suffering fuel poverty if their estimated annual fuel expenditure was greater than 10% of their annual income. Data on the householders’ annual income were gathered in the pre- and post-retrofit semi-structured surveys. Households’ annual fuel expenditure was based on the annual operational energy costs calculated as part of the LCC. Predictions were made as to whether householders would experience fuel poverty alleviations over 30 years, at both the pre- and post-retrofit fuel consumption levels, taking into consideration the annual average percentage increase in fuel costs over 30 years, and the annual rate of income inflation.

3.3.3. Householder Satisfaction

Householder satisfaction was assessed via both the face-to-face pre- and post-retrofit semi-structured surveys, and follow-up semi-structured surveys conducted in October 2019. This was assessed in terms of their satisfaction with (i) the benefits received as a result of the retrofit; (ii) the retrofit process, quality, and disruption (if any) caused; (iii) the level of engagement with contractors in both the retrofit design and the handover process; and (iv) the technologies implemented.

3.4. Methods for Assessing Environmental KPIs

3.4.1. Annual Energy Demand Savings

Heating energy data obtained from case study dwellings were normalised using heating degree days (HDDs) based on the external temperatures experienced in each case study dwelling’s locality, during the periods of time the heating data represented. It was not possible to disaggregate the amount of heating energy that accounted for space heating and water heating purposes in this study.

The normalised heating energy data (kWh/HDDs) for each phase (i.e., pre-retrofit and post-retrofit works) was multiplied by the average HDDs experienced in the case study dwellings locality during the heating season, for a period of 10 years beginning in 2005, to estimate an annual heating energy usage. A base temperature of 15.5 °C was assumed in this analysis. Base temperatures are considered the temperatures at which no heating is required for buildings to maintain comfortable indoor temperatures [98].

Electricity and oil energy usage levels were recorded once a month. For the purposes of the analysis, six months of pre-retrofit electricity and oil energy consumption data, and six months of post-retrofit electricity and oil energy consumption data have been utilised. Householders were asked to estimate their annual solid fuel costs during pre- and post-retrofit surveys. Solid fuel annual energy usage (including timber, briquettes, coal, and turf), was estimated using fuel costs per kWh for the relevant fuel type [99].

Electricity usage for lighting and appliances (that is, other than for space heating) was normalised to kWh/day and multiplied by the number of days in a year to determine the annual energy use. Specific details on how annual energy use for electricity was calculated in specific case studies where electricity usage is also used for space heating is outlined for each specific case in Section 4.1,Section 4.2,Section 4.3,Section 4.4,Section 4.5. By subtracting the post-retrofit energy demand from the pre-retrofit energy demand, the estimated annual energy demand savings were determined for each case.

3.4.2. Annual CO₂ Emissions Savings

Using the estimated annual energy consumption pre- and post-retrofit and multiplying each fuel type by its corresponding CO₂ emissions factor in kgCO₂/kWh [98], annual CO₂ emission savings were calculated. The annual rate of decarbonisation of the electricity grid should also be accounted for in the results.

3.4.3. Energy Performance Certificate

There are standard assessment procedures across EU member states for the assessment of building EPCs. Further details on the use of the standard assessment procedure in Ireland for this case study are given in Section 4.

4. Case Study

Data collected from a pre- and post-retrofit study of five case study dwellings in the west of Ireland are used to demonstrate the wider benefits of retrofitting based on the economic, social, and environmental KPIs of the integrated framework. Details of the procedure followed to recruit potential research participants is given in Appendix A. The five detached rural case study dwellings, otherwise referred to in this paper as Case A–Case E, were retrofitted as part of SEAI’s Better Energy Community scheme [100]. Each case study received varying levels of grant aid for their retrofit works from the SEAI (see

Table 3. Post-retrofit KPI results for each of the case study buildings. Values in brackets are the ^a percentage change of post-retrofit results relative to the pre-retrofit results or ^b pre-retrofit result.

KPI Category	KPI (Unit)	A	B	C	D	E
Cost	Investment (EUR)	45,739	25,000	10,000	9000	23,000
	Grant aid (%)	35	90	90	90	90
	Life cycle cost-with grant aid (EUR/m2) a	892 (−39%)	480 (−48%)	746 (7%)	955 (−13%)	809 (10%)
	Life cycle cost-without grant aid (EUR/m2) a	1062 (−27%)	671 (−27%)	830 (20%)	1067 (−3%)	1101 (50%)
	Payback period-with grant aid (years)	9.6	0.4	N/A	1.4	N/A
	Payback period-without grant aid (years)	15.5	13.3	N/A	23.2	N/A
Market Value	Market Value a	N/A (9.1%)	N/A (7.8%)	N/A (3.9%)	N/A (2.6%)	N/A (7.8%)
Thermal Comfort	Avg. Temp. (°C) a	19.8 (19%)	20.3 (−4%)	18.1 (6%)	16.4 (1%)	18.9 (7%)
	Avg. Temp. (% < 18 °C) b	7 (−34%)	11 (9%)	37 (−17%)	66 (−3%)	21 (−31%)
	Avg. Temp. (% > 26 °C) b	0 (0%) (0%)	2 (−4%)	0 (0%)	0 (0%)	0 (0%)
	Avg. RH (%) a	56 (−10%)	45 (5%)	56 (1%)	58 (0%)	66 (−3%)
	Avg. RH (% < 30%) b	0 (0%)	2 (1%)	0 (0%)	0 (0%)	0 (0%)
	Avg. RH (% > 60%) b	48 (−29%)	23 (21%)	53 (17%)	56 (5%)	65 (−10%)
	Survey	VS (VD)	VS (GS)	GS (NSD)	VS (NSD)	VS (VS)
Health and Wellbeing	-	-	-	-	-	-
Fuel Poverty	Fuel poverty status (Yes/No) b	No (No)	No (No)	Yes (Yes)	Yes (Yes)	Yes (Yes)
Satisfaction with retrofit	Benefits received	VS	VS	VS	NSD	S
	Retrofit process and quality of work	NSD	VS	VS	VS	GS
	Contractor engagement	GDS	NSD	VS	NSD	NSD
	Technology	NSD	VS	VS	GDS	NSD
Energy	Secondary (kWh/m2) a	98 (−71%)	152 (−70%)	228 (−3%)	412 (−21%)	202 (−59%)
	Primary (kWh/m2) a	233 (−65%)	212 (−66%)	289 (−1%)	503 (−20%)	369 (−43%)
Carbon	Emissions (CO2/m2) a	50 (−59%)	61 (−66%)	74 (−8%)	150 (−23%)	61 (−62%)
EPC	Theoretical (BER) b	C2 (G)	C2 (F)	D2 (C2)	D1 (E1)	C1 (E2)
	Measured (BER) b	D1 (G)	C3 (G)	D2 (D2)	G (G)	E2 (G)

VS—very satisfied, GS—generally satisfied, S—satisfied, NSD—neither satisfied nor dissatisfied, D—dissatisfied, GDS—generally dissatisfied, VD—very dissatisfied. Avg.—Average.

). Retrofits were completed between July and November 2016, with Case A, Case B, and Case E having photovoltaic (PV) systems installed in February 2017.

For the calculation of thermal comfort improvements in the dwelling, using the method described in Section 3.3.1, temperature and RH data loggers were installed in each case study dwelling, in the kitchen, living room, and three bedrooms of each dwelling. The data loggers used were ‘Easy Log EL-USB-2+’. It should be noted that temperature and RH readings recorded in one of the bedrooms in Case C were removed from the analysis due to the sensor recording erroneous data. Temperature and RH data were recorded every 15 min, both pre- and post-retrofit, from December 2015 to February 2018. The temperature and RH data are based on data recorded from the 16 December 2015 to 25 May 2016 (pre-retrofit) and the 15 December 2016–18 May 2017 (post-retrofit). The possible effects of external temperature on the thermal comfort results were considered, using historical weather data that was obtained from the Claremorris weather station, which is within 30 km of the case study dwellings [101]. However, detailed analysis of the effect of external temperature on temperature improvements resulting from retrofitting was outside the scope of this study.

For the calculation of fuel poverty alleviations, using the methodology described in Section 3.3.2, the annual rate of income inflation in the Irish context was taken as 1.0 %, based on data obtained from the Central Statistics Office (CSO, Cork, Ireland), for the period 2008–2019 [102]. To take into consideration any uncertainty with regard to the income inflation rate, future predictions were also made at income inflation rates of 3% and 5%.

For the calculation of annual energy demand savings, using the methodology described in Section 3.4.1, electricity and oil energy usage levels were recorded once a month. For the purposes of the analysis, six months of pre-retrofit electricity and oil energy consumption data (16 December 2015–25 May 2016) and six months of post-retrofit electricity and oil energy consumption data (15 December 2016–18 May 2017) have been utilised. Householders were asked to estimate their annual solid fuel costs during pre- and post-retrofit surveys. Solid fuel annual energy usage (including timber, briquettes, coal, and turf), was estimated using fuel costs per kWh for the relevant fuel type [99]. In this instance, fuel prices were sourced from SEAI’s domestic fuel cost archive [99]. The annual operational energy costs are inclusive of 13.5% VAT [99]. To determine the average annual price increase rate, fuel prices from 1998 to 2018 were sourced from SEAI’s domestic fuel cost archive [99]. Fuel prices for wood and turf were only available from periods 2005–2018 and 1999–2003, respectively. External temperature data for Claremorris (Ireland) was used to normalise this heating energy data, using the method described in Section 3.4.1. In this instance, a base temperature of 15.5 °C in Ireland was assumed by SEAI when using the HDD method [98]. Electricity was used for lighting and appliances in all cases. Specific details on how annual energy use for electricity was calculated in specific case studies where electricity usage is also used for space heating is outlined for each specific case in Section 4.1,Section 4.2,Section 4.3,Section 4.4,Section 4.5.

In the calculation of annual CO₂ emissions savings, using the methodology described in Section 3.4.2, the annual rate of decarbonisation of the Irish electricity grid was taken from [103] and accounted for in the results.

For the calculation of pre- and post-retrofit EPCs for each case study dwelling, the standard assessment procedure for Ireland was used. A BER was assessed using a standard assessment procedure referred to as Domestic Energy Assessment Procedure (DEAP). The theoretical energy demand was estimated based on the version of DEAP (when it was known as the Dwelling Energy Assessment Procedure) published in 2012 [104]. The following version, published in 2019 [105], provides a more detailed procedure for assessing the theoretical domestic hot water energy demand. Some of the information required for the new procedure was not collected during the building inspections. A BER was determined pre- and post-retrofit.

Using the estimated annual energy consumption pre- and post-retrofit and the level of energy demand associated with each BER ranging from A1-G, BERs based on the measured energy consumption were determined. The electricity demand for appliances was included when assessing the BER even though it is not accounted for in a theoretical BER estimation. This represented the energy demand for a full year, whereas the theoretical BER is based on the energy demand for the heating months (October–May).

Figure 1 shows the cases pre-retrofit with a description of the cases and the retrofit measures given in Section 4.1,Section 4.2,Section 4.3,Section 4.4,Section 4.5.

4.1. Case A

Case A is a solid masonry wall house (94 m² heated floor area) constructed in the 1960s occupied by a family of four, including two children. The house underwent substantial work, including retrofit measures to the building fabric and heating system. Double-glazed PVC windows and doors replaced single pane timber windows and doors; 100 mm expanded polystyrene external insulation and acrylic render were added to the solid masonry external walls. Electric storage radiators and instantaneous domestic hot water heaters were replaced with an air source heat pump (ASHP), heating controls, radiators, and a hot water tank. In addition, a 2.1 kWp PV system with an inverter was installed.

It should be noted that Case A used electricity for space heating and water heating pre- and post-retrofit (as well as for lighting and appliances). Case A had an ASHP installed as part of the retrofit works. In this case, to normalise the space heating energy using HDDs (as described in Section 3.4.1), it was assumed that 61% of the electricity usage for Case A was related to space heating, based on the finding that 61% of end-use residential energy in Ireland is accounted for by space heating [98]. The remaining 39% of electricity energy usage was normalised to kWh/day and multiplied by the number of days in a year to determine the annual energy use.

4.2. Case B

The retrofit works of Case B primarily focused on the heating system. The two-storey detached house (118 m² heated floor area) had an ASHP, heating controls, radiators, hot water tank, 2.1 kWp PV system and inverter installed. The new electricity-based heating system replaced an oil-based central heating system. The owner of the 1960s detached house complemented the oil-based heating system pre-retrofit with a solid fuel stove and maintained the option of using the solid fuel stove post-retrofit. The occupancy of the home also changed following the retrofit with one of the occupants (a woman of more than 75 years of age) passing away, leaving the son aged between 46 and 55 years of age as the sole occupant.

As the retrofit work consisted of changing the oil-based space and water heating system to an electricity-based ASHP, it was important to differentiate the electricity consumption for heating, lighting, and appliances, post-retrofit. To do this, the pre-retrofit daily electricity usage for lighting and appliances was assumed to be the same post-retrofit. Thus, the difference between the pre-retrofit and post-retrofit energy usage was assumed to be the post-retrofit space and water heating energy required in Case B. Case B also had a solid fuel stove acting as a secondary heating system that remained in operation post-retrofit. Data from the electricity meter for Case B was unavailable for the post-retrofit stage of the study. Thus, electricity bills from April 2018 to March 2019 were used to estimate the electricity energy usage post-retrofit, with the heating energy normalised for the period 1 April 2018 to the 31 March 2019 using the HDD procedure described earlier in Section 3.4.1.

As electricity was not used for space heating purposes in Case B pre-retrofit, the electricity usage data from the pre-retrofit six-month monitoring period was normalised per day and multiplied by the number of days in a year to determine their annual electricity usage for lighting and appliances.

4.3. Case C

Case C, with a heated floor area of 107 m², received retrofit measures for the building fabric and heating system. The bungalow home constructed in the 1950s and occupied by a woman more than 75 years old, installed 200 mm of earth wool insulation along the joists of the attic. The oil boiler and hot water tank components of the primary heating system were replaced with more energy efficient versions as part of the retrofit works. A solid fuel range, which acted as a secondary space heating system, was replaced with a solid fuel boiler. In addition, three radiators were replaced and heating controls installed. Oil and solid fuel were used for the space and water heating requirements of Case C before and after the retrofit works.

As electricity was not used for space heating purposes in Case C pre-retrofit or post-retrofit, the electricity usage data from the monitoring periods were normalised per day and multiplied by the number of days in a year to determine their annual electricity usage for lighting and appliances. By subtracting the post-retrofit energy demand from the pre-retrofit energy demand, the estimated annual energy demand savings were determined for each case.

4.4. Case D

Case D is a bungalow originally constructed in the late 1970s with a heated floor area of 72 m². The house, which is occupied by a woman more than 65 years old, underwent a shallow retrofit. The energy efficiency upgrades mainly focused on the space and water heating system. A more energy efficient hot water tank and oil boiler replaced less energy efficient versions. In addition, a new space and water heating control panel was installed for the homeowner. Oil and solid fuel were used for the space and water heating requirements of Case D before and after the retrofitting works

Pre-retrofit, the householder relied on their oil boiler for domestic hot water heating purposes. The householder used a solid fuel range installed in the kitchen working in tandem with a back boiler to circulate hot water to the radiators installed throughout the house to provide space heating. Following the retrofit, oil was used for both space and water-heating purposes, as the householder was advised she could not receive a grant for a new oil boiler without switching the central space heating to operate with the oil boiler. For the building fabric, an adhesive was added to cavity wall insulation that had been pumped into the cavity during previous retrofit work. Furthermore, air vents were added to each of the rooms in the house.

As electricity was not used for space heating purposes in Case D pre-retrofit or post-retrofit, the electricity usage data from the monitoring periods were normalised per day and multiplied by the number of days in a year to determine their annual electricity usage for lighting and appliances. By subtracting the post-retrofit energy demand from the pre-retrofit energy demand, the estimated annual energy demand savings were determined for each case.

4.5. Case E

Case E switched from a solid fuel-based heating system to an electricity-based heating system following the retrofit works. The couple, aged between 56 and 64 years, removed the solid fuel stove installed in their bungalow home and installed an ASHP in addition to heating controls, radiators, and a 2.1 kWp PV system with an inverter. The house, with a heated floor area of 71 m² and originally constructed in the 1960s, had undergone previous retrofit work including external wall insulation.

As electricity was not used for space heating purposes in Case E pre-retrofit, the electricity usage data from the pre-retrofit six-month monitoring period was normalised per day and multiplied by the number of days in a year to determine their annual electricity usage for lighting and appliances.

To differentiate between the electricity consumption for heating, lighting, and appliances for Case E post-retrofit, the pre-retrofit daily electricity usage for lighting and appliances was assumed to be the same post-retrofit. Thus, the difference between the pre-retrofit and post-retrofit energy usage was assumed to be the post-retrofit space and water heating energy required in Case E.

5. Results

The impact to each of the KPIs assessed using the data collected from the five case study buildings is given in Table 3 and discussed in the following sections.

5.1. Case A

The household of Case A experienced benefits across multiple KPIs. This household invested the largest amount of money into their retrofit works of the five cases. Despite having an investment of close to EUR 30,000 (net of grant), the householders are predicted to achieve payback for the works within 10 years, providing their post-retrofit energy demand remains constant during this period. Even without grant aid, the householders would achieve payback within 15 years.

The householders reported being very satisfied with their thermal comfort following the retrofit. Post-retrofit, the average household temperature increased by 3.1 °C and RH decreased by 10%, respectively. In addition, one householder felt that the thermal comfort improvements helped address the health risk their home posed pre-retrofit, stating “the house was a health risk before … something had to be done or we would have had to move. Our quality of life has improved”. Furthermore, the householder reported that previous issues with draughts, cold, condensation, mould, and dampness no longer persisted post-retrofit. While the householder noted that “no major hassle was caused” by the works, they believed the retrofit process overall could be improved with better engagement with contractors, more choice in the design of the retrofit, more information on maintenance and guarantee of the technologies, and a better financial payment plan for the works.

Regarding the level of engagement with contractors throughout the process, the householder stated, “no choice was given in the design of the windows and doors installed”, or in the location of the PV invertor to allow convenient monitoring of when the panels were generating energy. In relation to the technology installed, the householder stated, “no one showed us how to use anything. They simply installed them and left. They didn’t tell us how to maintain these technologies”, while also adding “no details on product guarantees were offered either”.

Furthermore, the householder was not satisfied with all the retrofit measures installed. In particular, the householders criticised the PV panels, stating, “the (PV) panels don’t generate electricity consistently, so I don’t think I’m getting the payback I thought I would. No one told me that I might need a battery to store the energy I’m not consuming until after, and those are very expensive”. However, despite the issues with the retrofit process, the household achieved a 71%, 65%, and 59% reduction in secondary energy, primary energy and carbon emissions, respectively.

5.2. Case B

The householder in Case B decided to switch from an oil and solid fuel based heating system to an electric heat pump complemented with PV panels, resulting in a 73% decrease in LCC. The householder achieved payback within one year, given the high percentage of grant aid the householder received for their retrofit works. Without the grant, the payback period would have been close to 13 years, based on the post-retrofit energy demand.

It is expected that the reduction in operational energy demand, which led to a lower LCC is a result of the retrofit technology installed, and the decrease in temperature to which the dwelling was heated. The decrease in average temperature could be partially attributed to several factors, including the post-retrofit change in occupancy in the dwelling, and the impact of the retrofit measures installed. Specifically, the occupancy profile changed post-retrofit, following the death of the elderly mother that had been living in the dwelling pre-retrofit. Post-retrofit, the son lived in the house alone. As a result, he did not heat the dwelling to as high a temperature as he had pre-retrofit to ensure his mother was comfortable. The second possibility is that the installation of heating controls gave the householder stricter control over the thermal conditions post-retrofit. Despite the quantitative data showing reductions in the amount of time the indoor environment was within recommended temperature and RH levels, the householder was very satisfied with his thermal comfort, illustrating the benefit of collecting both quantitative and qualitative data, and the subjectivity associated with householder’s thermal comfort.

While the householder was satisfied with many aspects of the retrofit, as reflected in Table 2, the householder believed that contractor engagement during the handover process could be improved, stating that when the heat pump cut out, he struggled to restart it as he was not shown how to “get it started if that happened”.

5.3. Case C

Despite a negligible decrease in energy demand (3%) and an increase in LCC (7%), the householder in Case C believed their dwelling to be “really great” post-retrofit. The householder also expressed complete satisfaction with the level of engagement with contractors and the quality of the works, stating that the workers “were absolutely brilliant”, and “did not stop working until the work was done perfect and were always here when they said they would be”.

The negligible decrease in energy demand was a result of the post-retrofit coal and briquette fuel savings only slightly outweighing the post-retrofit increase in electricity and oil demand. The additional cost of electricity and oil per kWh, relative to that of coal and briquettes, resulted in the dwelling’s operational cost increasing post-retrofit. Despite a more efficient oil boiler being installed, the householder felt she used the new oil boiler more post-retrofit than the previous boiler, as she preferred to keep the house warmer than it was pre-retrofit. This demand for a higher temperature is reflected in the thermal comfort results, with the average temperature increasing post-retrofit, and the amount of time the temperature was less than 18 °C decreasing post-retrofit.

5.4. Case D

The operational secondary energy, primary energy and carbon emissions of Case D reduced post-retrofit by 21%, 20% and 23%, respectively. Despite these environmental improvements, the LCC savings were not as high due to a higher oil demand post-retrofit. This higher post-retrofit oil demand can be accounted for by the fact that post-retrofit, oil was used for both space and water heating purposes. Pre-retrofit, oil was used for water heating purposes only, with the householder using a solid fuel range for central space heating purposes. The householder was advised that she could not receive a grant for the new oil boiler without switching the central space heating to oil also.

The householder still achieved a relatively quick payback on the investment (<2 years), given that the householder invested only EUR 900 into the works as she was entitled to substantial grant aid as part of the BEC scheme. If this grant was not available, however, it would take over 20 years for the householder to see a return on her investment, despite only undertaking shallow retrofit works.

The householder had some issues with the level of contractor engagement and the technologies installed, despite having no other issues with the retrofit process itself and the works carried out. The householder stated that she was not shown how to use her new heating controls. This eventually led her to request her old heating controls be reinstalled, as she found the new heating controls too complicated. She stated, “I liked my old heating controls better. The new ones were far too complicated. I just want to be able to switch them on and off…so they took out my new ones and put the old ones back in”. The householder also preferred the solid fuel range for central heating as opposed to the oil boiler, but this could not be reversed.

5.5. Case E

Case E switched from a solid fuel-based heating system to an electricity-based heating system post-retrofit. This resulted in increased electricity consumption post-retrofit. Despite this increased electricity consumption, the use of solid fuel in the dwelling post-retrofit resulted in a reduction in secondary energy demand of 59%. However, the LCC increased post-retrofit due to the cost per kWh of electricity relative to solid fuel.

The householder was completely satisfied with the heat pump installed, stating that they had “no regrets whatsoever in choosing the heat pump, as heating the house before was like a full-time job”. The householder also felt that the dwelling was warmer and more comfortable post-retrofit, noting that it was also “nice to have hot water during the day instead of switching on the immersion”. The thermal comfort improvements are reflected in the quantitative temperature data, whereby the average temperature increased by 1.3% post-retrofit, with the amount of time the temperature was less than 18 °C reducing by 31%.

While the householder was satisfied overall with the retrofit having “absolutely no regrets in undertaking the works”, the householder did note that the retrofit installers were sometimes “sloppy”, and often had a start-stop nature in completing the works.

The householder also noted problems with the level of engagement with the contractor and the technology installed. The householder stated that in the handover process, “there was no briefing on any of it”, in terms of how to use the installed technologies. The householder noted there were “no manuals for anything. If it doesn’t work, we have to figure out ourselves how to fix it”. Furthermore, the householder found the PV system unimpressive, stating that it “seemed more interesting than it is in reality”, and that they were not sure if they would invest in maintaining it in the future. Moreover, as a battery was not installed to complement the PV panels, the householder noted a similar complaint to that of Case A, whereby they found it difficult to match the electricity demand of the house with the electricity generation of the PV panels.

6. Discussion

To date, many efforts to evaluate retrofit projects have done so through a rather narrow lens, focusing primarily on economic and environmental KPIs, while failing to give due consideration to social KPIs. Moreover, many studies focus on evaluating one or two categories of KPIs. They very rarely, however, evaluate economic, social, and environmental KPIs, within an integrated framework. This is considered less than ideal, given the fact that other non-economic and environmental factors are important drivers to retrofit investment decisions. Thus, the full value of retrofit improvements is underestimated.

As demonstrated by the results of the case study dwellings, householders can experience various benefits from a building retrofit. For instance, although the householder in Case C spent more to heat the home following the retrofit, they were happy to have completed the work and believed their home to be “really great” post-retrofit thanks to their improved thermal comfort. The householder in Case A is estimated to achieve payback for their investment of nearly EUR 30,000 within 10 years, with the householders also experiencing a significant improvement in their thermal comfort. Additionally, Case B and Case E illustrate the different benefits homes can receive from a similar retrofit package. While both dwellings switched to an ASHP-based heating system complemented by a PV system, one saved energy and money (Case B), while the other had a warmer home (Case E).

It is acknowledged that the differences in the case study dwellings presented makes it difficult for a cross-comparison of the benefits received following each retrofit. While this is a limitation of this study, the importance of collecting data that showcases the different benefits (or lack thereof) people can experience from retrofitting cannot be underestimated. As householders are often unable to quantify and evaluate retrofit performance, there is large potential to increase satisfaction and reduce post-purchase regret [106], by demonstrating the multiple benefits they have received through pre- and post-retrofit comparisons. Furthermore, it has real potential in convincing householders to undertake further retrofit works (see [25,30,107]), and in inspiring others seeking similar benefits, especially given the importance of positive feedback, social proof and demonstration projects in householder retrofit decision-making [79,108,109]. In fact, the importance of such is demonstrated in the present study, as since the end of data collection, Case C has gone on to install further retrofit measures, including external wall insulation and a PV system.

The Irish government’s Climate Action Plan [2] aims to install 400,000 heat pumps to existing dwellings in Ireland by 2030, in addition to the retrofit of 500,000 homes to a B2-BER or better (i.e., primary energy consumption less than or equal to 125 kWh/m²/y). Three out of the five case study dwellings had state-of-the-art heat pumps installed as part of their retrofit works, with interesting findings as to their satisfaction and experience with these heat pumps. Existing research on heat pump adoption highlights that one of the key barriers to heat pump adoption is a concern among householders that heat pumps are slow to heat rooms [110], which may result from a persistent lack of understanding among householders as to how heat pumps operate, compared with gas or oil boilers [111]. Our study revealed, however, that householders had satisfactory experiences relating to the heat pump and its ability to heat the home, with a householder in Case E expressing their satisfaction with the heat pump’s ability to remove the effort needed to continually add fuel to the stove to heat the house pre-retrofit. However, all three householders (with varying demographic and socioeconomic statuses) in Case A, Case B, and Case E, highlighted the lack of information provided at handover as to how to best operate and maintain these technologies. Such findings agree with those of existing literature which finds that a poor understanding of how heat pumps and their controls work (see [110,112]), as well as a lack of training provision from installers on heat pump operation (see [113]), constitute significant barriers to heat pump uptake. Thus, our findings agree with the importance of adequate training at handover on the operation of such technologies in encouraging heat pump uptake, as recently suggested by the SEAI [111]. This is particularly pertinent, given the influence inappropriate heat pump use has on energy savings, and comfort levels obtained post-retrofit [111]. If these issues persist on a wider scale, there are significant challenges to be overcome for the achievement of the Irish government’s Climate Action Plan targets set.

The householders in Case A, Case B, and Case E, also installed state-of-the-art PV systems, which normally complement heat pumps from an electricity consumption perspective. However, a source of dissatisfaction with this technology in two out of the three households (Case A and Case E) agreed on the difficulty of matching the electricity demand of their dwellings, to the electricity generation from the PV system.

While the householders in Case C and Case D did not install heat pumps or PV systems as part of their works, they did install state-of-the-art heating controls. However, despite the similar occupant profiles in these dwellings (a widow, living alone, aged >65 years), the householders had very different experiences of using this technology, with one householder even having the new controls removed, given her dissatisfaction with the technology and its use. Existing research has shown that the energy savings resulting from a retrofit depend strongly on heating system operation; yet, heating controls are consistently regarded as difficult to use by homeowners, particularly, in older homeowners [114]. Our findings agree with such, and reinforce the importance of understanding and incorporating the energy impacts of poor usability of such controls into the future design of these technologies, as well as, the need for appropriate training provision on their use at retrofit handover, and continued customer support.

Thus, while the framework is not capable of explaining reasons for differences in benefit levels across homes (which is a limitation), it can allow for the identification of potential factors for differences in performance across households (e.g., temperature take-back, reduction in temperature demand, switching fuel source, etc.). It could also be used to identify patterns of underperformance for specific parameters (e.g., difficulty in matching the electricity demand of the house with the electricity generation of the PV panels as seen in Case A and E), which may need to be further investigated to understand why, and the level to which certain retrofit packages are not obtaining certain benefits. These insights are particularly useful for the future design of retrofit technologies, the future design of retrofit packages, as well as the design of future policy measures, that support householders not only in their adoption of retrofit measures, but also throughout their experiences of living in a retrofitted home.

Limitations

While this study has demonstrated the usefulness of an integrated framework of KPIs in demonstrating the economic, social, and environmental benefits of retrofitting, some limitations should be acknowledged. Firstly, even though the KPIs representing each core category assessed are justified in their inclusion, they are certainly non-exhaustive, with other KPIs in each category being discussed in the literature. They are considered to be the minimum captured for demonstrating the wider benefits of retrofitting to homeowners and encouraging homeowners to invest in building retrofits. As retrofit projects are complex and can have other benefits that may not be captured using the selected set of KPIs, other KPIs should be added if required. In addition, it should be noted that while the KPIs assessed, and the methods used for their assessment, resulted in useful knowledge on the wider benefits of retrofitting, it is not suggested that these methods be considered a standard methodology for the assessment of these KPIs in the future. There is a multitude of ways to assess these KPIs, and further, critical analysis of best practice methodologies for doing so is needed. This is an interesting direction for future research, particularly, for developing standardised integrated frameworks that can be applied to large sample sets. Other researchers agree on the importance of developing standardised methods for the assessment of such KPIs [33].

It is also acknowledged that a small sample of just five case study dwellings was used in this study. The case studies used in this paper were part of a wider monitoring study, consisting of a total of 13 dwellings. However, due to the loss of data collected by the data loggers in three of the eight dwellings that invested in a retrofit, five case study dwellings were selected for use in this study.

7. Conclusions and Implications for Policy, Practice, and Knowledge

This study reviewed the available literature on the state-of-the-art of the use of KPIs to evaluate energy retrofits and sustainable building performance. There are a lack of studies that assess the economic, social, and environmental benefits of retrofitting within an integrated framework. Thus, this paper applied a framework of KPIs to five case study dwellings located in the west of Ireland, to demonstrate the wider benefits householders can receive through retrofitting. This paper is significant in that it contributes to the literature by firstly providing a framework of core economic, social, and environmental KPI categories that should be included in retrofit evaluations at a minimum, and common KPIs for their assessment.

While the primary function of the KPI framework is to demonstrate the wider benefits obtained by householders following a retrofit investment as discussed, it also has a potential function from a retrofit design perspective, in that its application to a much larger, comparable sample size, could help designers distinguish the retrofit packages that typically perform best across all KPIs, across various household compositions, and dwelling types. As discussed, reasons for differences in performance, and underperformance, across households could be identified and rectified in retrofit design. Subsequently, rather like a feedback loop, designers can then demonstrate to future homeowners the benefits achieved in implemented retrofit projects following a straightforward retrofit journey for the homeowner, with a view to driving householder investment in residential retrofitting.

While the collection of such information on a large-scale basis would require a standardised data collection methodology to be implemented, it would serve as an evidence base for retrofit designers, contractors, and policy makers to incentivise and de-risk investment by homeowners in deeper building retrofits, which would aid in helping not only Ireland achieve its target of 500,000 deep retrofits by 2030, but also the EU in meeting its ‘renovation wave’ targets. Thus, the framework of KPIs developed in this study can serve as an invaluable tool to assess, demonstrate, and maximise the wider retrofitting benefits of various policy measures, retrofit schemes and/or retrofit packages to drive retrofit investment by householders in the residential sector.

However, research gaps remain. Future research, which aims to develop standardised KPIs for the assessment of energy retrofit projects, as well as research that develops standardised methods and guidelines for their assessment, is considered critical to the expansion of knowledge in this area, to support design decisions, and to inform national and international retrofit policies alike.

Overall, this paper made several key findings:

• There remains a need to develop a set of standardized economic, social, and environmental KPIs for the evaluation of building retrofits, as well as standardized methods for their assessment.
• The assessment of an integrated framework of economic, social, and environmental KPIs, in a pre- and post-retrofit case study is not only useful for the demonstration of the wider benefits of retrofitting, but also for identifying potential issues for differences in performance, and underperformance, of state-of-the-art retrofit technologies.
• Householders report difficulties in understanding the operation and maintenance of state-of-the-art technologies, including heat pumps, PV systems, and heating controls.
• Householders with similar demographics and socioeconomic backgrounds can have very different experiences in the use of retrofit technologies.