Can ChatGPT AI Replace or Contribute to Experts’ Diagnosis for Renovation Measures Identification?

Abstract

Building energy renovations demand expertise from professionals to guide processes, including diagnostics, project planning, interventions, and maintenance. The emergence of open-access AI, like ChatGPT in November 2022, offers new possibilities for improving these processes by assisting or potentially replacing human experts. This study explores the effectiveness of ChatGPT in diagnosing energy renovation measures. Initial assessments involve basic queries to the AI, followed by the inclusion of additional data and secondary questions to gauge its full diagnostic potential. An existing building case from the literature is given to the AI to define the best energy renovation measures. Expert evaluations and comparisons with research-backed solutions assess the AI’s performance using different degrees of questioning details over 60 repetitions. The results indicate that ChatGPT can provide valuable insights and generate comprehensive lists of feasible measures and preliminary cost calculations and payback, but, in general, it lacks depth and quality without specialized input and preparation. A significant quality improvement was found between the tests with 2023 and 2024 AI versions. Open-access AI proves capable of enhancing renovation diagnostics but remains a complement rather than a replacement for building renovation expert judgment. This research underscores the potential of mainstream AI to democratize access to knowledge, albeit with limitations tied to its dependence on quality inputs and contextual expertise.

1. Introduction

Artificial intelligence (AI) has revolutionized numerous industries, and the construction sector is no exception. With its capacity to process vast datasets, identify patterns, and make informed decisions, AI is a potential key tool to transform building refurbishment processes. This paper explores the potential of AI, specifically ChatGPT, to function as a virtual energy specialist in building refurbishments. By examining the capabilities of this advanced language model, we aim to assess its feasibility as a tool to support human experts in the renovation process.

Recent research highlights the transformative potential of AI across various domains, particularly in construction, where it can enhance productivity and address challenges through its subfields, such as Machine Learning (ML), robotics, natural language processing, and optimization techniques [1]. Among these, AI chatbots stand out for their cost-effectiveness and practical applicability, providing realistic insights at a favorable cost–benefit ratio [2].

The roots of AI date back to the mid-20th century, with foundational contributions from pioneers like Alan Turing and John McCarthy. Innovations such as ELIZA, SHRDLU, and MYCIN catalyzed research into ML algorithms and neural networks [3].

On the other hand, AI’s potential for different purposes relates to the need for longer or shorter training periods. That is, even though ML systems must be adequately researched and trained, for general society or mainstream purposes, they can help with shorter training periods by becoming dissemination tools, mainly oriented to communicating main messages and adjusting the technical speech to each individual user. For specialized uses, ML or AI tools in general must be adequately researched and trained in the field in which they will be used following consolidated and highly specialized databases. But this all will require further developments, which should not hinder the mainstream use of AI chatbots or other basic ML tools.

Accordingly, this study focuses on mainstream users and free-access AI, specifically ChatGPT [4], a pivotal development built on OpenAI’s GPT-3.5 architecture, trained until July 2023. Its advanced language modeling and natural language processing capabilities showcase AI’s potential for simulating human-like interactions and delivering actionable insights.

AI’s versatility extends across industries, revolutionizing healthcare, finance, transportation, and more. In building energy renovations, AI can analyze large datasets to optimize energy efficiency and reduce environmental impact [5]. While promising, this study critically examines whether ChatGPT can replace or contribute to expert diagnoses in renovation projects, balancing its advanced abilities with its current limitations.

On the limitations side, it is important to keep in mind that AI chatbots can respond with “hallucinations” or invented references to fill some gaps and avoid not responding to the requested prompt. Recent research from medical studies raises concern due to the risk of following fast decisions in their vast literary field and recommends systematic reviews [6,7].

In this matter, a key role is played by defining and reviewing the proper use of Harmonized Standards for AI. These legal aspects can contribute to or hinder the development of these tools and their use at multiple levels. According to the newest JRC report, “the European Union adopted the AI Act in August 2024, and the provisions for high-risk AI systems will start to apply after a transition period of 2 or 3 years” [8]. So, AI is still developing, and the results of this harmonization are still unclear.

2. Materials and Methods

To understand whether AI can replace or contribute to building renovation experts’ diagnoses, a two-stage experiment was conducted using the most recent free-access AI tool of ChatGPT [4], first in July 2023 and later in December 2024 [9]. This experiment consisted of 2 stages with a set of 10 iterations of 3 questions asking the AI what measures we could use to retrofit a reference house. Similar approaches of repetition of questions and answers from AI chatbots have been described in recent studies (“This could lead to the development of even more advanced chatbots and virtual assistants to handle complex tasks and provide personalized recommendations and advice” [10]), in addition to how the ChatGPT model “highlights conversational AI’s role in fostering learning, critical thinking, and iterative information use” [11].

These 10 iterations help to visualize the variability of the AI tool greatly. The 3 questions are prompts that include the key data of our research case and allow us to evaluate the responses using different preparation degrees for the AI tool. This preparation represents the knowledge of the user, meaning that while a non-expert would ask simple questions using basic data on a building, an experienced user would create a more complex request, adding details, regulations, methods, etc. The fast development of this tool during 2023 and 2024 suggested the need for running a second stage, as the results confirmed afterwards.

The first question (Q1) is asked without any further preparation and facilitates standard information that any owner could have. The second question (Q2) repeats the same request but adds technical data about the building envelope features and construction regulations, to enlarge its preparation. The third and final question (Q3) specifically requests the AI tool to follow a specialized method to see if the AI tool can apply it or whether it would need an expert’s help. If any response forgets to add any references, the AI tool is asked once again to remember to include justifications and references.

Table 1. Questionnaire template.

Questions	Prompts
Q1	We want you to respond like if you were a building energy specialist. We are going to describe a case study. A wooden terraced house in Oslo, constructed in 1997, with a basement and two upper floors. The basement is reinforced concrete and the upper floors are made of a wood frame. The dwelling has not been retrofitted, so we think it has around 20 cm of insulation. These are some extra details of the building: Basement, Floor area: 46 m2 Volume: 102.0 m3 Ground floor, Floor area: 53 m2 Volume: 127.2 m3 First floor, Floor area: 47 m2 Volume: 112.8 m3 Heating system: Electric panel heaters Heating source: Electricity Ventilation system: Centralized mechanical extract ventilation, airflow: 1.36 m3/hm2 We want to ask you for measures to retrofit the house. We want these measures to be passive solutions. Explain your recommendations for different ambition levels for renovation. Do not forget to justify all your results with reasons and adding their references, like reports, documents, papers, case studies, etc.
Q2	We want to add some details: The applicable building energy regulation for this building, the Norwegian standard TEK 87 (1987), indicates that the U-values are the following U-Wall: 0.30 U-Basement wall: 0.38 U-Roof: 0.30 U-Window: 2.40 U-Door: 2.00 U-Floor: 0.20. With this extra information, could you please answer the request again: We want to ask you for measures to retrofit the house. We want these measures to be passive solutions. Explain your recommendations for different ambition levels for renovation. Do not forget to justify all your results with reasons and adding their references, like reports, documents, papers, case studies, etc.
Q3	We want you to develop your previous answer using the Life Cycle Cost methodology (LCC) to evaluate the renovation potential using the following passive measures: wall insulation, ground floor insulation, internal floor insulation, ceiling/roof insulation, windows replacement, door replacement and airtightness. With this extra information, could you please answer the request again: We want to ask you for measures to retrofit this house in Oslo, as described before. We want these measures to be passive solutions. Explain your recommendations for different ambition levels for renovation. Do not forget to justify all your results with reasons and add their references, like reports, documents, papers, case studies, etc.
Requesting references	Could you please tell me the references for your previous recommendations?

gathers all the questions.

The case was selected from the literature because it is a simple single-family house that was studied by experts using comprehensive research [12]. The results of that study were considered for the evaluation of the AI responses.

Each of the 60 responses is analyzed using five parameters: the number of ambition levels, the number of renovation measures, the number and type of references, types of assessment (qualitative, quantitative or mixed), and finally the quality of the answer. This quality level is evaluated as follows:

• Low: General recommendations but missing important aspects.
• Low–Medium: General recommendations, embracing passive and active solutions.
• Medium: General recommendations, with a few numbers or descriptions to fit the case better.
• Medium–High: Detailed recommendations, with some building and location aspects.
• High: Detailed and comprehensive answers, fitting most building and location aspects.

The comparison of the number of ambition levels, renovation measures, and references can help identify the connection between preparation and response. The potential correlations between ambition levels and renovation measures of all the responses are studied.

Finally, the evaluations are also assessed globally, using total and average scores combining the 10 iterations of each of the 3 questions.

To respond to this challenge, three questions are stated:

• Can open-access AI replace or contribute to building renovation experts’ diagnoses?
• Can AI give good energy-efficient recommendations to renovate a certain building?
• Is the quality of the answer connected to the level of knowledge of the user and the detail included in the question?

3. Results and Discussion

3.1. Particular Assessment of the AI Responses

Analyses are performed in each stage on the ten iterations and three questions, registering how each response fits the five parameters studied. The main results of the first stage, using the free version of ChatGPT in July 2023, are presented in

Table 2. Summary and evaluation of the responses from Q1, free version of ChatGPT, 2023.

Iterations	Number of Ambition Levels	Number of Renov. Measures	Number of References	Type of Assessments	Quality of the Answers
Test 1	3	8	no references (4 guides more after request)	Qualitative	Medium
Test 2	5	8	1 guide 3 institutions 2 scientific papers	Qualitative	Medium–High
Test 3	4	14	2 guides 3 scientific papers	Qualitative	Low–Medium
Test 4	3	11	5 guides 1 unclear publication (1 institution more after request)	Qualitative	Low–Medium
Test 5	4	7	4 institutions	Qualitative	Low–Medium
Test 6	3	8	7 institutions	Qualitative	Low–Medium
Test 7	3	6	1 guide 3 institutions 2 scientific papers	Qualitative, a few quantitative	Medium
Test 8	4	17	4 institutions (2 institutions more after request)	Qualitative	Low–Medium
Test 9	3	7	1 guide 1 institution 1 scientific paper	Qualitative	Medium
Test 10	3	11	5 guides 1 institution 3 scientific paper	Qualitative	Low–Medium
Total/ Average	3.5	9.7	1.2 guides/reports 2.3 institutions 1.0 scientific papers 0.1 unclear publications	90% Qualitative 10% partly quantitative	60% Low–Medium 30% Medium 10% Medium–High

,

Table 3. Summary and evaluation of the responses from Q2, free version of ChatGPT, 2023.

Iterations	Number of Ambition Levels	Number of Renov. Measures	Number of References	Type of Assessments	Quality of the Answers
Test 1	3	6	no references (4 guides more after request)	Qualitative, a few quantitative	Medium
Test 2	5	8	1 guide 3 institutions	Qualitative	Medium–High
Test 3	3	9	2 guides 3 scientific papers	Qualitative	Medium
Test 4	3	9	8 guides 1 unclear reference	Qualitative, a few quantitative	Medium
Test 5	4	7	4 institutions	Qualitative, a few quantitative	Medium
Test 6	3	6	1 guide 5 institutions	Qualitative, a few quantitative	Medium
Test 7	3	7	1 guide (7 institutions more after request)	Qualitative	Medium
Test 8	4	10	4 institutions	Qualitative, a few quantitative	Medium
Test 9	3	6	1 guide 1 institution 1 scientific paper	Qualitative	Medium
Test 10	3	10	4 guides/reports 2 institutions	Qualitative, a few quantitative	Medium–High
Total/ Average	3.4	7.8	1.8 guides/reports 1.9 institutions 0.1 scientific papers 0.1 unclear publications	40% Qualitative 60% partly quantitative	80% Medium 20% Medium–High

, and

Table 4. Summary and evaluation of the responses from Q3, free version of ChatGPT, 2023.

Iterations	Number of Ambition Levels	Number of Renov. Measures	Number of References	Type of Assessments	Quality of the Answers
Test 1	1 (not used)	7 (7 asked items)	7 guides	Qualitative	Low–Medium
Test 2	4	10	3 institutions	Qualitative	Medium
Test 3	3	8	1 guide 3 scientific papers	Qualitative	Low–Medium
Test 4	3	9	6 institutions	Qualitative	Low–Medium
Test 5	(Not performed)	(Not performed)	(Not performed)	(Not performed)	AI requested more information on energy prices, lifespan, costs
Test 6	3	13	4 guides	Qualitative and quantitative	Medium–High
Test 7	3	7	1 guide 2 scientific papers	Qualitative	Low–Medium
Test 8	4	9	4 guides	Qualitative, a few quantitative	Medium–High
Test 9	1 (not used)	7 (7 asked items)	4 guides 4 institutions 2 scientific papers	Qualitative	Low–Medium
Test 10	3	11	6 guides/reports	Qualitative, a few quantitative	Medium–High
Total/ Average	2.8	9.0	2.7 guides/reports 1.3 institutions 0.7 scientific papers	60% Qualitative 20% partly quantitative 10% qualitative and quantitative 10% not completed	50% Low–Medium 10% Medium 30% Medium–High 10% not completed

for Q1, Q2, and Q3, respectively. The main results of the second stage, using the free version of ChatGPT in December 2024, are in

Table 5. Summary and evaluation of the responses from Q1, free version of ChatGPT, 2024.

Iterations	Number of Ambition Levels	Number of Renov. Measures	Number of References	Type of Assessments	Quality of the Answers
Test 1	3	10	3 guides/reports 5 institutions 1 scientific paper 1 unclear publication	Qualitative, a few quantitative	Medium
Test 2	3	10	guides/reports 1 institutions 9 scientific papers	Qualitative, a few quantitative	Medium
Test 3	3	11	2 guides/reports 5 institutions	Qualitative, a few quantitative	Medium
Test 4	3	10	6 guides/reports 1 institution	Qualitative, a few quantitative	Low–Medium
Test 5	3	11	4 guides/reports 5 institutions 1 scientific paper	Qualitative, a few quantitative	Low
Test 6	3	10	5 guides/reports 1 institution 2 scientific papers	Qualitative, a few quantitative	Low–Medium
Test 7	3	11	4 institutions	Qualitative, a few quantitative	Low–Medium
Test 8	3	12	2 guides/reports 1 institution 6 scientific papers	Qualitative, a few quantitative	Low–Medium
Test 9	3	10	5 guides/reports 4 institutions	Qualitative, a few quantitative	Low
Test 10	3	10	5 guides/reports 4 institutions	Qualitative, a few quantitative	Low–Medium
Total/ Average	3.0	10.5	3.2 guides/reports 3.1 institutions 1.9 scientific papers 0.1 unclear publications	90% Qualitative 10% partly quantitative	20% Low 50% Low–Medium 30% Medium

,

Table 6. Summary and evaluation of the responses from Q2, free version of ChatGPT, 2024.

Iterations	Number of Ambition Levels	Number of Renov. Measures	Number of References	Type of Assessments	Quality of the Answers
Test 1	3	14	4 guides/reports 5 institutions 2 unclear publications	Qualitative and quantitative	Medium–High
Test 2	3	11	1 institution 8 scientific papers	Qualitative, a few quantitative	Medium
Test 3	3	9	5 institutions	Qualitative, a few quantitative	Medium
Test 4	3	11	3 guides/reports 3 institutions 1 scientific paper	Qualitative, a few quantitative	Low–Medium
Test 5	3	14	3 guides/reports 3 institutions 5 scientific papers	Qualitative and quantitative	Low–Medium
Test 6	3	12	5 guides/reports 2 institutions 3 scientific papers	Qualitative, a few quantitative	Low–Medium
Test 7	3	13	1 guide/report 5 institutions	Qualitative, a few quantitative	Low–Medium
Test 8	3	16	1 guide/report 2 institutions 5 scientific papers	Qualitative, a few quantitative	Low–Medium
Test 9	3	11	3 guides/reports 1 institution	Qualitative, a few quantitative	Low–Medium
Test 10	3	11	7 guides/reports 3 institutions	Qualitative, a few quantitative	Low–Medium
Total/ Average	3.0	12.2	2.7 guides/reports 3.0 institutions 2.2 scientific papers 0.2 unclear publications	80% partly quantitative 20% qualitative and quantitative	70% Low–Medium 20% Medium 10% Medium–High

, and

Table 7. Summary and evaluation of the responses from Q3, free version of ChatGPT, 2024.

Iterations	Number of Ambition Levels	Number of Renov. Measures	Number of References	Type of Assessments	Quality of the Answers
Test 1	3	14	7 guides	Qualitative	Low–Medium
Test 2	3	8	3 institutions	Qualitative	Medium
Test 3	3	8	1 guide 3 scientific papers	Qualitative	Low–Medium
Test 4	3	7	6 institutions	Qualitative	Low–Medium
Test 5	3	12	(Not performed)	(Not performed)	AI requested more information on energy prices, lifespan, costs
Test 6	3	10	4 guides	Qualitative and quantitative	Medium–High
Test 7	3	16	1 guide 2 scientific papers	Qualitative	Low–Medium
Test 8	3	13	4 guides	Qualitative, a few quantitative	Medium–High
Test 9	3	9	4 guides 4 institutions 2 scientific papers	Qualitative	Low–Medium
Test 10	3	17	6 guides/reports	Qualitative, a few quantitative	Medium–High
Total/ Average	3.0	11.4	1.0 guide/report 1.8 institutions 0.8 scientific papers 0.3 unclear publications	60% qualitative 20% partly quantitative 10% qualitative and quantitative 10% not completed	50% Low–Medium 10% Medium 30% Medium–High 10% not completed

for Q1, Q2, and Q3.

3.2. Levels of Ambition, Number of Renovation Measures, and Number of References

In the first stage with the 2023 tool, the responses do not show a clear pattern in the way the AI tool responds; perhaps the ambition levels in Q1 and Q2 are reduced slightly in Q3. The most detailed third question was often responded to without considering any ambition levels. The answers are plotted for visual analysis in Figure 1 and Figure 2.

As shown in Figure 1, the trend changes when focusing on the number of suggested measures. The AI tool responds initially with a larger number of items compared to the more detailed second and third questions. This helps open the scope to include a variety of aspects of building renovations. The responses are very wide and propose many passive improvements, like in the envelope, active system renovation, and the use of renewables. Also, certain, less common, measures are mentioned in some cases, like control improvement and smart energy management. On top of these, some of the enhancements not only focus on energy efficiency but also prioritize sustainability, like the installation of a green roof.

Looking into the number of references, Q1 and Q2 are rather similar, but this number goes down in Q3 slightly. In some cases, the recommendations do not include references, unless you ask the AI tool repeatedly; these additional references are not taken into account for the global result but are presented to show how the AI tool’s potential is limited in some cases. The references given correspond to general guides or websites of public institutions or companies, not specific study results or applicable measures. In detail, the average Q1 answers involve 1.2 guides/reports, 2.3 institutions, 1.0 scientific papers, and 0.1 unclear publications. On average, Q2 shows 1.8 guides/reports, 1.9 institutions, 0.1 scientific papers, and 0.1 unclear references. On average Q3 indicates 2.7 guides/reports, 1.3 institutions, and 0.7 scientific papers.

The second-stage results, obtained with the December 2024 version of this AI tool, showed some clear differences, as depicted in Figure 2. First, all the answers fit what was asked in Q1 and include three ambition levels for this building renovation. Second, the number of renovation measures presents a subtle trend of increasing after the preparations of Q2 and Q3, from 10.5 to 12.2 and 11.4 as the average measures, which are always more than the previous stage, with 9.7, 7.8, and 8.2 as the average measures. Finally, the number of references is richer than in the anterior stage and shows a clear downward slope from Q1 to Q3, partly when more details are provided and greatly when a specific, detailed calculation method is requested.

3.3. Types of Assessment and Answer Quality

During the first stage with the 2023 tool, Figure 3 shows that the AI recommendations are, in general, qualitative, and only some include quantitative assessments. On the other hand, even though the AI tool was asked to follow the LCC methodology, it failed to use it for recalculating or modifying the recommendations in most of the iterations. In other words, the AI tool continues responding using general recommendations from the literature that include LCC.

In general, all the responses evidence a medium or somewhat low quality of assessments. The responses are general, indicating qualitative concepts as renovation possibilities, but without giving any quantitative assessments, the user could not know which ones are better for a certain case.

The ulterior stage, using the ChatGPT free version of December 2024, presented some relevant differences, as gathered in Figure 4. First, all the recommendations are now supported at least partly by numerical values. This can be seen already in Q1 when asked without any preparation. Furthermore, the type of assessment and the quality improve slightly in Q2 and clearly in Q3. This happens because the AI starts calculating paybacks for this building’s size and conditions, and some LCCs as well. The extension and detail of around half of the Q3 responses are remarkable because they show quantitative and Medium–High-quality answers.

On the other hand, looking at the potential correlations between the three numerical indicators, the correspondences are weak, with R2 below 0.34 and a high scattering. For this reason, the results are not included in detail in this discussion.

3.4. Examples of AI Responses

It is important to understand that AI responses can gather significant variability. Apart from the proposed methodology of this study, the questionnaire and the iterations allowed for a good variety of AI responses to be seen. Some examples of stage 1, using the 2023 tool, are listed in

Table 8. AI 2023 response examples for Q1 and Q2: Low–Medium quality (Stage 1, Test 3, Q1) and High–Medium quality (Stage 1, Test 10, Q2).

In Test 3, When Prompted with Q1 (See Table 1), ChatGPT 2023 [4] Generated the Following Text:

In Test 10, When Prompted with Q2 (see Table 1), ChatGPT 2023 [4] Generated the Following Text:

“As a building energy specialist, I understand the importance of recommending energy-efficient and passive solutions for retrofitting the wooden terraced house in Oslo. Passive solutions aim to reduce energy consumption without relying heavily on mechanical systems. Here are my recommendations for different ambition levels of renovation, along with their justifications: Ambition Level 1: Basic Energy Retrofit 1. Improved Insulation: Enhance the building envelope’s thermal performance by adding additional insulation to the walls, roof, and basement. I recommend increasing the insulation thickness to at least 30 cm, which will significantly reduce heat loss and improve energy efficiency. 2. Airtightness: Address any air leaks and drafts in the building by sealing gaps around windows, doors, and other potential air infiltration points. A well-sealed building envelope will prevent heat loss and improve indoor comfort. 3. Energy-Efficient Windows: Replace the existing windows with energy-efficient double or triple-glazed windows. High-performance windows with low U-values will minimize heat transfer and contribute to better thermal comfort. 4. Passive Solar Design: Utilize passive solar design principles to maximize solar gain during the winter months. This can involve optimizing window placement and size to capture natural heat and light. 5. Shading and Ventilation: Incorporate external shading elements such as awnings or deciduous trees to prevent overheating during the summer while allowing solar gain during winter. Additionally, natural ventilation strategies can be employed to reduce the need for mechanical cooling. 6. Heat Recovery Ventilation (HRV): Upgrade the existing mechanical extract ventilation system to a heat recovery ventilation system. HRV will recover heat from the exhaust air and preheat the incoming fresh air, reducing the heating load. Ambition Level 2: Advanced Energy Retrofit Deep Energy Retrofit: Implement a comprehensive deep energy retrofit that include…” [4].

“Thank you for providing the additional information about the applicable building energy regulation for the wooden terraced house in Oslo (TEK 87). With the U-values specified, I can now provide more accurate and targeted recommendations for retrofitting the house with passive solutions. Here are the suggestions for different ambition levels of renovation: 1. Basic Level: At this level, the focus is on cost-effective measures that improve energy efficiency without major structural changes. a. Increased Wall and Roof Insulation: Since the current U-values for walls and the roof are relatively high (U-Wall: 0.30, U-Roof: 0.30), it is advisable to add additional insulation. External wall insulation can be applied to improve U-values to at least 0.15 for walls and roofs, ensuring better thermal performance. b. Window and Door Upgrades: Upgrade the existing windows and doors to models with lower U-values. Aim for double or triple-glazed windows with U-values close to or below 1.0 and doors with U-values below 1.2. c. Air Sealing and Draft Proofing: Address air leaks and drafts by applying weatherstripping and caulking around doors, windows, and other potential leakage points. Justification: ○ The Passive House Institute, in its “Passive House Planning Package (PHPP),” emphasizes the importance of achieving lower U-values for walls, roofs, windows, and doors to meet passive house standards (Passive House Institute, 2021). ○ The International Energy Agency (IEA) report on “Energy Efficiency Policies and Measures Database” reinforces the significance of increasing insulation and upgrading windows and doors for energy-saving in buildings (IEA, 2019). 2. Moderate Level: This level builds upon the basic measures and introduces more comprehensive upgrades. a. Basement Insulation: Since the basement currently has a U-value of 0.38, consider…” [4].

and

Table 9. AI 2023 response examples for Q3: Low-Medium quality (Stage 1, Test 5, Q3) and High-Medium quality (Stage 1, Test 8, Q3).

In Test 5, When Prompted with Q3 (see Table 1), ChatGPT 2023 [4] Generated the Following Text:

In Test 8, When Prompted with Q3 (see Table 1), ChatGPT 2023 [4] Generated the Following Text:

“Certainly! Let’s evaluate the renovation potential of the wooden terraced house in Oslo using the Life Cycle Cost (LCC) methodology for the passive retrofit measures you mentioned: wall insulation, ground floor insulation, internal floor insulation, ceiling/roof insulation, windows replacement, door replacement, and airtightness. The LCC methodology takes into account the initial costs, operational costs, and savings over the building’s lifecycle to assess the economic feasibility of different retrofit options. Before proceeding, I need some additional information to perform a comprehensive LCC analysis: 1. Local Energy Prices: Please provide the current energy prices for electricity used for heating and other energy sources (if applicable). This information will be essential to calculate the operational costs associated with each retrofit measure. 2. Expected Lifespan and Maintenance Costs: Provide the expected lifespan of each retrofit measure and any associated maintenance costs over that period. Once you provide this information, I will proceed with the LCC analysis and provide detailed recommendations for different levels of ambition for renovation.” [4].

“Certainly! To evaluate the renovation potential using the Life Cycle Cost (LCC) methodology for the wooden terraced house in Oslo, I will consider the passive retrofit measures you mentioned: wall insulation, ground floor insulation, internal floor insulation, ceiling/roof insulation, windows replacement, door replacement, and airtightness improvement. LCC takes into account both the initial investment cost and the operational savings over the building’s life cycle. The aim is to determine the most cost-effective retrofit solutions. Here are the recommendations for different ambition levels of renovation: 1. Low Ambition Level: Enhancing Insulation For the low ambition level, the focus is on enhancing insulation to reduce energy consumption and operating costs. a. Wall Insulation: Adding external insulation with a U-value of 0.15 to achieve a significant improvement from the current U-value of 0.30. Justification: External wall insulation is a proven and effective measure to reduce heat loss and improve energy efficiency (source: “Building Envelope Retrofit Strategies”—National Renewable Energy Laboratory). b. Ground Floor Insulation: Increase the floor insulation to achieve a U-value of 0.10 or lower. Justification: A well-insulated ground floor prevents heat loss to the ground and improves overall energy efficiency. 2. Medium Ambition Level: Airtightness Improvement For the medium ambition level, airtightness improvement is combined with insulation measures for better energy efficiency. a. Airtightness Measures: Implement airtightness measures to achieve an airtightness level of at least 3.0 air changes per hour (ACH) at 50 Pascals. Justification: Reducing air leakage…” [4].

. Two response examples of stage 2, using the 2024 version, are long and have been included in Appendix A.

4. Discussion

This study has evaluated the potential for assessing a single-family building renovation with two versions of the ChatGPT AI in 2023 and 2024, and found a significant improvement between them. The second stage, conducted with a recent version from December of 2024, can perform quantitative calculations and reach high-quality answers by presenting some basic tables with the LCC method and payback estimations.

The results reflect that AI chatbots can provide good responses as a preliminary review that can outline building renovation potentials. As the literature suggests, the answers to the tests proved that the current publicly available and free AI tool can be a great tool to start with a certain topic, like building renovation. However, it is still not adequate for deeper work without the support of specialists because of the variability of responses’ quality, particularly due to the need for preparations like the ones of the Q2 and Q3 prompts.

Study limitations include the qualitative approach and the use of the free public version of ChatGPT, which uses information from before 2022 and only from copyright-free sources like laws, regulations, open-source guidebooks, recommendations, and so on.

Regarding the usability of the AI tool, the results show two main aspects: on the one hand, the quality and level of detail of the AI responses can depend greatly on the level of detail and knowledge included in the questions. This would point to more difficulties in obtaining good responses from non-expert users. On the other hand, despite providing detailed questions with additional information, the quality of the responses remains medium in most of the tested iterations.

One of the most positive sides of the AI tool’s potential is that, from the first question, a non-expert user can obtain a wide response covering many energy-saving measures. This is better than conventional internet searches because AI can open the view and improve the understanding of a certain topic in a few minutes.

As one of the negative sides, references are missing in many cases (particularly in stage 1, using the 2023 version) unless you ask for them repeatedly, and they correspond more to general guides and public institutions and not so much to specific results or applicable measures. All the checked references were real, the majority belonging to institutions or public guides, although not all of the references of the 60 iterations were checked.

Regarding the first research question, the results indicate that open-access AI cannot replace building renovation experts’ diagnoses but can clearly contribute to providing some useful insights to non-expert users from the beginning. We can trust the findings, but for now, renovation experts’ guidance is clearly needed. Indeed, the AI tool recommends contacting local experts: “These references can provide you with further insights and specific details on energy-efficient retrofitting measures. It’s always recommended to consult local construction codes, regulations, and seek professional advice to tailor the solutions to your specific case and ensure compliance with local standards.” [4].

Looking at the second research question, the findings reflect that open-access AI can give good recommendations such as energy efficiency measures to renovate a certain building. The responses are rich and varied, opening the potential results further than conventional internet searches. The majority of the responses focus on passive measures, and significantly less active measures are proposed.

The evaluation of the third research question about the knowledge of the user got different results in the versions from 2023 and 2024. The second stage indicated that a higher preparation level would probably give a better response. So, despite being a limited study, we can say that the quality of the answer is likely to be connected to the knowledge of the user and the details included in the question prompt. However, some iterations obtained poorer answers in Q3, despite showing longer or more detailed texts, and the content itself is sometimes less accurate.

Further studies would be interesting to analyze this study and review in detail which measures are suggested in each test and question, to validate their correspondence with the recommendation of the studied literature case study. On the other hand, this study could be conducted with other AI tools with more training, like the paid version of ChatGPT or other AI tools, such as Google Gemini, Microsoft CoPilot, Chat Sonic, or others. A preliminary check, following the same method, and in the latest AI version of Gemini [13] from late 2024 was carried out. This complementary test found some similar results, as presented in

Table 10. Supplementary tests with AI chatbot Gemini [13] in November of 2024.

AI Tool and Date	Question	Ambition Levels	Renov. Measures	References	Type of Assessments	Quality of the Answers
Gemini, Nov. 2024	Q1	3	7	2 institutions	Qualitative	Low–Medium
	Q2	3	10	2 institutions	Qualitative, a few quantitative	Medium
	Q3	3	5	1 institution	Qualitative	Low–Medium

. Another approach could consist of improving the prompt and focusing on how other fields of research use key prompt elements to push AI towards the desired direction.

5. Conclusions

This study explored the potential of ChatGPT, a free, publicly available, and open-access AI tool, to replace or contribute to experts’ diagnoses in identifying building energy renovation measures. The findings demonstrate a significant improvement from the 2023 to 2024 versions and that, while AI provides a promising starting point for non-expert users, its current capabilities fall short of substituting professional expertise.

The AI tool performed well in generating broad and diverse recommendations, often surpassing conventional internet searches in scope and accessibility. This capacity to provide initial insights quickly can enhance understanding and support early-stage planning in energy renovation projects. However, its limitations include medium-quality responses in general and the need for an experienced user to obtain better responses, as proven by the Q2 and Q3 preparation and responses. In addition, inconsistent citation of references and reliance on outdated or public information are present in many answers. These factors highlight the necessity of building renovation experts’ involvement to verify, refine, and tailor AI-generated recommendations to each specific building context. In any case, the iterations evidence that AI’s response quality may still be unpredictable in complex scenarios.

The second stage, conducted with recent versions and other AI tools in late 2024, found promising results that can apply more quantitative calculations to improve the original trend of 2023 AI chatbots. For future studies, the instructions given and, in particular, the definition of the prompt should be reviewed to fulfil AI chatbots’ potential.

During this study, the given building renovation recommendations were more focused on passive improvements (thermal envelope, airtightness, and ventilation) and less focused on active improvements, where they mainly mentioned solutions like PV panels and heat pump devices. Further studies would be required to continue assessing the validity of these recommendations and the AI’s given calculations.

Overall, while ChatGPT can complement building renovation expert efforts, it cannot yet replace the nuanced judgment and adaptability of human professionals in the challenging field of building energy renovation.