Climate-Resilient Reinforcement Learning Control of Hybrid Ventilation in Mediterranean Offices Under Future Climate Scenarios ^†

Abstract

This study develops an explainable reinforcement learning (RL) control framework for hybrid ventilation in Mediterranean office buildings to enhance thermal comfort, energy efficiency, and long-term climate resilience. A working environment was created Using EnergyPlus to represent an office test cell equipped with natural ventilation and air conditioning. The RL controller, based on Proximal Policy Optimization (PPO), was trained exclusively on present-day Typical Meteorological Year (TMY) data from Beirut and subsequently evaluated, without retraining, under future 2050 and 2080 climate projections (SSP1-2.6 and SSP5-8.5) generated using the Belcher morphing technique, in order to quantify robustness under projected climate stressors. Results showed that the RL control achieved consistent, though moderate, annual HVAC energy reductions (6–9%), and a reduction in indoor overheating degree (IOD) by about 35.66% compared to rule-based control, while maintaining comfort and increasing natural ventilation hours. The Climate Change Overheating Resistivity (CCOR) improved by 24.32%, demonstrating the controller’s resilience under warming conditions. Explainability was achieved through Kernel SHAP, which revealed physically coherent feature influences consistent with thermal comfort logic. The findings confirmed that physics-informed RL can autonomously learn and sustain effective ventilation control, remaining transparent, reliable, and robust under future climates. This framework establishes a foundation for adaptive and interpretable RL-based hybrid ventilation control, enabling long-lived office buildings in Mediterranean climates to reduce cooling energy demand and mitigate overheating risks under future climate change.

1. Introduction

Mediterranean offices are facing intensified heat waves, while climate change is driving up average temperatures. In such circumstances, where natural ventilation is leveraged as a cooling and ventilation strategy in a Mediterranean climate, it is becoming risky to rely on natural ventilation (NV) due to its dependency on outdoor conditions. In Lebanon, Younes et al. assessed NV resilience using morphed weather forecasts and observed a substantial NV comfort hours loss from 40% of annual hours under the typical current weather conditions to 26% by 2080 in a high-emissions pathway with reduced building resilience [1]. The increasing heat waves and future warming are narrowing the availability of comfortable NV hours, thus forcing buildings to use hybrid ventilation (HV), a control-driven switching mechanism between NV and air conditioning (AC). Control strategies for HV include rule-based, predictive, fuzzy logic, reinforcement learning, and adaptive control algorithms that switch between NV and AC based on indoor and outdoor conditions. Hamdy and Mauro studied rule-based control hybrid ventilation strategies in an open-plan office and found that all strategies reduce energy consumption by up to 75%, while maintaining thermal comfort and air quality [2]. Vallianos et al. also developed model predictive control strategies for hybrid ventilation in an institutional building using both simulations and experiments. These strategies resulted in substantial energy savings while maintaining thermal comfort and outperforming rule-based control strategies [3]. Predictive control outperforms rule-based control due to its ability to foresee the impact of the current action on the long term and to foresee the changing future inputs and initial conditions. However, predictive control is model-based, which requires a comprehensive and extensive building energy modeling framework including on-site validation. Therefore, reinforcement learning (RL) overcomes the rigidity of rule-based control and the model-dependence of model predictive control by learning optimal control strategies from data, adapting to changing conditions, and handling highly complex non-linear building dynamics. RL is a data-driven control strategy where an agent learns the optimal control policy through trial-and-error interactions with the environment, guided by a reward function [4].

Reinforcement learning has been increasingly investigated for HVAC and building energy control problems due to its ability to learn adaptive control policies in complex, sequential decision environments, where traditional model-based approaches such as PID and MPC often struggle with high dimensionality and dynamically varying conditions [5]. Recent comprehensive literature surveys show that RL applications in intelligent HVAC control have grown substantially in the past few years, with studies covering algorithmic choices, state/action formulation, and evaluation methodologies, revealing both significant potential and persistent challenges in real-world deployment [5]. Several empirical works have benchmarked state-of-the-art deep reinforcement learning (DRL) algorithms such as Soft Actor-Critic (SAC) and Twin Delayed DDPG (TD3) in terms of energy efficiency and comfort trade-offs, demonstrating advantages over conventional controllers but also highlighting issues in robustness and generalization when environments are complex or non-stationary [6]. Other research has explored improvements in RL efficiency and practicality, including offline RL approaches that leverage historical operational data to mitigate training costs and enhance feasibility [7], and advanced architectures such as pretrained decision models aimed at improving generalization across buildings without extensive retraining [8]. Moreover, hybrid frameworks that integrate expert knowledge with RL have been proposed to accelerate convergence and incorporate domain constraints more effectively [9].

In the context of hybrid ventilation, Chen et al. developed an RL controller for HV to optimize thermal comfort and energy efficiency under two different US climates, Miami and Los Angeles [10]. RL reduced Heating, Ventilation, and Air Conditioning (HVAC) systems’ energy consumption by 13% and 23% and uncomfortable hours by 62% and 80% compared to rule-based control for Miami and Los Angeles, respectively [10]. Dai et al. also developed an RL algorithm for mixed-mode ventilation in a tropical climate [11]. By controlling window opening area and dynamic cooling AC setpoints, the RL strategy achieves a 52% reduction in cooling energy compared to conventional control, while maintaining thermal comfort and indoor air quality [11].

Recent research has also explored transfer learning and domain adaptation to improve generalization of reinforcement learning-based building control strategies across different buildings, climates, and operating conditions. For instance, Kadamala et al. demonstrated that pre-training an RL agent on a source building and fine-tuning it for new environments can outperform training from scratch, saving time and computational cost while improving control performance [12]. Coraci et al. proposed a heterogeneous transfer learning framework that enhances the scalability of deep RL controllers across buildings with diverse energy systems and environmental conditions by transferring pre-trained models to new target domains [13]. More recently, continual reinforcement learning approaches have been developed that allow a single agent to adapt efficiently across multiple tasks and environments, mitigating catastrophic forgetting and improving sample efficiency [14]. While these works actively adapt or fine-tune policies for new domains, the present study instead evaluates the robustness of a controller trained under present-day conditions when deployed under projected future climates without retraining, providing a baseline resilience assessment for climate-stressed operation.

The aforementioned studies demonstrate the strong potential of RL for optimizing HV in buildings to enhance thermal comfort and energy efficiency. However, the RL agents were trained and tested on current typical weather files and datasets that represent previous average climate conditions, without accounting for extreme events. Therefore, the trained agents are not fully prepared to handle severe and extreme weather conditions such as future heat waves. The agents are neither trained nor evaluated explicitly under future climate scenarios such as 2050 or 2080 projections, which heavily impacts the controller’s robustness. These 2050 and 2080 projections have higher baseline temperatures, different humidity patterns, and increased frequency of heat waves, which significantly alters the buildings’ cooling demands and occupant comfort. This omission raises concerns about the resilience and adaptability of these RL-based control strategies when deployed in real-world changing climates. A controller that performs optimally under typical weather conditions might experience performance degradation, unstable control behavior, or increased discomfort hours when faced with unprecedented environmental conditions.

Although RL demonstrates significant potential in HV and HVAC control, it is accompanied by a critical barrier that its policy decisions are often vague and not easily comprehended or trusted by operators. This means that RL has a black-box nature, which raises concerns about safety, reliability, and user acceptance, especially in the built environment where human thermal comfort is at stake. This limitation is nowadays addressed by Explainable Artificial Intelligence (XAI), which provides interpretable and transparent explanations of policy actions, thus allowing operators and users to understand, verify, and trust automated decisions by AI algorithms such as RL. XAI uses techniques like surrogate models, Shapley values (SHAP), counterfactual analysis, and decision-tree distillation to clarify why certain policy actions were implemented by the RL agent, without compromising its performance. Su et al. developed an XAI framework with RL for a multi-zone household HVAC system that achieved 22.4% lower energy consumption while ensuring thermal comfort [15]. XAI enabled interpretable continuous control decision trees, thus maintaining transparency for non-expert auditing and monitoring [15]. An et al. also developed an interpretable RL framework for HVAC systems, extracting decision trees representing the policy decisions to ensure deterministic control logic [16]. This framework saved 68.4% more energy consumption and enhanced thermal comfort by 14.8% compared to standard RL [16]. Gokhale et al. developed a novel framework for extracting interpretable control policies from deep reinforcement learning (DRL) agents managing residential HVAC systems that turns complex RL HVAC policies into human-readable decision trees [17]. The results indicate a reduction in energy use of 20–25% and a boost in comfort of 12–15% over rule-based control, while keeping performance close to the original RL agent [17]. Therefore, explainable AI (XAI) can optimize the performance of RL by revealing the nature of decision-making, thus allowing humans to intervene, fine-tune logic, deploy RL more confidently, improve resilience to extreme events, and analyze hidden optimization patterns, leading to enhanced performance over RL alone.

Despite the promising results of RL in optimizing hybrid ventilation systems and the added transparency provided by XAI, existing studies present several limitations. First, most RL-based HV controllers have been trained and evaluated only under present-day typical meteorological year (TMY) conditions, neglecting projected climate scenarios with extreme weather conditions. This omission risks overestimating performance and failing to guarantee operational resilience in the long term. Moreover, existing RL–XAI studies for mixed-mode or HVAC control primarily emphasize short-term energy and comfort optimization, without explicitly framing controller robustness as a climate resilience problem. Second, while XAI has been applied to HVAC systems, XAI-enhanced RL specifically for hybrid ventilation control under both present and future climates, particularly in the Mediterranean context, has remained unexplored. Third, resilience metrics are evaluated using RL control, leaving a gap in quantifying the robustness of such controllers against climate change stressors.

Therefore, this study addresses the identified gaps by proposing an explainable reinforcement learning-based control framework for hybrid ventilation in a Mediterranean office, with a particular focus on long-term climate resilience under projected future weather conditions. The specific contributions and outcomes of this work are summarized as follows:

• A climate-resilient reinforcement learning control framework for hybrid ventilation is proposed, enabling Mediterranean office buildings to maintain thermal comfort while reducing reliance on energy-intensive mechanical cooling under both present-day and projected future climate conditions;
• A post-training resilience evaluation methodology is introduced, in which an RL controller trained exclusively under present-day TMY conditions is systematically assessed under morphed 2050 and 2080 climate scenarios, providing a practical approach for evaluating long-term robustness without future-data training;
• Quantitative resilience indicators are applied to objectively measure the resistance of RL-based hybrid ventilation control against climate-induced thermal stress, relative to conventional rule-based strategies;
• Explainable artificial intelligence is integrated with reinforcement learning through SHAP-based feature attribution, enabling transparent interpretation of control decisions and revealing physically consistent relationships between environmental variables and ventilation actions;
• A Mediterranean office case study is presented to demonstrate the practical implications of the proposed framework, highlighting its potential to enhance long-term operational resilience while sustaining energy efficiency under increasing heat-wave intensity.

While energy efficiency improvements are reported, the primary contribution of this study is the evaluation of long-term climate resilience and interpretability of reinforcement learning-based hybrid ventilation control, rather than energy maximization.

2. Methodology

The methodology of this study consists of six main parts: (i) describing the experimental office cell and RL control operation; (ii) developing a building energy model of the typical office; (iii) formulating an RL framework for hybrid ventilation using the building energy model; (iv) implementing the developed RL control on current and future weather conditions; (v) evaluating the impact of RL in enhancing building resilience compared to a rule-based control; and (vi) using an XAI framework to provide insights into the RL strategy, according to Figure 1.

2.1. Experimental Office Cell and System Description

The office under study corresponds to a full-scale experimental test cell located on the top floor of a four-story building in Beirut, Lebanon. The building geometry, envelope characteristics, internal loads, ventilation configuration, and HVAC system were previously developed, experimentally validated, and comprehensively described in Krayem et al. [18]. In the present study, the same validated office model is directly adopted without modification to ensure physical realism and comparability of results. The office corresponds to a lightweight, single-zone experimental test cell with a dominant south-facing glazed façade and limited thermal mass, representative of Mediterranean office typologies. The cell is equipped with an automated window opening system and an inverter-driven split air conditioning unit, featuring a coefficient of performance (COP) of 3.8 and a gross rated total cooling capacity of 2.7 kW. The control of the window actuation or AC activation is achieved using RL to guide the switching between NV and AC. The RL controller is trained on present-day TMY data for an office space in the Mediterranean climate. The agent, which consists of the developed policy and the RL algorithm, learns from the environment’s rewards and observations after sending actuating signals, as shown in Figure 2. The environment in this context represents the building energy model of the test cell. Then, the developed RL policy will be evaluated on the TMY data and the morphed future climates in 2050 and 2080, with two different scenarios depending on possible net-zero emissions pathways in the future. Although reinforcement learning is commonly associated with large-scale datasets, the training data in this study are generated through online interaction with a high-fidelity simulation environment rather than collected from field measurements. Specifically, the RL agent acquires data by repeatedly interacting with the validated building energy model, which provides state observations and rewards at every control timestep. This simulation-based training paradigm enables the generation of large and diverse state–action–reward trajectories in a controlled and reproducible manner, while avoiding the practical limitations of long-term real-world data acquisition.

To test whether the RL policy remains reliable under future weather conditions, its performance is compared to a rule-based control algorithm in terms of different thermal resilience metrics that evaluate indoor overheating and ambient warmness degrees due to climate change and future heat waves, along with evaluating the resistivity of the RL control to climate change. Furthermore, explainability algorithms will also be implemented for the developed RL agent to evaluate the most influential conditions that favor the implementation of a specific policy or action. Therefore, this study aims to design a control policy that (i) limits indoor overheating under heat waves; (ii) maintains energy efficiency; and (iii) remains interpretable to users. A closely related reinforcement-learning–based hybrid ventilation control framework was previously demonstrated by the authors for a Mediterranean classroom environment, providing additional methodological grounding for the present office-based evaluation [19].

2.2. Building Energy Modeling

To develop an RL framework, a building energy model is required to serve as the environment to be utilized by the agent. Therefore, the reinforcement learning environment is based on a previously developed and experimentally validated model of the office test cell, originally presented in a previous study [18]. Accordingly, the present work focuses on RL control behavior, resilience assessment, and explainability, rather than on building envelope performance or sensitivity. In this context, the office envelope corresponds to a lightweight, single-zone test cell with limited thermal mass and a dominant south-facing glazed façade, which governs solar-driven overheating dynamics and wind-assisted heat removal mechanisms considered in the subsequent analysis. The model is developed using EnergyPlus 24.2.0, a physics-based building performance simulation tool widely adopted for assessing indoor thermal conditions and energy consumption in buildings under different ventilation systems [20]. The model consists of building envelope characteristics, occupancy schedules, equipment and internal loads, HVAC configuration, natural ventilation system, and weather data obtained from the Beirut International Airport station. To account for the site’s urban morphology, which differs from that of the meteorological station, the local wind speed coefficients used in the simulation are corrected using the terrain-based wind profile coefficients recommended in ASHRAE 2001, assuming an urban (towns/cities) terrain category to better represent the microclimatic wind profile at the office site [21]. The internal loads include two occupants, each performing sedentary work (1.2 met), 36-Watts of lighting, and 50-Watts of electric equipment, synchronized with the same operating schedule of working hours from 8:00 AM till 5:00 PM [22]. Natural ventilation is modeled using the Airflow Network (AFN) feature, which captures multi-zone air movement driven by pressure differentials and wind effects. The AFN framework is chosen for its flexibility in toggling window openings for individual zones, as previously demonstrated by Sakiyama et al. [23]. In this study, the openings are treated as binary states (fully open or closed). For mechanical cooling, the office utilizes a Variable Refrigerant Flow (VRF) system configured to emulate the behavior of an inverter-type split air conditioner since EnergyPlus lacks a direct split-unit object. The VRF’s part-load performance curves are modified to match the real system’s compressor-speed characteristics, ensuring accurate representation of energy performance.

The developed EnergyPlus model is then configured to interface with the RL framework through a co-simulation setup. The essential simulation variables such as the weather conditions, zone mean air temperature, relative humidity, and HVAC energy consumption are exposed as observation handles accessible to the RL agent at every control timestep. Similarly, the control variables governing the window opening, the AC activation schedule, and cooling setpoint values are defined as actuator handles through which the agent interacts with the environment. These two sets of handles enable the two-way communication: the RL agent observes the office’s real-time thermal and energy states, determines the optimal control action, and sends it back to EnergyPlus for implementation in the subsequent timestep, then receives the new observation to perform policy updates.

2.3. Reinforcement Learning

The hybrid ventilation control problem is formulated as a Markov Decision Process (MDP), defined by a tuple ⟨S, A, P, R, γ⟩, where S denotes the state space, A the action space, P the environment transition dynamics governed by the building physics, R the reward function, and γ the discount factor. The RL agent interacts with the EnergyPlus-based environment at discrete control timesteps of 20 min.

At each control timestep, the agent observes a state vector $S_t={\left[T_{in}, R{H}_{in}, T_{out}, R{H}_{out},WS, WD, Occ, E_{AC}\right]}_t$ consisting of the indoor temperature and relative humidity, outdoor temperature and relative humidity, wind speed and direction, occupancy, and AC energy consumption, respectively [4], resulting in large-scale interaction data over long-horizon simulations that are sufficient for stable policy learning. This state formulation captures both the internal thermal response of the office and the external driving forces governing natural ventilation potential. With a control timestep of 20 min, this results in 26,280 state transitions per simulated year, enabling large-scale interaction data over long-horizon simulations.

The action space is discrete and maps directly to EnergyPlus actuators $A_t=[NV, AC, T_{sp}, Off]$. Safety guards are enforced in the environment to ensure that $NV$ and $AC$ are not simultaneously activated, that NV and AC are off during unoccupied hours, and that $Off$ is not allowed during occupied hours. These constraints restrict physically infeasible or operationally unsafe actions, allowing the agent to focus on learning optimal hybrid ventilation decisions within a realistic control envelope rather than rediscovering basic physical or operational rules.

Thermal comfort is evaluated using the Predicted Mean Vote (PMV) using the ASHRAE 55 standard [24]. The PMV has a 7-point scale between −3 (too cold) and +3 (too warm), with 0 defined as thermoneutrality. The PMV calculation model requires the indoor operative temperature, relative humidity, clothing insulation, metabolism rate, and indoor relative velocity. The indoor operative temperature is assumed to be equal to the zone mean air temperature $T_{in}$, using the same office test cell and EnergyPlus model [18]. In that work, experimental measurements showed that, due to the lightweight construction, small room size, limited thermal mass, partial shading of the south-facing façade, and high air mixing under both natural and mechanical ventilation modes, the mean radiant temperature closely tracked the indoor air temperature, resulting in negligible operative–air temperature differences within the comfort assessment range. The clothing insulation is set to 0.5 in summer and 1 in winter [25]. The metabolism rate is set to 1.2 met, corresponding to sedentary work [26]. Indoor relative velocity is set to 0.2 m/s [27]. However, the PMV predictions can be used only during mechanical ventilation modes with fixed indoor thermal conditions, and not NV. Thus, to account for behavioral adaptation and physiological aspects of NV, the adaptive PMV (aPMV), developed by Yao et al. [28] and recently used by Conceição et al. [29] in a Mediterranean climate, is used to replace the PMV using the following Equation (1) and the physiological–psychological adaptation factor, $\lambda$.

$$\begin{array}{c}aPMV={\displaystyle \frac{PMV}{1+\lambda \times PMV}}\hfill \\ \lambda =\left\{\begin{array}{c} 0.293, \hspace{1em}for PMV\ge 0\\ -0.125, \hspace{1em}for PMV<0\end{array}\right.\hfill \end{array}$$

(1)

Therefore, the PMV (and aPMV in the case of NV) predictions to each indoor temperature at every control timestep are used in the reward function.

The reward function merges comfort, energy, and percentage of NV hours during the year, but in normalized terms, as shown in Equations (2)–(5). ${\omega }_1,{\omega }_2, {\omega }_3$ are weighted factors for each instant reward, respectively, 0.5, 3.5, and 1. This design strongly promotes NV when feasible, yet preserving comfort at minimal AC use. The weighting factors are selected to preserve the same relative priority between comfort, energy, and natural ventilation preference as adopted in prior reinforcement learning-based mixed-mode ventilation studies [11]. These weights reflect a design preference rather than an optimized scalarization and are fixed throughout training and evaluation to ensure consistent control operation. The reward function is designed to balance thermal comfort, energy consumption, and natural ventilation utilization, guiding the agent toward physically realistic and energy-efficient hybrid ventilation strategies.

$$r_t=-{\omega }_1\times r_{comfort}-{\omega }_2\times r_{energy}+{\omega }_3\times r_{NV}$$

(2)

$$r_{comfort}={\displaystyle \frac{\left|PMV\right|}{PM{V}_{max}}}={\displaystyle \frac{\left|PMV\right|}{3}}$$

(3)

$$r_{energy}={\displaystyle \frac{EC}{E{C}_{max}}}$$

(4)

$$r_{NV}={\displaystyle \frac{N{V}_{occ}}{Number of Occupied Hours}}$$

(5)

All instantaneous reward terms are normalized using min–max scaling to ensure consistent magnitude ranges between comfort, energy, and natural ventilation objectives. This normalization ensures comparable numerical ranges across reward components, preventing any single term from disproportionately dominating the policy gradient during training.

The environment transition dynamics $P\left(S_{t+1}∣S_t,A_t\right)$ are implicitly governed by the physics-based EnergyPlus model, which simulates heat transfer, airflow, and HVAC system response to control actions.

The control policy $\pi \left(A\right|S)$ is learned using Proximal Policy Optimization (PPO), selected for its stability in continuous-state, discrete-action control problems with long horizons. The PPO agent is trained following an on-policy interaction–update loop. At each control timestep, the agent samples an action from the stochastic policy based on the current state and applies it to the EnergyPlus environment. The environment advances one timestep and returns as output the next state and corresponding reward. These state–action–reward transitions are collected over a fixed rollout horizon and stored in a buffer. Once the rollout length is reached, the policy and value networks are updated using the PPO clipped objective function, with advantages estimated through generalized advantage estimation (GAE) to improve training stability. This interaction and cycle update are repeated over the full training horizon until convergence is achieved. The PPO is built with a shared multi-layer perceptron with 512-512 architecture policy and tanh activations. Four parallel EnergyPlus environments are wrapped with VecMonitor and VecNormalize that perform observation normalization. The key hyperparameters upon calibration are as follows:

• Total timesteps = 12 million timesteps (with four parallel envs);
• Learning rate = $2\times {10}^{-4}$;
• $\gamma =0.995$, ${\lambda }_{GAE}=0.95$;
• Rollout length $n_{steps}=8192 per env$;
• Batch size = 1024, Epochs = 12;
• Entropy coefficient 0.05, clip range 0.10, value-clip 0.10;
• Value-function coefficient 1.5, max-grad-norm 0.5.

The designed PPO agent is trained and developed end-to-end on the present-day Beirut TMY to learn robust NV/AC switching strategies. After achieving convergence, the developed policy that is trained on TMY weather data is saved to be used to perform switching on future weather forecasts. No retraining, fine-tuning, or policy adaptation is performed under future climate scenarios; the controller trained under present-day TMY conditions is directly deployed for all future climate evaluations.

The observation space was designed to ensure causality and control relevance while remaining compact for stable learning. Indoor temperature, outdoor conditions, occupancy, and HVAC energy consumption were included to provide direct feedback on comfort states and the immediate energetic impact of control actions. Although energy is penalized in the reward, its inclusion in the state improves convergence by reducing delayed credit assignment. Solar radiation and spatial temperature gradients were not explicitly included, as their effects are inherently captured through the physics-based building model and indoor temperature dynamics. Occupancy was modeled as a deterministic binary schedule to isolate climate-driven effects on control performance.

The performance of the RL control is compared to a rule-based control strategy developed and used as a benchmark. The control logic relies solely on outdoor air temperature to determine the operation mode. The RBC strategy is formulated as a deterministic switching logic based solely on the outdoor air temperature $T_{out}$. At each control timestep, the ventilation mode is selected according to the following rules:

• Natural Ventilation

  $$if 18 °\mathrm{C} \le T_{out}\le 26 °\mathrm{C}, NV is activated$$
• Mechanical Ventilation—High Setpoint:

  $$if T_{out}<18 °\mathrm{C}, AC is activated with T_{sp}=26 °\mathrm{C}$$
• Mechanical Ventilation—Low Setpoint:

  $$if T_{out}>26 °\mathrm{C}, AC is activated with T_{sp}=21 °\mathrm{C}$$

This RBC provides a transparent and interpretable baseline against which the adaptive and data-driven behavior of the RL controller is evaluated. The RBC is intentionally formulated as a simple outdoor-temperature-based strategy to serve as a transparent and conservative benchmark, rather than an optimized controller that explicitly accounts for indoor state feedback.

To assess the robustness of the proposed RL controller with respect to reward formulation, a sensitivity analysis was conducted by varying the relative weights assigned to comfort (ω₁), energy consumption (ω₂), and natural ventilation preference (ω₃). Without retraining the policy, cumulative annual rewards were recomputed for both the RL and rule-based controllers across a broad range of plausible weight combinations using the recorded reward components.

2.4. Future Weather Conditions

To evaluate the impact of the RL control of hybrid ventilation on comfort, energy, and natural ventilation potential in the face of climate change, the RL policy was evaluated on TMY weather data and future climate projections in 2050 and 2080 under two Shared Socioeconomic Pathways (SSPs): SSP1-2.6, a low-emission sustainable development trajectory; and SSP5-8.5, a high-emission, fossil-fueled growth pathway. Thus, the future climate data were derived by adapting the baseline TMY dataset through the Belcher morphing technique [30]. This tool combines recent IPCC projections with higher resolution to produce localized weather files that reflect changing climatic trends to morph air temperature, humidity, solar intensity, and wind profiles [31]. The Belcher morphing technique was applied using the Future Weather Generator tool. Dry-bulb temperature, humidity-related variables, solar radiation, wind speed, atmospheric pressure, and other meteorological parameters were morphed, while wind direction was assumed constant. The morphing approach preserves the temporal structure of the original weather file while shifting statistical properties to reflect future climates. While this method may underrepresent short-duration extreme heat wave amplification, the present study evaluates resilience through cumulative overheating and indoor–outdoor sensitivity metrics over long simulation periods, capturing sustained thermal stress rather than isolated extremes.

Therefore, the developed RL controller was evaluated on the TMY weather file, along with the following morphed weather files:

• 2050 SSP1-2.6;
• 2050 SSP5-8.5;
• 2080 SSP1-2.6;
• 2080 SSP5-8.5.

The evaluation of the RL controller on the five weather files produced in control timestep-based EnergyPlus output files showing the observation space and actuating signals to be used for evaluation in terms of building thermal resilience metrics.

2.5. Building Thermal Resilience Assessment

Building thermal resilience is defined as the building’s capacity to withstand extreme heat events while maintaining acceptable indoor thermal comfort without excessive reliance on traditional cooling systems [32]. Thermal resilience is quantitatively evaluated through three interconnected metrics: Indoor Overheating Degree (IOD), Ambient Warmness Degree (AWD), and the Climate Change Overheating Resistivity (CCOR), all endorsed by the International Energy Agency Energy in Buildings and Communities Programme (IEA EBC) Annex 80 [33]. The operative comfort temperature ($T_{OC}$), from the de Dear and Brager model according to ASHRAE standards 2017 [24] is computed using Equation (6) to determine upper ($T_{OC, high}$) and lower ($T_{OC,low}$) bounds shown in Equations (7) and (8) to be used in the computation of the resilience metrics [22].

$$T_{OC}=17.8+0.31 T_{out}$$

(6)

$$T_{OC, high}=T_{OC}+2.5 °\mathrm{C}$$

(7)

$$T_{OC,low}=T_{OC}-2.2 °\mathrm{C}$$

(8)

IOD quantifies the intensity and duration by which indoor temperatures exceed comfort thresholds, reflecting cumulative indoor overheating [34]. It is calculated as the cumulative sum of the positive differences between the indoor operative temperature ($T_{op}$) and $T_{OC, high}$ only during occupied hours, as shown in Equation (9).

$$IOD={\displaystyle \frac{{\sum }_{k=1}^{Nocc}{(T_{op,k}-T_{oc,high})}^{+}·t_k}{{\sum }_{k=1}^{Nocc}{T}_k}}$$

(9)

Here, $k$ denotes the simulation control timestep, $N_{occ}$ is the number of occupied control timesteps, $T_{op,k}$ is the operative temperature at timestep $k$, $T_{oc,high}$ is the upper comfort temperature limit, and $t_k$ is the duration of each timestep, which is 20 min in this case. The notation ${\left(X\right)}^{+}$ indicates that only positive temperature differences are counted towards calculating IOD, while negative ones are ignored [35].

AWD represents the intensity of outdoor heat stress by accumulating the cooling degree hours above a baseline temperature [34]. It is computed using the following Equation (10):

$$AWD={\displaystyle \frac{{\sum }_{k=1}^{Nocc}{(T_{out,k}-T_b)}^{+}·t_k}{{\sum }_{k=1}^{Nocc}{t}_k}}$$

(10)

Here, $T_{out,k}$ is the outdoor dry-bulb temperature and $T_b$ is the base temperature set at $18 °\mathrm{C}$ for a temperate climate. The AWD base temperature was fixed at $18 °\mathrm{C}$ in accordance with the established resilience assessment literature to ensure comparability and to represent outdoor thermal forcing rather than adaptive comfort thresholds [35].

CCOR represents the building’s cooling ability to limit indoor overheating with regard to the increasing outdoor heat levels [34]. CCOR is determined by performing a linear regression between IOD and AWD and expressed as Equation (11) below:

$${\displaystyle \frac{1}{CCOR}}={\displaystyle \frac{{\sum }_{Sc=1}^M\left({IOD}_{sc}-\overline{IOD}\right)\times ({AWD}_{sc}-\overline{AWD}) }{{\sum }_{Sc=1}^M{\left({AWD}_{sc}-\overline{AWD}\right)}^2}}$$

(11)

where $Sc$ denotes the weather scenario index, and $M$ is the total number of scenarios. $IO{D}_{Sc}$ and $AW{D}_{Sc}$ represent the indoor and outdoor degree metrics for each scenario, while $\overline{IOD}$ and $\overline{AWD}$ are their mean values across all scenarios. The regression quantifies how indoor overheating responds to outdoor temperature variations under different climate conditions. The CCOR value is obtained as the reciprocal of the regression slope, thus reflecting the sensitivity of indoor overheating to outdoor warming. The CCOR is interpreted as a deterministic sensitivity indicator that characterizes the trend of indoor overheating response to increasing outdoor thermal stress across standardized climate scenarios, rather than as a statistical estimator, consistent with previous CCOR-based resilience studies [1,36].

In practical terms, optimal thermal resilience corresponds to low IOD and high CCOR even if under high AWD. Thus, the resilience metrics such as the IOD and CCOR will be evaluated to address the ability of RL in enhancing a building’s thermal resilience under the five different weather files.

2.6. Explainable AI

To interpret the decision-making process of the RL control, it is necessary to reveal the impact of each input variable and element in the observation space on the selected ventilation mode. Thus, the Kernel SHAP (Shapley Additive Explanations) method is employed. SHAP assigns each feature an importance value, known as the Shapley value, which constitutes the average contribution of that feature to the model output across all possible feature combinations. This provides a transparent understanding of which parameters, as the indoor and outdoor conditions, most strongly affect the agent’s actions [37].

Kernel SHAP integrates the local linear approximation concept with the Shapley value estimation principle, enabling model-agnostic explanations that capture the local importance of features around a given state. The Shapley value for each feature $i$ is computed using Equation (12):

$${\varphi }_i\left(f,x\right)= \sum _{z^{\prime }\subseteq z^{\prime }}{\displaystyle \frac{\left|z^{\prime }\right|!\left(M-\left|z^{\prime }\right|-1\right)!}{M!}}\left[f_x\left(z^{\prime }\right)-f_x\left(0\right)\right]$$

(12)

where $M$ is the number of simplified input features, $z^{\prime }$ represents the binary vectors indicating feature inclusion (1) or exclusion (0), and $f_x\left(z^{\prime }\right)$ is the model output when only the subset $z\prime$ of features is active.

Kernel SHAP combines the local linear approximation principle with the Shapley value framework to approximate feature contributions through a locally weighted linear model:

$$g\left(z^{\prime }\right)={\varphi }_0+\sum _{i=1}^M{\varphi }_i{z}_i\prime$$

(13)

Here, ${\varphi }_0$ is the baseline model output, and $g\left(z^{\prime }\right)$ represents the simplified surrogate model fitted around the original instance. The model is trained by minimizing a weighted least-squares loss function:

$$\xi =\arg \underset{\mathit{g}\in \mathit{G}}{\min } L(f,g,{\pi }_{x^{\prime }})+\Omega (g)$$

(14)

$L(f,g,{\pi }_{x^{\prime }})$ represents the weighted loss across sampled feature subsets, ${\pi }_{x^{\prime }}$ is the kernel weighting function, and $\Omega \left(g\right)$ regularizes the model complexity. These are defined as

$$L\left(f,g,{\pi }_{x^{\prime }}\right)=\sum _{z^{\prime }\in Z}{\left[f\left(h_x\left(z^{\prime }\right)\right)-g\left(z^{\prime }\right)\right]}^2{\pi }_{x^{\prime }}\left(z^{\prime }\right)$$

(15)

$${\pi }_{x^{\prime }}\left(z^{\prime }\right)={\displaystyle \frac{M-1}{\left(\genfrac{}{}{0pt}{}{M}{\left|z^{\prime }\right|}\right)\left|z^{\prime }\right|\left(M-\left|z^{\prime }\right|\right) }}$$

(16)

Following Equations (15) and (16), the kernel weighting function ${\pi }_{x^{\prime }}\left(z^{\prime }\right)$ assigns greater weight to samples closer to the original input, ensuring that the estimated Shapley values ${\varphi }_i$ capture the local influence of each feature on the RL agent’s decision. This weighting mechanism enables Kernel SHAP to approximate Shapley values efficiently for complex, non-linear, and black-box RL models. In this study, the SHAP values quantify the relative importance of each input on the agent’s decision to activate a ventilation mode. Consequently, the Kernel SHAP framework provides a transparent interpretation of the RL control strategy, offering valuable insights into the underlying decision logic under varying environmental conditions. In this study, SHAP values are computed over the PPO policy network’s action-probability logits, allowing for interpretation of how each observational input influenced the selection likelihood of every ventilation or cooling action. SHAP explanations are generated using all timesteps from the full-year simulations of the trained policy. While SHAP explanations largely reflect expected physical trends, their value here lies in demonstrating that the learned hybrid ventilation policy remains physically consistent and interpretable under future climate scenarios, thereby supporting trust and robustness in resilience-oriented control applications. It is noted that SHAP provides post hoc explanations of learned policy behavior and is used here to assess physical consistency and transparency, rather than to establish causal relationships or validate physical laws.

Kernel SHAP was employed as an offline post hoc analysis tool applied to the trained policy, where computational cost was manageable due to the moderate state dimensionality and discrete action space. As a model-agnostic approximation method, Kernel SHAP trades exact Shapley value computation for tractable surrogate-based explanations, which are sufficient here to assess the physical consistency and robustness of the learned control logic rather than exact causal attribution.

3. Results and Discussion

3.1. Performance of the Reinforcement Learning Controller

3.1.1. Training Convergence and Learning Stability

Figure 3 presents the convergence behavior of the PPO-based reinforcement learning controller trained under present-day TMY conditions. The mean reward per timestep initially starts at a low negative value (approximately −3.5), reflecting exploratory behavior dominated by high cooling energy use and suboptimal ventilation decisions. As training progresses, the reward increases monotonically, indicating that the agent progressively learns to reduce energy penalties while maintaining thermal comfort and exploiting natural ventilation when feasible. At the end, the reward stabilizes around −0.1, demonstrating stable convergence of the learned policy. The PPO agent exhibits rapid learning during the early training phase, followed by gradual stabilization, reaching convergence after approximately 8–9 million timesteps. The absence of oscillatory or divergent behavior confirms that the PPO agent achieves reliable and consistent learning dynamics, and that the resulting policy is well-converged prior to its evaluation under present-day and future climate scenarios.

3.1.2. Energy Performance and Natural Ventilation Utilization

The performance of the developed RL control policy is evaluated and compared to the rule-based control strategy under the five different weather conditions for hybrid ventilation. All results reported for 2050 and 2080 climates are obtained using the policy trained under present-day TMY conditions.

Table 1. Annual HVAC energy consumption, energy savings, and natural ventilation utilization under present and future climate scenarios.

Scenario	Controller	Annual HVAC Energy (kWh/year)	Energy Savings (%)	NV During Occupied Hours (%)
TMY	RBC	332.73	–	49.34
	RL	302	9.23	56.79
2050-SSP1-2.6	RBC	451.57	–	34.18
	RL	412.40	8.67	41.92
2050-SSP5-8.5	RBC	472.28	–	32.16
	RL	432.12	8.51	40.05
2080-SSP1-2.6	RBC	475.60	–	32.05
	RL	433.74	8.80	40.13
2080-SSP5-8.5	RBC	502.54	–	32.39
	RL	467.91	6.89	39.80

and Figure 4 compare the annual HVAC energy consumption and the percentage of natural ventilation (NV) during occupied hours for RBC and RL under the TMY and four future climate scenarios (2050 and 2080 for SSP1-2.6 and SSP5-8.5).

Under typical weather conditions (TMY), the RL control was able to achieve an annual reduction of 9.23% in HVAC energy consumption compared to 332.7 kWh/year for RBC. This improvement highlights the ability of the RL agent to dynamically plan and program the optimal switching between NV and AC modes at different temperature setpoints, thus exploiting favorable outdoor weather conditions and predictions rather than relying on fixed temperature thresholds that RBC uses. A similar energy saving trend still persists under the future weather files, where in 2050 SSP1-2.6 and SSP5-8.5 scenarios, energy consumption is reduced by 8.67% and 8.5%, respectively. In 2080 SSP1-2.6 and SSP5-8.5, the RL achieves 8.8% and 6.89% reductions relative to RBC. These consistent savings across the increasingly warm weather conditions confirm that the developed RL control policy is robust and adaptable to be employed under future heat wave conditions, with increased temperature and humidity profiles, thus highlighting its ability to sustain its performance without retraining the whole policy again on all weather conditions. While the absolute energy savings achieved by the RL controller are moderate, the primary contribution of this work lies in enhancing thermal resilience, extending natural ventilation operation, and enabling interpretable control decisions under future extreme climate conditions, rather than solely maximizing energy reduction.

The RL controller also increased the percentage of natural ventilation hours during occupied hours in all scenarios and weather conditions, as shown in Figure 4. For TMY, NV operation rises from 49.3% under RBC to 56.8% under RL, marking a 7.72% relative increase. The same increasing trend exists under the future weather conditions, where in 2050 SSP1-2.6 and SSP5-8.5 scenarios, the percentage of natural ventilation hours increases by 7.46% and 7.74%, respectively. In 2080 SSP1-2.6 and SSP5-8.5, the RL achieves increases of 8.08% and 7.41% in NV hours relative to RBC. The agent learns to extend NV usage potentials by recognizing the transitional outdoor weather conditions where NV still satisfies comfort requirements, along with controlling the switching and allowing for intermittent cooling by the AC when needed. This ability to exploit thermoneutral outdoor weather conditions leads to both enhanced occupant thermal comfort, especially when using NV, and energy reduction, since the AC is engaged only when strictly needed. A notable finding is that while total energy demand rises from 332 kWh (TMY) to approximately 500 kWh (2080 SSP5-8.5) due to warmer baseline temperatures and more frequent heat waves, the RL policy preserves its relative energy advantage over RBC. This confirms that the learned policy generalizes effectively to unseen weather conditions. The increasing difference between RL and RBC performance under warmer scenarios demonstrates the potential of RL control over static rule-based methods in climate thermal resilience.

3.1.3. Thermal Comfort Performance

Figure 5 shows that the RL controller maintains thermal comfort during occupied hours within acceptable bounds across all climate scenarios, while exhibiting a more stable and consistent comfort distribution compared to the baseline RBC. Across TMY and future projections, the RL strategy generally shifts the median comfort index closer to neutral and reduces the spread of extreme discomfort events, particularly under high-warming scenarios. This tightening of the comfort distribution indicates reduced oscillatory behavior and more persistent control actions, leading to improved comfort regulation without increasing variability. Notably, under the most severe future scenarios (2080-SSP5-8.5), the RL controller preserves comfort levels comparable to or better than those under milder conditions, suggesting enhanced robustness of the learned policy as climatic stress intensifies. It is noteworthy that the comfort distributions combine PMV during AC periods and aPMV during NV operation, reflecting the appropriate comfort model active at each timestep.

3.1.4. Reward-Weight Sensitivity Analysis

Figure 6 presents the three-dimensional sensitivity analysis, where each data point corresponds to a unique combination of comfort (ω₁), energy (ω₂), and natural ventilation preference (ω₃) weights applied to the same underlying control trajectories. The vertical axis represents the difference in cumulative annual reward between the RL controller and the rule-based benchmark (RL−RBC), while the color scale reflects variations in the natural ventilation preference weight.

The results demonstrate that the RL controller consistently outperforms the rule-based control across the entire explored reward-weight space. Importantly, the relative performance advantage remains positive for all tested combinations and exhibits a smooth and monotonic variation rather than abrupt transitions or isolated peaks. This behavior indicates that the observed performance gains are not the result of a narrowly tuned reward configuration, but instead reflect a structurally robust control policy that maintains superiority under diverse prioritizations of comfort, energy efficiency, and natural ventilation usage.

Furthermore, the absence of sharp sensitivity to any single weight confirms that the learned policy captures underlying physical and operational trade-offs of hybrid ventilation, rather than overfitting to a specific scalarization of the reward. As a result, the controller’s effectiveness is preserved even when stakeholder preferences shift between comfort-oriented, energy-driven, or ventilation-dominant objectives. This robustness is particularly important for real-world deployment, where exact reward weighting may vary across buildings, operators, or regulatory contexts.

3.2. Climate Resilience Assessment

The thermal resilience of the office under different weather conditions was assessed through IOD and CCOR, as presented in Figure 7. These indices quantify the ability of the RL control in maintaining indoor thermal comfort under increased outdoor warming. Lower IOD values reflect reduced indoor overheating in terms of frequency and magnitude, and higher CCOR values reflect greater resistance of the indoor temperature to the increasing outdoor heat levels. Although IOD is reported as a dimensionless metric due to normalization by occupied time, its magnitude directly reflects the average intensity and persistence of indoor overheating above the adaptive comfort threshold; therefore, reductions in IOD values correspond to fewer hours and lower severity of comfort exceedance during occupied periods. Although reinforcement learning training involves stochastic exploration, resilience metrics in this study are evaluated using a single converged policy deployed deterministically on fixed present-day and future weather files; thus, IOD and CCOR reflect deterministic performance of the learned controller under standardized climate scenarios rather than statistical uncertainty across multiple training realizations.

Table 2. Indoor Overheating Degree (IOD) comparison between rule-based control and RL under present and future climate scenarios.

Scenario	Controller	IOD	IOD Reduction (%)
TMY	RBC	0.21	–
	RL	0.09	57.14
2050-SSP1-2.6	RBC	0.26	–
	RL	0.2	23.08
2050-SSP5-8.5	RBC	0.27	–
	RL	0.2	25.93
2080-SSP1-2.6	RBC	0.35	–
	RL	0.21	40
2080-SSP5-8.5	RBC	0.37	–
	RL	0.25	32.43

and Figure 7a shows that the RL control substantially reduces indoor overheating across all weather conditions compared to RBC. Under TMY conditions, the IOD drops from 0.21 for RBC to 0.09 for RL, representing a 57% reduction in indoor overheating intensity. This illustrates that the RL agent effectively prevents indoor temperature exceeding comfort thresholds by adapting its actions to fluctuating outdoor conditions and maintaining comfort more efficiently. This performance advantage remains consistent in future climates, with IOD reductions from 0.26 to 0.2 and 0.27 to 0.2 in 2050, in SSP1-2.6 and SSP5-8.5, respectively, and from 0.35 to 0.21 and 0.37 to 0.25 in 2080. On average, RL control was able to reduce indoor overheating by an average of 35.66% relative to RBC, even as the overall ambient warmness degree increased due to climate warming. The improvement in IOD demonstrates the capacity of RL to learn adaptive ventilation and cooling patterns that mitigate overheating without excessive reliance on HVAC systems. By dynamically extending natural ventilation operation when outdoor conditions are favorable, and by activating AC during heat build-up phases, the RL policy achieves a more stable indoor temperature profile throughout the year. In contrast, the fixed-threshold approach of RBC delays response to rapid outdoor changes, leading to temperature rises and higher cumulative overheating hours. While the RL controller reduces IOD through a combination of extended natural ventilation operation and timely mechanical cooling, the present analysis focuses on the net resilience outcome rather than attributing overheating reductions to individual control actions. The concurrent increase in natural ventilation hours and reduction in HVAC energy consumption indicate that improved resilience is achieved primarily through enhanced exploitation of favorable outdoor conditions, with AC engaged selectively to suppress heat accumulation rather than through increased cooling intensity.

Figure 7b quantifies the resilience advantage through CCOR. The RL controller achieves a CCOR of 19.95, compared to 16.05 for RBC, representing a relative increase of 24.3% in the building’s resistance to outdoor warming. This higher CCOR value indicates that indoor overheating under RL rises more slowly with increasing outdoor heat stress, confirming a more resilient thermal behavior. The RL policy effectively isolates indoor comfort control performance from rising outdoor temperature profiles, demonstrating a vital feature for preserving acceptable indoor conditions during the more frequent and severe heat waves expected in future climates. These findings demonstrate the potential of data-driven adaptive control to support long-term building resilience planning and compliance with IEA EBC Annex 80 resilience objectives. From a sustainability perspective, the observed reductions in indoor overheating and improved resistivity to climate change indicate that RL-based hybrid ventilation can simultaneously enhance thermal resilience while limiting cooling demand, thereby mitigating both energy consumption and climate-induced performance degradation. The proposed framework complements standard local RL deployment by introducing a post-training resilience assessment layer, enabling designers to evaluate long-term controller robustness without requiring future data during training.

3.3. Explainability and Policy Interpretability

To unveil the black-box nature of the trained RL controller, SHAP analysis was applied to the final policy across all ventilation and cooling actions. Figure 8 presents the SHAP summary plots for three representative actions: NV On, AC 20 °C, and AC 22 °C, which are considered representative actions of the full operational spectrum from passive to active cooling. In each plot, the color gradient (red = high feature value; blue = low) denotes the magnitude of the outdoor or indoor variable, while the SHAP value quantifies its contribution to the probability of selecting that specific action. Figure 8 is interpreted as follows: each point represents a simulation timestep, where the horizontal axis (SHAP value) indicates the contribution of a given feature to increasing or decreasing the probability of selecting a specific control action. Positive SHAP values indicate that the feature promotes the action, while negative values suppress it. The color gradient represents the feature magnitude, enabling joint interpretation of both directionality and sensitivity.

The SHAP distribution for the NV On action, as shown in Figure 8, reveals a physically consistent and valid rationale. Indoor temperature and outdoor temperature show strongly negative SHAP values at high magnitudes, confirming that both elevated indoor and outdoor thermal conditions suppress natural ventilation activation to prevent overheating. Wind speed displays uniformly positive SHAP values, demonstrating that higher outdoor airflow promotes buoyancy- and wind-driven exchange, thus encouraging window opening. Indoor and outdoor temperature jointly suppress natural ventilation activation at high magnitudes, reflecting increased indoor heat load and reduced ventilation effectiveness under warm ambient conditions. This illustration reflects the physical mechanism of hybrid ventilation: NV is favored when the outdoor conditions can effectively remove internal heat without causing discomfort, and it is discouraged when outdoor air would increase indoor heat and radiant load. The minimal effect of indoor relative humidity reflects its narrow variation under the prevailing dry-summer Mediterranean conditions. Therefore, the NV On SHAP profile indicates that the RL agent learned control patterns consistent with expected physical reasoning used in typical building control operations.

The SHAP pattern for the AC 20 °C action, as shown in Figure 8, corresponds to the system’s most intensive cooling mode and demonstrates a perfectly aligned cause–effect relationship. Indoor temperature dominates, with large positive SHAP values, indicating that overheating directly increases the probability of selecting this low-setpoint cooling mode. Outdoor temperature also contributes positively, reflecting the RL agent’s recognition of persistent high ambient heat as a key trigger for intensive mechanical cooling. However, wind speed exhibits negative SHAP values, indicating that favorable outdoor airflow conditions suppress the activation of AC 20 °C by offering room for NV for convective heat removal. This feature influence confirms that the agent learned a thermal comfort logic: simultaneous indoor and outdoor heat accumulation, supported by high solar radiation, produces a strong incentive for active mechanical cooling, while adequate wind discourages it and supports room for NV instead. The pattern matches both physical laws and the reward design that penalized temperature deviations, showing that the agent’s learned behavior directly reflects its formulated objective function. The absence of any conflicting feature directions confirms the robustness and internal coherence of the policy.

Similarly, as shown in Figure 8, the SHAP plot for the AC 22 °C action represents a cooling mode that balances comfort with energy efficiency. The results display a smooth and monotonic sensitivity influence. The indoor temperature remains the principal positive driver, yet with a smaller magnitude than in AC 20 °C, reflecting proportionally reduced cooling intensity. Outdoor temperature retains a moderate positive impact, consistent with warm but not extreme ambient conditions, supporting the use of moderate cooling setpoints. Wind speed shows a mild negative relationship, highlighting the agent’s preference for natural ventilation when airflow is available. This SHAP profile demonstrates that the RL controller applies AC 22 °C as a regulatory action rather than an emergency response: it intervenes when temperatures rise above comfort thresholds, achieving stabilization without over-cooling. The uniformly ordered SHAP gradients across all features indicate a stable and generalizable policy capable of modulating its behavior between NV and intensive cooling regimes.

Across these three actions, the RL policy exhibits a continuous, physically grounded control logic that mirrors the principles of adaptive thermal comfort and building physics. The NV action’s feature relationships are the exact inverse of the AC actions, where NV is promoted by low indoor and outdoor temperatures and strong wind, while AC activation increases proportionally with heat load and decreases with the potential of wind ventilation. The consistent ranking of feature importance, dominated by indoor temperature, followed by outdoor temperature and wind-related variables, validates the correctness of the state representation and the proportional weighting of comfort and energy terms in the reward function. It is noteworthy that the SHAP results exhibit monotonic and directionally coherent feature influences that align with expected qualitative thermal comfort and ventilation principles, with no trends contradicting established physical intuition. Each feature showed a clear and monotonic relationship with the main drivers of heat exchange and thermal comfort. This confirms that the RL agent not only delivered effective thermal regulation but also developed a transparent and physically sound decision-making logic. Therefore, when trained within a physics-based environment, the data-driven RL framework successfully reproduced the same thermodynamic reasoning used by human experts in hybrid ventilation control, while remaining interpretable in the sense that dominant feature influences align with established physical reasoning for hybrid ventilation control. In this study, physical consistency is assessed qualitatively by verifying whether SHAP-attributed feature influences exhibit monotonic and directionally coherent relationships with known drivers of thermal comfort and ventilation performance, rather than through predefined quantitative thresholds.

In addition to action-specific interpretations, the SHAP beeswarm plots inherently provide a global feature importance ranking. Features in each beeswarm plot are ordered by their mean absolute SHAP value across all evaluated samples, reflecting their overall contribution to the controller’s decision-making. This ordering reveals that indoor temperature, outdoor temperature, and wind-related variables consistently dominate the policy logic across actions, while variables such as energy consumption and indoor relative humidity play secondary roles. SHAP analysis was performed exclusively during occupied periods, as actions during unoccupied hours are constrained by the control logic and do not contribute meaningful decision variability. This ensures that the extracted explanations reflect operationally relevant control behavior. Consistency across climate scenarios was evaluated by comparing SHAP feature rankings and distribution patterns obtained under present-day and future weather simulations. The stable ordering of dominant features and the preservation of monotonic influence trends across scenarios indicate that the learned control logic remains robust and physically consistent under changing climatic conditions.

The extracted feature importance hierarchy is inherently linked to the thermal characteristics and operating assumptions of the studied office. In buildings with higher thermal mass or larger internal heat gains, the relative influence of features such as indoor temperature, solar radiation, and energy consumption may shift, as thermal inertia and internal loads can delay or dampen the immediate response to outdoor drivers. Similarly, occupancy was modeled as a fixed binary schedule to reflect standard office operation; introducing stochastic or uncertain occupancy patterns could increase the relative importance of internal variables and modify action sensitivities. Nevertheless, the dominant features identified in this study reflect fundamental physical drivers of hybrid ventilation in lightweight Mediterranean offices and are expected to remain qualitatively consistent, even if their relative ranking varies under different building typologies or usage patterns. It is also noted that the SHAP analysis targets the instantaneous state–action mapping of the learned policy, consistent with the Markov formulation of the control problem. While this enables physically grounded interpretation of individual decisions, explaining longer-term sequential behavior or temporal dependencies represents a complementary extension beyond the scope of the present study.

4. Limitations and Future Work

This study evaluates climate-resilient reinforcement learning control of hybrid ventilation using a single-zone, lightweight Mediterranean office test cell. This controlled setup is appropriate for isolating control behavior, resilience trends, and explainability under climate stress, but it does not capture multi-zone airflow interactions, inter-zone thermal coupling, higher thermal mass, or stochastic occupant behavior. Consequently, the reported resilience improvements should be interpreted as context- and typology-specific, rather than universally transferable to all office buildings or climatic regions. The rule-based controller employed as a benchmark is intentionally simple and transparent, relying solely on outdoor temperature. This design choice enables clear attribution of adaptive behavior and resilience-related gains to the learning-based controller rather than to heuristic complexity or climate-specific tuning. However, it does not represent the full range of mixed-mode control strategies used in practice, and the observed performance gap should therefore be interpreted as demonstrating the advantage of reinforcement learning over conservative, non-adaptive baselines rather than over all possible engineered rule-based solutions. The reinforcement learning policy is trained and evaluated within a simulation-based digital twin using EnergyPlus, enabling repeatable, physics-consistent evaluation under standardized present-day and future weather scenarios. While this approach is suitable for assessing structural robustness to climate-induced thermal stress, it does not account for real-world uncertainties such as sensing noise, actuator delays, model mismatches, or occupant-driven disturbances. Experimental validation under operational conditions remains an important next step to assess deployment-level robustness. Finally, the reward function relies on fixed weighting factors reflecting a specific comfort–energy–ventilation preference consistent with prior studies. Although robustness to reward formulation is partially examined through post-training sensitivity analysis, adaptive or multi-objective reward formulations, broader building typologies, and data-efficient learning strategies such as transfer or offline reinforcement learning represent promising directions for future research.

5. Conclusions

This study developed and evaluated an explainable reinforcement learning-based control framework for hybrid ventilation in Mediterranean office buildings, with a particular focus on long-term climate resilience under projected future climates. The main quantitative findings and conclusions are summarized as follows:

• The proposed RL-based hybrid ventilation controller reduced annual HVAC energy consumption by 6.9–9.2% across present-day and future climate scenarios relative to rule-based control;
• Natural ventilation utilization during occupied hours increased consistently under RL control, with an improvement range of 7–8% across all evaluated climates;
• Indoor Overheating Degree (IOD) was substantially reduced, with reductions ranging from 23% to 57%, demonstrating enhanced resistance to climate-induced thermal stress;
• Climate resilience, quantified through the CCOR, improved by approximately 24%, indicating that the RL controller maintains a higher proportion of its thermal performance under future warming conditions compared to conventional control.

These improvements were achieved without retraining the controller under future climates, confirming the robustness and transferability of a policy trained on present-day TMY data. Although the energy savings achieved are moderate, their consistency across present-day and future climates reinforces the main contribution of this work: demonstrating a resilient and interpretable control strategy capable of sustaining performance under climate change rather than maximizing short-term energy reductions. While the proposed framework demonstrates strong quantitative performance and robustness, it remains subject to limitations related to simulation-based training, reward formulation choices, and single-zone system representation. These limitations pave the way for future research directions in experimental validation, scalability, and data-efficient learning.

From a sustainability perspective, the proposed framework enhances building system resilience to future climate conditions while conserving operational energy by prioritizing adaptive natural ventilation and limiting reliance on mechanical cooling. Overall, this work establishes a physics-informed, interpretable, and climate-resilient RL control approach that offers a scalable pathway toward more sustainable and climate-robust office buildings in Mediterranean regions.