Abstract
Traditional Multisensory Environments often use static protocols that lack real-time responsiveness to the user’s internal state. The proposed framework, called IMMERSI, addresses these limitations by developing an innovative Immersive-Proprioceptive Seat, a compact assistive device designed for personalized rehabilitation and well-being. The technological heart of this seat lies in its advanced customized sensor-integrationlayer, powered by proprietary low-power electronics. Developed in-house to ensure seamless synergy with the seat’s architecture, the hardware minimizes energy overhead. It features a non-invasive sensor system, using photoplethysmographic techniques and hypoallergenic biocompatible interfaces and monitoring physiological parameters such as heart rate, electrodermal activity, body temperature and blood oxygenation level. The energy-efficient hardware layer works in tandem with a computer vision system, enabling an integrated multi-parametric software analysis. By performing real-time data fusion across all physiological and vision-based inputs, the Immersive-Proprioceptive Seat is designed to enable a multidimensional assessment of the user’s emotional and cognitive state. Simultaneously, the seat delivers a high-fidelity multisensory experience across five sensory modalities, most notably introducing proprioceptive stimulation through a system of inflatable cushions and vibrational actuators. This allows for active and passive body perception enhancement, which is often missing in current market solutions. A key factor and contribution of the IMMERSI platform is the Clinical Decision Support System driven by Artificial Intelligence. By analyzing complex time-series data in real time, the AI dynamically adapts stimulation scenarios to the user’s psychophysical needs. The platform implements Explainable Artificial Intelligence methodologies, including LIME, to ensure transparency and interpretability of the AI’s decisions. The Explainable Artificial Intelligence-driven approach is essential for building trust among clinicians and caregivers, facilitating the adoption of the Immersive-Proprioceptive Seat in therapeutic contexts such as autism spectrum disorders and elderly care. In a preliminary evaluation on the public WESAD benchmark, the classifier reaches 95.4% accuracy and an AUC-ROC of 0.989 in leave-one-subject-out validation, with an end-to-end decision latency of about 70 ms. These results support the technical feasibility of the IMMERSI infrastructure as a scalable and ethically designed candidate for the next generation of immersive health technologies, pending clinical validation in real-world settings.
1. Introduction
As worldwide populations grow older, healthcare and social care professionals face an ever-increasing demand for services that keep older adults independent, emotionally stable and satisfied with their daily lives [1,2]. Within this group, stress, anxiety and agitation are common and clinically significant phenomena, affecting, in particular, people with dementia, whose behavioral and psychological symptoms are a source of distress for both family members and healthcare professionals.
A non-pharmacological response to this need is the Multisensory Environment (MSE), derived from Snoezelen, i.e., a room or installation providing combined stimuli of light, sound, touch and smell to promote calm, self-regulation and engagement; a considerable body of clinical research literature links such installations to reduced agitation and improved mood in people with dementia [3,4,5], despite the methodological heterogeneity of this evidence and the recognized need for a more standardized and outcomes-based evaluation. Comparable installations serve rehabilitation and neurodevelopmental purposes more broadly, for instance with autistic users [6,7]. However, conventional MSE devices all share a fixed pre-set stimulation sequence: they are bulky and expensive to install, and are unable to sense how a person is feeling at any given moment and adapt accordingly. This limits their clinical impact and, in particular, means that proprioception—the perception of one’s own body’s position and movement—remains largely overlooked by existing commercial offerings. Meanwhile, a separate line of research in the Ambient Assisted Living (AAL) domain has developed a comprehensive toolkit comprising discrete sensors, smart home infrastructure and assistive devices for older people [8,9], providing much of the technology that a sensitive and adaptive successor to static MSE devices would require.
Automatic stress and affect recognition from wearable physiological signals, in particular, photoplethysmography (PPG) and electrodermal activity (EDA), have been studied extensively; furthermore, public benchmarks such as WESAD [10] are widely used to evaluate stress-detection pipelines. A recurring challenge is the limited labeled data available per subject, combined with strong inter-individual variability. Siamese neural networks [11,12], originally proposed for verification tasks, address this by learning a discriminative embedding space through pairwise or triplet comparisons rather than direct per-subject classification, and are well suited to physiological classification under data scarcity; their generalization is typically assessed with a leave-one-subject-out (LOSO) cross-validation protocol, which conservatively estimates performance on unseen users.
A second key concern is the adoption of AI-based Clinical Decision Support Systems (CDSSs) in sensitive healthcare settings: advanced deep-learning classifiers are often opaque, limiting clinician and caregiver trust. Explainable Artificial Intelligence (XAI) techniques such as LIME [13] and SHAP [14] complement black-box predictions with local feature-level explanations, increasingly used in healthcare to support this trust-building process. Complementary to contact-based sensing, contactless affective computing approaches exploit facial landmarks and Action Units from the Facial Action Coding System (FACS) [15], extracted in real time even on embedded hardware [16], mapped onto interpretable representations of emotions such as Russell’s circumplex model [17], and further strengthened by RGB-thermal fusion exploiting correlations with the autonomic nervous system visible in facial thermal maps [18].
Based on this concept, the IMMERSI project (“Immersive Multisensory Module and Experience for Relax and Self-Improvement”) is developing the Immersive Proprioceptive Seat (IPS), a standalone device that continuously monitors the user’s physiological and emotional states and adjusts a personalized multisensory stimulation program accordingly, promoting relaxation and well-being in frailty and autism spectrum disorder care settings. XAI technology is integrated into every step of the process, enabling medical and healthcare staff to understand and interpret the system’s recommendations.
This paper describes the IMMERSI architecture, the IPS sensing/actuation layers, the Siamese classifier estimating physiological state from PPG/EDA, the complementary contactless RGB-thermal affective module, and the explainability layer feeding a CDSS. Classification performance and explainability insights obtained on WESAD under a LOSO protocol are reported and discussed.
2. Materials and Methods
2.1. System Architecture: Sensing and Actuation Layers
The IPS is built around two Raspberry Pi 5 single-board computers that jointly orchestrate a sensing layer and an actuation layer (Figure 1). A dedicated vision node runs the contactless RGB-thermal affective-analysis pipeline (Section 2.4) close to the cameras, while a separate CDSS node hosts the Siamese-CATSA physiological classifier, the XAI module and the clinical dashboard (Section 2.5 and Section 2.6); the two boards exchange the affective vector and status information via a lightweight HTTP REST endpoint, decoupling the vision workload from the classification/CDSS workload and improving fault isolation. The sensing layer comprises (i) a proprietary, low-power, non-invasive physiological subsystem, based on biocompatible and hypoallergenic interfaces, acquiring heart rate/HRV via PPG, EDA, body temperature and SpO2 and (ii) a vision subsystem combining a high-resolution RGB camera with a long-wave infrared (LWIR) thermal camera for facial thermal mapping. The actuation layer provides a multisensory experience through five sensory modalities: video/lighting, audio, olfactory, haptic and proprioceptive actuation (inflatable cushions and vibrational motors for active/passive body-perception enhancement).
Figure 1.
Sensing and actuation layers of the IPS.
2.2. Signal Acquisition and Preprocessing
The current implementation of the classification pipeline uses the PPG/BVP and EDA channels of the WESAD dataset [10]. Since these signals are characterized by different sampling frequencies, they were resampled to 50 Hz using linear interpolation to match the sampling frequency of the signals acquired in real time by the IPS and ensure consistency between the training and deployment pipelines. A thread-safe circular buffer retains the last 5 s of signal and produces non-overlapping windows of 250 samples (with a stride equal to the window length). Each window is independently normalized per channel via z-score standardization (mean subtraction and division by the standard deviation, with a small regularization term for numerical stability), which makes the model robust to inter-subject and inter-session scale and offset differences. For the EDA channel, a detrending step isolates the phasic component supporting the identification of skin-conductance responses (SCR). Labels are reduced to the two classes of interest (baseline, stress), discarding irrelevant segments.
The normalized windows feed the classifier directly; alongside this, 15 hand-crafted features covering both the time and frequency domain are computed from the same signals: heart-rate-variability indicators such as HRV_std, PPG amplitude/quantile statistics (e.g., PPG_peak2peak, PPG_Q25/Q75/mean) and EDA-derived markers of sympathetic activation (SCR count, amplitude, slope). This descriptor set is shared downstream by the clinical rule engine and by the XAI module described in Section 2.5.
2.3. Siamese-CATSA Classifier for Physiological State Estimation
Classification of the preprocessed 5 s windows (250 samples at 50 Hz) is performed by a Siamese-style network with two structurally identical but independently parameterized convolutional branches, one per physiological channel (PPG and EDA, Figure 2). Each branch stacks five -regularized () Conv1D layers of increasing width—64 filters (kernel size 7), 128 (kernel size 5), 128 (kernel size 5), 256 (kernel size 3) and 256 (kernel size 3)—each followed by batch normalization and ReLU activation, interleaved with two MaxPooling1D (pool size 2) layers and Dropout at rates , , and across the stack; a GlobalAveragePooling1D layer then condenses the resulting feature maps, and a final Dense(128, ReLU, ) layer produces the 128-dimensional branch embedding. The PPG and EDA embeddings are concatenated (late fusion, dimensions; Figure 3) and passed through a classification head of three fully connected layers of decreasing width (256, 128 and 64 units, all -regularized, with batch normalization on the first two and dropout rates of 0.4, 0.3 and 0.2, respectively), followed by a 2-way Softmax layer that separates the baseline from stress at the decision threshold of 0.40; Table 1 summarizes the resulting layer-by-layer configuration. Training uses the Adam optimizer [19] (learning rate ) with sparse categorical cross-entropy loss and class weights compensating for residual class imbalance (stress class weighted ), for up to 150 epochs (batch size 64) with early stopping (patience 20 on validation accuracy) and a learning rate plateau scheduler (factor 0.5, patience 10, min. learning rate ), in line with the Siamese paradigm [11,12] that is particularly appropriate given the limited availability of subjects. For deployment, the trained Keras model is exported to TensorFlow Lite with post-training INT8 quantization and runs on the CDSS Raspberry Pi 5 node with an inference time per window of less than 200 ms.
Figure 2.
PPG/EDA branches of the Siamese-CATSA network (simplified schematic): each branch applies a stack of Conv1D feature-extraction layers with pooling before the embeddings are concatenated (Figure 3) and classified.
Figure 3.
Late fusion and state classification: the PPG and EDA embeddings are concatenated and passed through a classification head of three Dense layers before the Softmax layer, which separates Baseline from Stress using the decision threshold.
Table 1.
Layer-by-layer configuration of the Siamese-CATSA network: PPG/EDA branches (feature extraction) and classification head (late fusion).
2.4. Contactless Affective Analysis from RGB-Thermal Vision
As a non-invasive complement to the physiological branch, a contactless affective-analysis module processes the visible-spectrum (RGB) video stream. Video is captured with a low-light (zero-lux) camera equipped with a software-switchable infrared filter, at 640×480 px and 30 fps, and processed on the Raspberry Pi 5 vision node described in Section 2.1 by a multi-threaded Python application. A lightweight quantized face-mesh model optimized for ARM processors [16] extracts 478 three-dimensional facial landmarks in real time; subsets of these landmarks are used to estimate facial-muscle activations mapped onto Action Units (AUs) of the Facial Action Coding System [15], producing an expression vector .
The expression vector feeds an interpretable classification of the affective state according to Russell’s circumplex model [17], expressed in terms of Valence and Arousal (both in ). In line with the principles of transparency adopted throughout the CDSS, this module deliberately avoids end-to-end black-box neural layers and instead uses explicit fuzzy linguistic rules based on the activations of the tracked landmarks, so that the physical cues underlying each decision remain verifiable. Preliminary validation showed robust landmark tracking, with a success rate exceeding 95%, even under low-illumination conditions. All processing is performed locally, with no persistence of raw video frames, in compliance with GDPR.
The RGB branch is complemented by an LWIR thermal camera monitoring temperature variations in facial regions of interest (nose tip, eyes, forehead) correlated with autonomic activity. The two modalities are combined using a gated late-fusion architecture [18] (Figure 4): FACS action units (RGB, 640 × 480 px, 30 fps) and thermal descriptors for each ROI (160×120 px, 9 fps) are aggregated into a common window and combined via a learned gate weighting each modality prior to classification. The resulting RGB-thermal affective vector is sent from the vision node to the CDSS node via the HTTP REST endpoint described in Section 2.1, where it is fused with the physiological estimate inside the CDSS.
Figure 4.
Gated late-fusion RGB-thermal architecture combining FACS action units and per-ROI thermal descriptors for contactless affective estimation.
Face and landmark detection, FACS AU extraction, thermal ROI statistics and window aggregation (mean, standard deviation, slope) are all pre-trained or purely algorithmic upstream steps and contribute no trainable parameters; only the gated-fusion stage downstream of aggregation is learned. After window aggregation, the RGB and thermal branches produce compact feature vectors of dimensions and , respectively, concatenated into a vector of dimension . A scalar gate (a single Dense(1, sigmoid) layer, ) rescales the two modalities before a compact MLP head (Dense(32, ReLU) → Dense(16, ReLU) → Dense(2, Softmax)) performs the Baseline/Stress decision, as reported in Table 2, for a total of about 2200–2300 trainable parameters. This parameter number is deliberately kept three to four orders of magnitude below full-image CNN architectures (e.g., dual ResNet-50 with cross-attention and a temporal transformer, ∼87 M parameters), which are only justified by very large training sets; with the tens of subjects available here, keeping the trainable part in the few thousands of parameters, well below the number of independent subjects, is essential to control overfitting in the LOSO validation. MLP dropout (0.3–0.5) and a modality-dropout rate (0.15–0.20, randomly zeroing one modality during training) are used for regularization, together with the Adam [19]/AdamW [20] optimizer.
Table 2.
Trainable-parameter budget of the gated RGB-thermal late-fusion network (scalar-gate variant).
2.5. Explainable AI Module
Combined with the Siamese-CATSA classifier, an Explainable AI module based on LIME [13] generates, for each decision, a local model-agnostic explanation of the classifier output, rather than exposing the network’s internal embeddings. LIME builds a local surrogate by perturbing the 15 hand-crafted physiological descriptors of Section 2.2 (HRV-derived parameters from PPG and sympathetic-activation indices such as the number and amplitude of SCRs from EDA) in a neighborhood of the current window, querying the Siamese-CATSA classifier on the perturbed samples and fitting a sparse linear model whose coefficients approximate the local decision boundary; the resulting coefficients quantify the contribution of each feature and are ranked to obtain the feature importance vector. An example is reported in Figure 5, identifying the heart-rate variability standard deviation (HRV_std) as, by a wide margin, the single most influential feature for the Baseline/Stress decision, outperforming amplitude-based PPG descriptors and quartiles. The perturbation background is computed once over the entire training set and cached on the CDSS node, so that explanations are representative, stable and interpretable and so that the LIME call for each decision need only perturb and recalculate samples around the current window.
Figure 5.
Example of LIME feature-importance ranking for the Baseline/Stress decision: HRV_std dominates the explanation, well ahead of the PPG amplitude- and quartile-based descriptors. Green highlights the dominant feature (HRV_std); red bars indicate secondary features with markedly lower importance.
The feature importance vector produced by LIME is used by an explanation layer that transforms numerical contributions into the clinical narrative described in Section 2.6. This layer operates on a deterministic, rule-based template backend, which maps the dominant features and their sign to predefined clinical phrasings (status, rationale, critical-parameter alerts and actuator rationale) with negligible latency (<1 ms) and full operational stability on the edge device. This keeps LIME’s interpretability decoupled from the natural language generation phase, so that the formulation of an explanation never affects the features that determined the decision.
2.6. Clinical Decision Support System and Software Platform
The LIME explanations feed into a CDSS that translates the model’s results into human-readable clinical descriptions: current status and diagnostic confidence, a rationale describing each parameter’s contribution, critical-parameter alerts (e.g., critically low heart-rate variability) and a recommendation regarding the actuator along with the corresponding rationale (e.g., rhythmic music and moderate-frequency vibrations to promote activation, or soft lighting and no active stimulation to promote rest). A template-based module translates the classification output and LIME explanations into these clinically interpretable messages, together with operational indications for the IPS actuators.
The software platform is implemented with the Django (Python) framework and integrates real-time data acquisition through a physiological simulator in the current development stage, a multi-patient inference engine capable of managing simultaneous sessions, a relational database for the persistence of decisions and their history, and a clinical dashboard visualizing the current status, vital signs, XAI explanation and stimulus recommendation.
Each CDSS decision is packaged as a JSON payload and pushed over MQTT (QoS 1) to the topic [ROOT]/[NODE_ID]/command, in accordance with the project’s technical specification. The message header contains a message identifier (MsgId), a timestamp (TS) and a list of commands, whose payload depends on the target actuator. The MQTT publisher is fail-tolerant, with asynchronous reconnection and gradual degradation to logging-only operation in case of connectivity loss.
The AI pipeline was implemented in Python 3.11, using TensorFlow 2.15/TensorFlow Lite 2.15 for model training and embedded inference, NumPy 1.26 and Pandas 2.1 for signal processing, Scikit-learn 1.3 for evaluation metrics, and lime 0.2.0.1 for explainability; the CDSS software (ver. 1.0) platform is built on Django 4.2 (LTS).
3. Results and Discussion
The Siamese-CATSA network was preliminarily evaluated on the WESAD dataset [10] using LOSO cross-validation (LOSO-CV): each subject is excluded in turn from training and used as test data, providing a conservative estimate of the model’s ability to generalize to unseen users, an essential requirement for the IPS’s intended clinical use. Table 3 reports the resulting classification metrics together with the corresponding deployment notes. The decision threshold on the Softmax output was set at 0.40 to maximize balanced accuracy and sensitivity to acute stress episodes.
Table 3.
Classification performance of the Siamese-CATSA model on WESAD (LOSO-CV).
The near-unity AUC-ROC reported in Table 3 corresponds to a receiver operating characteristic (ROC) curve lying close to the upper-left corner across the entire range of operating thresholds (Figure 6), indicating a robust separation between the baseline and stress classes that is largely independent of the specific decision threshold, consistent with the high weighted F1-score.
Figure 6.
ROC curve of the Siamese-CATSA stress classifier on WESAD.
In addition to classification accuracy, the LIME-based explanation module described in Section 2.5 was evaluated for both interpretability and runtime cost. Consistent with the feature importance ranking of Figure 5, the heart-rate variability standard deviation (HRV_std) emerges as the single most influential feature for the Baseline/Stress decision, outperforming the amplitude- and quartile-based PPG descriptors and confirming the well-established link between reduced heart-rate variability and sympathetic (stress-related) activation; this gives clinicians a physiologically grounded explanation rather than an unclear probability score. Generated alongside the CDSS inference, the end-to-end LIME explanation pipeline adds approximately 67 ms to the decision process, for a total closed-loop CDSS latency of approximately 70 ms, compatible with real-time therapeutic use. Figure 7 illustrates an example of a reference decision taken from the clinical dashboard. Specifically, it reports a clinical description, a confidence score, the corresponding vital signs and a recommendation regarding the actuators and their status. Figure 8 shows an illustrative example of a stress state with a 100% confidence level, with a recommendation for light vibrational stimulation (40–60 Hz).
Figure 7.
Clinical Decision Support System (CDSS) dashboard, showing the latest decision narrative, vital signs and actuators panel.
Figure 8.
Example of CDSS decision detail for the illustrative stress-state example.
Preliminary validation of the RGB-based affect analysis module showed a success rate of over 95% in tracking facial landmarks, including in low-light conditions, supporting its use as a non-invasive complement to the physiological classifier described above. The gated RGB-thermal fusion stage (Figure 4) combines this visible-spectrum estimate with thermal descriptors for each ROI, providing a unified contactless affective vector for the CDSS.
The results indicate that a Siamese-CATSA classifier operating on PPG and EDA signals, combined with a LIME-based explainability layer, supports real-time, interpretable stress classification with high accuracy (95.4%) and discrimination ability (AUC-ROC 0.989) under a LOSO protocol on WESAD, with a total decision latency compatible with closed-loop therapeutic feedback (∼70 ms).
The complementary RGB-thermal affective vision module extends the assessment to a fully contactless modality based on FACS action units, Russell’s Valence/Arousal circumplex model and thermal descriptors for each ROI, without introducing an additional black-box classifier; the gated fusion adaptively weights the two modalities, which is expected to improve robustness over a single-modality estimate (still to be verified on a labeled benchmark), and is particularly relevant for users, such as children with autism spectrum disorders, for whom skin-contact sensors may be poorly tolerated.
Translating model outputs into narratives, critical-parameter alerts and actuator rationales is intended to address the trust barrier limiting the use of deep-learning systems in sensitive therapeutic contexts.
4. Conclusions
The IMMERSI platform provides a proof of concept of a fully integrated, real-time, closed-loop system that couples a proprietary low-power multisensor seat with an explainable AI-driven CDSS, designed to enable personalized multisensory therapy. Its main contributions are (i) a Siamese-CATSA classifier for PPG/EDA-based stress estimation, preliminarily validated on WESAD with LOSO-CV (95.4% accuracy, AUC-ROC 0.989, weighted F1-score 0.953) and deployed as a TensorFlow Lite model on Raspberry Pi 5, (ii) a complementary, interpretable contactless affective vision module based on FACS action units and Russell’s circumplex model, (iii) an XAI layer (LIME) aimed at fostering clinician trust by making every recommendation explainable, (iv) a gated late-fusion RGB-thermal architecture combining FACS action units with thermal descriptors for each ROI into a unified contactless affective estimate, and (v) a total CDSS latency of approximately 70 ms, together with a JSON/MQTT actuation protocol. The resulting architecture is scalable and potentially suitable for elderly care and wider neurorehabilitation contexts, although its clinical efficacy remains to be established.
Some limitations remain and are worth discussing openly. Because the reported performance was obtained on WESAD, a benchmark collected from healthy volunteers rather than from frail and older users, some degree of domain shift toward the target clinical population is to be expected once the system moves out of the laboratory. Similarly, the physiological descriptors used by LIME are not independent of one another, and this correlation can make individual attributions somewhat unstable across otherwise similar decisions, which motivates a direct comparison with SHAP [14] and Integrated Gradients [21] on the same feature set. The contactless RGB-thermal module, for its part, has so far only been characterized in terms of landmark-tracking robustness (above 95%): although its gated-fusion network was deliberately kept to a few thousand trainable parameters to control overfitting, it has not yet been evaluated on a labeled classification benchmark of its own. Finally, every result discussed here was obtained on bench hardware under controlled conditions, and no clinical trial with the intended user population has been carried out at this stage. Moreover, neither the safety and tolerability of the multisensory and proprioceptive stimulation nor its effect on actual rehabilitation and well-being outcomes has been assessed so far.
These open points naturally shape the directions for future work. Priority will be given to per-subject baseline calibration and to a preliminary clinical validation protocol with older adults, so as to narrow the gap between WESAD-based performance and real-world use; in parallel, the RGB-thermal module will be evaluated on a labeled accuracy benchmark, both on its own and fused with the physiological branch inside the CDSS. On the explainability side, LIME will be benchmarked against SHAP and Integrated Gradients, and we plan to explore whether a compact on-device LLM can be integrated alongside the template backend to produce more naturalistic clinical narratives, provided its inference latency can be brought within the real-time constraints of the closed-loop CDSS. Finally, the robustness of the HTTP communication between the vision and CDSS nodes will be stress-tested under realistic network conditions, and the classifier will be integrated with the real IPS sensing hardware ahead of field deployment.
Beyond technical validation, future research will prioritize (i) validation on independent and more diverse cohorts (e.g., older adults, people with dementia, autistic children and adults), to assess generalizability across populations and the reliability of emotion recognition beyond laboratory-induced stress, (ii) controlled clinical studies, ideally comparing the closed-loop personalized stimulation with conventional static MSE sessions, to quantify its effect on clinically relevant outcomes such as agitation, anxiety, mood and engagement, and (iii) systematic, long-term safety assessments of the multisensory and proprioceptive stimulation, including tolerability thresholds, adverse-event monitoring and safety limits enforced at the actuator level (e.g., maximum vibration intensity, cushion pressure and exposure duration), conducted under appropriate ethical approval.
Author Contributions
Conceptualization, A.L., A.C., G.R. and A.M.; methodology, A.C., G.R. and A.M.; software, A.C. and A.M.; validation, A.C., G.R. and A.L.; writing—original draft preparation, A.L., A.C., G.R., M.D.P. and A.M.; writing—review and editing, A.L., A.C., G.R., M.D.P., P.A.S., A.D.R., L.D.G., C.L., L.D. P.L. and A.M.; supervision, A.L.; project administration, A.L.; funding acquisition, A.L. All authors have read and agreed to the published version of the manuscript.
Funding
This research was funded by Apulia Region within the project “IMMERSI—Immersive Multisensory Module and Experience for Relax and Self-Improvement” (grant number 0JOQJF4, PR Puglia FESR 2021–2027, Azione 1.5 “Avviso RETI”). The views and opinions expressed are only those of the authors and do not necessarily reflect those of the funder.
Institutional Review Board Statement
Not applicable. This study did not involve new data collection from human participants; the AI models were trained and evaluated on the publicly available de-identified WESAD dataset [10].
Informed Consent Statement
Not applicable.
Data Availability Statement
The WESAD dataset used for model training and evaluation is publicly available [10]. Restrictions apply to the availability of the proprietary firmware and CDSS source code, which were developed as part of the IMMERSI project and are not publicly available.
Acknowledgments
During the preparation of this manuscript, the authors used Claude Opus 4.8 (Anthropic) for language editing and text revision. The authors have reviewed and edited the output and take full responsibility for the content of this publication.
Conflicts of Interest
Alessandro De Robertis was employed by Cooperativa Sociale Genteco. Luca De Guglielmo was employed by Exeling S.r.l. Claudio Lepore was employed by Cream S.r.l.s. Leonardo D’Alessandro and Pierfrancesco Losito were employed by Ingel S.A.S. The remaining authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
References
- World Health Organization. World Report on Ageing and Health; WHO: Geneva, Switzerland, 2015. [Google Scholar]
- European Commission. The 2021 Ageing Report: Economic and Budgetary Projections for the EU Member States (2019–2070); Institutional Paper 148; European Commission: Brussels, Belgium, 2021. [Google Scholar]
- Chung, J.C.C.; Lai, C.K.Y.; Chung, P.M.B.; French, H.P. Snoezelen for dementia. Cochrane Database Syst. Rev. 2002, 4, CD003152. [Google Scholar] [CrossRef] [Scilit]
- Staal, J.A.; Sacks, A.; Matheis, R.; Collier, L.; Calia, T.; Hanif, H.; Kofman, E.S. The effects of Snoezelen (multi-sensory behavior therapy) and psychiatric care on agitation, apathy, and activities of daily living in dementia patients on a short term geriatric psychiatric inpatient unit. Int. J. Geriatr. Psychiatry 2007, 37, 357–370. [Google Scholar] [CrossRef] [Scilit]
- Sánchez, A.; Millán-Calenti, J.C.; Lorenzo-López, L.; Maseda, A. Multisensory stimulation for people with dementia: A review. Am. J. Alzheimers Dis. Other Demen. 2013, 28, 7–14. [Google Scholar] [CrossRef] [Scilit]
- Stephenson, J.; Carter, M. The use of multisensory environments with individuals with developmental disabilities: A systematic review. J. Dev. Phys. Disabil. 2024, 37, 731–757. [Google Scholar] [CrossRef] [Scilit]
- Leonardi, S.; Di Cara, M.; Giliberto, S.; Piccolo, A.; De Domenico, C.; Leonardi, G.; Alito, A.; Siracusano, R.; Calabrò, R.S.; Quartarone, A.; et al. The use of multisensory environments in children and adults with autism spectrum disorder: A systematic review. Autism 2025, 29, 1921–1938. [Google Scholar] [CrossRef] [Scilit]
- Rashidi, P.; Mihailidis, A. A survey on ambient-assisted living tools for older adults. IEEE J. Biomed. Health Inform. 2013, 17, 579–590. [Google Scholar] [CrossRef] [Scilit]
- Cicirelli, G.; Marani, R.; Petitti, A.; Milella, A.; D’Orazio, T. Ambient assisted living: A review of technologies, methodologies and future perspectives for healthy aging of population. Sensors 2021, 21, 3549. [Google Scholar] [CrossRef] [Scilit]
- Schmidt, P.; Reiss, A.; Duerichen, R.; Marberger, C.; Van Laerhoven, K. Introducing WESAD, a Multimodal Dataset for Wearable Stress and Affect Detection. In Proceedings of the 20th ACM International Conference on Multimodal Interaction (ICMI 2018), Boulder, CO, USA, 16–20 October 2018; pp. 400–408. [Google Scholar]
- Bromley, J.; Guyon, I.; LeCun, Y.; Säckinger, E.; Shah, R. Signature Verification Using a “Siamese” Time Delay Neural Network. In Proceedings of the 6th International Conference on Neural Information Processing Systems (NeurIPS 1993), Denver, CO, USA, 29 November–2 December 1993; pp. 737–744. [Google Scholar]
- Chicco, D. Siamese Neural Networks: An Overview. In Artificial Neural Networks; Methods in Molecular Biology; Springer: New York, NY, USA, 2021; Volume 2190, pp. 73–94. [Google Scholar]
- Ribeiro, M.T.; Singh, S.; Guestrin, C. “Why Should I Trust You?”: Explaining the Predictions of Any Classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD 2016), San Francisco, CA, USA, 13–17 August 2016; pp. 1135–1144. [Google Scholar]
- Lundberg, S.M.; Lee, S.I. A Unified Approach to Interpreting Model Predictions. In Proceedings of the 31st International Conference on Neural Information Processing Systems (NeurIPS 2017), Long Beach, CA, USA, 4–9 December 2017; pp. 4768–4777. [Google Scholar]
- Ekman, P.; Friesen, W.V. Facial Action Coding System (FACS): A Technique for the Measurement of Facial Movement; Consulting Psychologists Press: Palo Alto, CA, USA, 1978. [Google Scholar]
- Kartynnik, Y.; Ablavatski, A.; Grishchenko, I.; Grundmann, M. Real-time facial surface geometry from monocular video on mobile GPUs. In Proceedings of the CVPR Workshops (CVPRW 2019), Long Beach, CA, USA, 16–20 June 2019. [Google Scholar]
- Russell, J.A. A circumplex model of affect. J. Pers. Soc. Psychol. 1980, 39, 1161–1178. [Google Scholar] [CrossRef] [Scilit]
- El Othmani, O.; Naouali, S. Cross-spectral fusion of thermal and RGB imaging for objective pain estimation. PLoS Digit. Health 2026, 5, e0001424. [Google Scholar] [CrossRef] [Scilit]
- Kingma, D.P.; Ba, J. Adam: A Method for Stochastic Optimization. In Proceedings of the 3rd International Conference on Learning Representations (ICLR 2015), San Diego, CA, USA, 7–9 May 2015. [Google Scholar]
- Loshchilov, I.; Hutter, F. Decoupled Weight Decay Regularization. In Proceedings of the 7th International Conference on Learning Representations (ICLR 2019), New Orleans, LA, USA, 6–9 May 2019. [Google Scholar]
- Sundararajan, M.; Taly, A.; Yan, Q. Axiomatic attribution for deep networks. In International Conference on Machine Learning; PMLR: New York, NY, USA, 2017; Volume 70, pp. 3319–3328. [Google Scholar]
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content. |
© 2026 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license.







