Review Reports - Multimodal Healthcare in the Metaverse: Integrating Multimedia Information Retrieval, Physiological Sensing, and Markerless Kinematic Analytics Within the CareSync Framework

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

The empirical dataset is incorrectly described. Section 5.1 states that 27 participants completed three experimental conditions, producing 81 independent sessions. The cited rPPG-10 dataset contains 26 healthy university students. Subject 4 was excluded, and each participant contributed one 10-minute recording divided into three facial regions of interest.

These regions are not independent sessions or experimental conditions. Table 1 contains only 26 subjects, which confirms this discrepancy. Rewrite the dataset description, define the actual unit of analysis, and treat multiple regions or windows from the same participant as repeated observations. Participant-level separation is required in any statistical analysis or parameter tuning.

The respiratory-rate validation is unsupported. The cited dataset documents synchronized ECG but no respiratory-belt recordings. Table 1 repeats exactly 15.0 RPM for every subject, reports no predicted respiratory rate, and provides no respiratory error despite defining MAE. Remove all RR validation claims unless the authors can provide traceable respiratory-reference files, individual predictions, synchronization details, and complete error results.

The principal heart-rate conclusion is contradicted by Table 1. Recalculation from the 26 displayed rows gives a raw-POS MAE of approximately 13.746 BPM and a filtered-POS MAE of approximately 13.788 BPM. Thus, filtering slightly worsens MAE rather than “breaking the 13.75 BPM barrier.” The filtered method also has only 9/26 results within 5%, compared with 12/26 for raw POS, and introduces a strong negative mean bias of approximately -12.83 BPM. The claims of “marked” and “statistically significant” improvement must be removed or supported by corrected data and a prespecified paired statistical analysis. Report MAE, median absolute error, RMSE, bias, confidence intervals, Bland–Altman limits of agreement, and participant-level results. The term “Zero-Bias” is misleading and should not be used unless zero bias is demonstrated.

The three research questions are not answered by the experiments. RQ1 requires validation of physiological, emotion, and movement fusion, but only heart-rate estimates are partially assessed. RQ2 requires a comparison between simulated and real markerless tracking, but no such baseline or accuracy experiment is presented. RQ3 requires evidence of fast and scalable retrieval, yet no retrieval relevance, latency, throughput, memory, database-size, or concurrency results are reported. Either redesign the evaluation around all three research questions or narrow the title, abstract, questions, and conclusions to the component that was actually tested.

Emotion, pain, stress, movement safety, joint angles, repetition counting, synchronization, graph retrieval, and 3D replay are presented without validation. Provide ground truth and appropriate metrics for each claimed capability. Kinematics should be compared with a reference system or clinical goniometry. Emotion, pain, and stress require validated labels or instruments and participant-level evaluation. Retrieval requires a relevance benchmark, baseline methods, top-k ranking metrics, and performance tests at increasing collection sizes. Synchronization accuracy should be measured rather than assumed.

The physiological methodology is not reproducible. Specify the exact POS implementation, ROI selection, sampling and missing-frame handling, filter type and order, window length and overlap, detrending parameters, autocorrelation peak-selection rule, signal-quality formula, rejection threshold, and parameter-tuning procedure. Explain how 30-second windows were selected from each 10-minute recording. The dataset supplies 64×64 ROI videos rather than ordinary full-face recordings, so the statement that MediaPipe Face Landmarker detected forehead and cheek regions requires clarification. RMSSD-based stress estimation also requires reliable beat-to-beat intervals; a window-level heart-rate estimate from autocorrelation is insufficient.

The “human-centered,” “clinical,” and “personalized decision-support” claims exceed the evidence. The evaluation uses healthy young students outside rehabilitation and does not include patients, therapists, clinicians, an XR intervention, usability testing, clinical outcomes, or prospective deployment. Reframe the system as an unvalidated technical prototype until clinical and human-factors studies are completed.

The graph-similarity method needs justification and validation. The three-second co-occurrence window and state-discretization thresholds are arbitrary. An undirected co-occurrence graph discards event order and detailed timing, contradicting the claim that temporal relationships are preserved. Raw edge counts will also be confounded by session duration and event frequency. Define normalization, missing-state handling, zero-denominator behavior, and all discretization rules; compare the weighted Jaccard approach with suitable baselines and evaluate whether its explanations are meaningful to clinicians.

The claim that complete body posture can be reconstructed from stored joint angles is technically incomplete. Joint angles alone do not determine global position, root orientation, segment lengths, or a unique 3D pose. Specify the stored landmarks and calibration data, interpolation and occlusion handling, coordinate systems, and reconstruction errors. Mouse or touch input should not be described as an alternative wrist-measurement modality because it does not provide equivalent kinematic information.

The privacy discussion is inadequate. Derived facial blendshapes, body trajectories, physiological measurements, and session events remain sensitive and potentially identifying even when raw video stays on the device. Add an ethics and data-governance section covering consent, approval for secondary data use, licensing, pseudonymization, encryption, access control, retention, deletion, re-identification risk, and demographic or skin-tone performance bias.

Table 1 must be corrected and completed. It is labelled as a session-level HR and RR evaluation but contains one row per available participant, omits predicted RR, and does not show 81 sessions. Figure 4 should be replaced by an identity scatterplot and Bland-Altman plot, connecting participants with lines creates a meaningless sequence. Figures 2 and 3 need clear units, readable labels, data provenance, and an explicit statement indicating whether they show real, synthetic, or demonstration data.

The submission still contains template placeholders and editorial errors, including the correspondence address, dates, DOI, patent text, supplementary-material placeholders, generic Data Availability and Acknowledgments text, incorrect author initials “A.A. and B.B.,” duplicated punctuation, and inconsistent system names such as CareSync, T-Rehab, and RehabXR. Replace these sections with final information and provide the exact dataset DOI, analysis code, parameter files, derived JSON data, and a versioned repository. The current problems require a complete reanalysis, not only textual revision.

Author Response

Reviewer Comment 1

Response:

Thank you for this detailed comment. We have addressed all of these issues in the revised manuscript. The dataset description and unit of analysis have been corrected to reflect the 26 analyzed participants, with each participant contributing one recording divided into three facial ROIs. The evaluation methodology has been completely revised, using consecutive 30-second windows and treating ROI/window measurements as repeated observations rather than independent participants. All respiratory-rate validation claims have been removed because the dataset provides ECG reference data but no respiratory reference. The previous POS evaluation and related conclusions have also been replaced with a new quantitative evaluation comparing GREEN, CHROM, POS, PCA, and ICA against the ECG reference. Accordingly, the previous claims concerning “Zero-Bias POS,” statistical significance, and the reported improvement have been removed. The revised results and evaluation protocol are presented in Section 5.

Reviewer Comment 2

Response

Thank you for this important comment. We have revised the research questions to align them with the actual scope and evidence of the study. The previous RQs addressing physiological, emotion, and movement fusion validation, simulated versus real markerless tracking, and retrieval scalability have been replaced with narrower questions focusing on the CareSync architecture, the implementation of the updated T-Rehab prototype, and the quantitative evaluation of heart-rate estimation using five rPPG methods. The title, abstract, evaluation methodology, results, discussion, and conclusions have also been revised accordingly to ensure consistency with the updated research questions and experimental evidence.

Reviewer Comment 3

Response:

Thank you for this comment. We agree that these capabilities are not quantitatively validated in the current study. We have revised the manuscript to clearly distinguish prototype implementation and developer-conducted feasibility testing from quantitative validation. The quantitative evaluation is now limited to heart-rate estimation using the ECG reference and the five evaluated rPPG methods. Claims of validation for emotion, pain, stress, movement safety, joint angles, repetition counting, synchronization, graph retrieval, and 3D replay have been removed or explicitly qualified as prototype functionalities. Their quantitative validation against appropriate ground-truth references, participant-level evaluation, retrieval benchmarks, and synchronization measurements is identified as future work.

Reviewer Comment 4

Response

Thank you for this detailed comment. We have revised the physiological evaluation methodology to improve its reproducibility and have clarified the data-processing pipeline, including the ROI structure, 30-second non-overlapping windowing, physiological frequency band, processing of incomplete windows, and the implementation of the five evaluated rPPG methods, GREEN, CHROM, POS, PCA, and ICA. We also clarified that the dataset provides 78 ROI videos, and therefore the evaluation does not rely on MediaPipe Face Landmarker to generate the facial ROIs used in the benchmark. The previous description suggesting otherwise has been corrected.

We have also removed the unsupported RMSSD-based stress estimation from the validated physiological evaluation. The revised study focuses on quantitative heart-rate estimation against the available ECG reference. The revised methodological details and processing procedure are described in Section 5.

Reviewer Comment 5

Response

Thank you for this important comment. We agree that the previous claims exceeded the evidence provided by the current study. We have therefore reframed the work as an unvalidated technical prototype and moderated the “human-centered,” “clinical,” and “personalized decision-support” claims throughout the manuscript. We now clearly distinguish technical feasibility from clinical and human-factors validation. Clinical evaluation involving patients and clinicians, usability studies, clinical outcomes, and prospective deployment are identified as future work.

Reviewer Comment 6

Response

Thank you for this detailed comment. We have clarified the graph construction and implementation details in the revised manuscript. The session graph uses a fixed 3-second sliding window and an undirected weighted representation, with edge weights representing co-occurrence counts. We have also clarified that this representation captures co-occurrence patterns within a bounded time window rather than event order or precise timing.

The discretization rules, missing-state handling, zero-denominator case, and weighted-Jaccard similarity formulation are now explicitly provided in the revised manuscript. We have also clarified the current normalization approach and identified duration- and event-frequency normalization as opportunities for further improvement.

No baseline comparison or clinician evaluation was performed in this study, and the corresponding claims have been moderated. Future work will evaluate the approach against established retrieval baselines and assess the relevance and interpretability of the retrieved patterns with clinical experts.

for 3D Posture Reconstruction and Wrist-Measurement Modality, We have clarified the limitations of the stored kinematic representation in the revised manuscript. The current telemetry contains local per-joint angle values but does not include global position, root orientation, segment lengths, or depth information. Therefore, it does not support a unique, spatially calibrated 3D posture reconstruction.

The 3D replay description has accordingly been revised to present the feature as an angle-driven Three.js animation based on the recorded joint angles. We also explicitly state that no reconstruction error against a ground-truth 3D reference has been measured. Mouse and touch input are now described only as coarse interaction signals, rather than alternative wrist-measurement modalities.

Reviewer Comment 7

Response

Thank you for this important comment. We have expanded the revised manuscript with an Ethics and Data Governance section and clarified that the implemented components were exercised through developer-conducted feasibility testing, rather than participant or clinical evaluation.

The revised section now addresses consent and ethics approval requirements, secondary use of existing datasets, dataset licensing and usage conditions, pseudonymization, encryption, access control, data retention and deletion, re-identification risks, and potential demographic and skin-tone-related performance bias. We also clarify that derived data, including facial blendshapes, body-motion trajectories, physiological measurements, and session events, may remain sensitive even when raw video is not retained.

These additions clarify the distinction between the technical implementation and future human-subject evaluation, while identifying privacy, data governance, and fairness assessment as important requirements for subsequent participant-based studies.

Reviewer Comment 8

Response:

Thank you for this detailed comment. We agree that the original evaluation required substantial revision. We have therefore replaced the previous evaluation methodology with a new analysis based on the available ECG reference data.

The revised evaluation now compares five rPPG methods, GREEN, CHROM, POS, PCA, and ICA, using consecutive, non-overlapping 30-second analysis windows. The analysis covers 26 participants, 78 ROI recordings, and 1,554 complete windows. Respiratory-rate evaluation has been removed because the dataset does not provide a corresponding respiratory reference.

Table 1 has been completely revised and now reports the performance of the five methods using MAE, RMSE, mean bias, valid estimates, and the percentage of estimates within a 5% tolerance threshold. Table 2 provides representative window-level results, while the complete results for all 1,554 windows are provided in Supplementary Table S1. The generated JSON results file is also provided as supplementary material.

The previous Figure 4 and the associated participant-sequence visualization have been removed. The revised figures now present the ECG reference and rPPG estimates across the standardized 30-second windows and the corresponding error comparison, providing a representation consistent with the revised analysis.

The previous Figure 4 has been removed and replaced with figures consistent with the revised 30-second window-based evaluation. Figures 2 and 3 are identified as developer-conducted feasibility-testing outputs rather than participant or clinical evaluation.

We corrected remaining editorial and metadata issues, standardized CareSync and T-Rehab terminology, and provided the complete JSON results as supplementary material.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

comments are attached below

Comments for author File: Comments.pdf

Author Response

Reviewer Comment 1:

Research questions are not fully evaluated. RQ2 requires evidence that real

markerless tracking improves assessment accuracy, and RQ3 asks about retrieval

speed/scalability, but neither receives quantitative experimental evaluation. Indeed,

retrieval/synchronization evaluation is deferred to future work.

Response:

Thank you for this important observation. We agree that the original research questions were broader than the quantitative evidence provided in the manuscript. In the revised version, we have therefore reformulated the research questions to align them with the actual scope of the study and the evaluation performed. In particular, RQ2 no longer claims to quantitatively demonstrate an improvement in assessment accuracy through markerless tracking, and RQ3 no longer addresses retrieval speed or scalability, as these aspects were not quantitatively evaluated in the current study. The revised research questions focus on the integration and implementation of the CareSync/T-Rehab framework and on the quantitative evaluation of the physiological sensing component using five rPPG methods against the ECG reference.

Reviewer Comment 2:

The HR results contain an important numerical inconsistency. Table 1 appears to

contain only 26 subjects rather than 81 sessions, with Subject_4 absent. Moreover,

the listed filtered deviations average approximately 13.8 BPM, which seems

inconsistent with the conclusion that filtering overcomes a 13.75-BPM MAE barrier;

the authors should provide complete 81-session results and explicit raw-versus-

filtered aggregate metrics.

Response:

We sincerely thank the reviewer for pointing out these vital formatting and evaluation errors. We fully agree that Table 1 should reflect the exact session-level cohort (N = 26subjects, M = 78 sessions from Subject 1 to Subject 27, excluding Subject 4), restrict its physiological scope exclusively to heart rate (as no respiratory reference sensor is present in the rPPG-10 benchmark.

In the revised manuscript, we have changed the evaluation methodology to provide a more systematic quantitative assessment of the physiological sensing component. The revised evaluation compares five rPPG methods, GREEN, CHROM, POS, PCA, and ICA, against the synchronized ECG reference using standardized 30-second analysis windows. Performance is now assessed using MAE, RMSE, mean bias, and the percentage of estimates within a 5% tolerance threshold.

Reviewer Comment 3:

Respiratory-rate validation is incomplete. Although an RR MAE equation is given,

Table 1 reports only the reference respiratory rate and does not provide predicted RR

values or an overall RR MAE, so the claimed RR validation cannot be assessed.

Response:

Thank you for pointing this out. We agree that the respiratory-rate validation was not sufficiently supported by the available reference data. The evaluation dataset used in this study provides synchronized ECG reference measurements for heart-rate evaluation, but does not provide a corresponding reference respiratory-rate measurement suitable for quantitative validation. Therefore, we have removed the respiratory-rate validation claim, the RR MAE analysis, and the corresponding RR results from the revised manuscript. The quantitative evaluation is now focused exclusively on heart-rate estimation using the ECG reference.

Reviewer Comment 4:

Emotion, pain, stress, and kinematic outputs need validation. These are

important components of the claimed system, yet emotion/pain/stress are heuristic

and the paper acknowledges the absence of internal-state labels; likewise,

markerless joint measurements are not compared with ground truth in this study.

Response:

Thank you for this important comment. We agree that emotion, pain, stress, and markerless kinematic outputs were not quantitatively validated against independent ground-truth annotations or motion measurements in the current study. These components are implemented as part of the T-Rehab prototype and were assessed only through developer-conducted feasibility testing to verify their end-to-end operation and expected output generation. We have therefore revised the manuscript to explicitly distinguish technical feasibility from quantitative validation and to avoid presenting these outputs as clinically validated measurements. The absence of ground-truth validation for affective and kinematic outputs is now explicitly identified as a limitation, and their quantitative validation against appropriate reference data is included as future work.

Reviewer Comment 5:

Support the claim of “statistically significant improvement.” The manuscript

states statistical significance but does not report the statistical test, confidence

intervals, effect size, or p-value. The term “Zero-Bias POS” also needs clearer

justification and comparison with standard POS plus conventional physiological

band-pass filtering.

Response

Thank you for this important comment. We agree that the previous statement regarding a “statistically significant improvement” was not sufficiently supported because the manuscript did not report the statistical test, confidence interval, effect size, or p-value. As part of the revision, we have changed the evaluation methodology and removed the unsupported claim of statistical significance. The revised evaluation instead provides a quantitative comparison of five rPPG methods, GREEN, CHROM, POS, PCA, and ICA, against the ECG reference using standardized 30-second analysis windows. Performance is assessed using MAE, RMSE, mean bias, and the percentage of estimates within a 5% tolerance threshold.

We have also removed the previous “Zero-Bias POS” formulation and its associated improvement claim. The revised manuscript evaluates the standard POS method within the common physiological-band processing pipeline and compares it directly with the other four rPPG methods. This change provides a more consistent and reproducible comparison across the evaluated methods and avoids making claims that cannot be supported by the revised experimental design.

Reviewer Comment 6:

Standardize terminology and presentation. CareSync, T-Rehab, and RehabXR are

used somewhat inconsistently; figure numbering and terminology should be checked

throughout.

Response:

Thank you for this comment. We have standardized the terminology, figure and table numbering, captions, and cross-references throughout the revised manuscript.

Reviewer Comment 7:

Remove remaining journal-template text. The manuscript still contains

placeholder correspondence information, Patents/Supplementary/Data Availability

instructions, and incomplete author-contribution placeholders.

Response:

Thank you for this comment. We have removed all remaining template text and incomplete placeholders from the revised manuscript.

Reviewer Comment 8:

Cite the following.

TriViT-Lite: a compact vision transformer–MobileNet model with texture-aware

attention for real-time facial emotion recognition in healthcare

Smartphone-Based Acoustic Sensing for Breathing and Heartbeat Detection via AoA Clustering in Indoor Environments

Response:

Thank you for this comment. The two requested references have been added to the revised manuscript.

Author Response File: Author Response.pdf

Round 2

Reviewer 1 Report

Comments and Suggestions for Authors

The revised manuscript has addressed most of the major concerns raised in the previous review, particularly by correcting the description of the rPPG-10 dataset, removing unsupported respiratory-rate validation, narrowing the research questions, and clearly distinguishing prototype functionality from quantitatively validated components. The new heart-rate analysis based on 26 participants, 78 ROI recordings, and 1,554 30-s windows represents a substantial improvement and is now much better aligned with the available ECG reference data. In my view, the manuscript could be considered for publication after a minor revision, provided that the remaining methodological and presentation inconsistencies are corrected.

Before acceptance, the authors should carefully correct several reporting errors. In particular, the weighted-Jaccard equation should use the maximum edge weight in the denominator rather than the minimum, and the heart-rate equations should be checked because the reported “signed deviation” is currently written as an absolute difference and the RMSE expression is missing the squared error term. Table 1 also requires corrected column headings so that MAE, RMSE, mean bias, and the percentage within ±5% are labelled properly. The figure captions should likewise be checked: Figure 5 is described in the text as the ECG–rPPG comparison, but its current caption refers to joint-angle visualization, and Figure 6 should be described as a comparison of aggregate errors rather than an “error distribution” unless an actual distribution is shown.

Finally, the manuscript and response letter should be made fully consistent. The response states that an Ethics and Data Governance section was added, but the current manuscript does not contain the corresponding dedicated section, despite this having been explicitly promised to address consent, secondary data use, data protection, re-identification risk, and possible demographic/skin-tone bias. The authors should also remove the remaining template placeholders and internal references, correct the manuscript numbering and punctuation, and moderate the few residual statements implying clinical decision support or validated clinical utility where only technical feasibility has been demonstrated. These changes appear achievable without additional experiments and would improve the internal consistency and publication readiness of the revised paper.

Author Response

Reviewer Comment 1:

The weighted-Jaccard equation should use the maximum edge weight in the denominator rather than the minimum

Response: Thank you for pointing this out. We corrected the weighted-Jaccard equation to use the maximum edge weight in the denominator, ensuring the formulation is mathematically correct.

Reviewer Comment 2:

the heart-rate equations should be checked because the reported “signed deviation” is currently written as an absolute difference and the RMSE expression is missing the squared error term.

Response

The corresponding equations and their descriptions have been revised accordingly.

Reviewer Comment 3

Table 1 also requires corrected column headings so that MAE, RMSE, mean bias, and the percentage within ±5% are labelled properly

Response: Thank you for the comment. We corrected the column headings in Table 1 to clearly and consistently label MAE, RMSE, mean bias, and the percentage of estimates within ±5% of the ECG reference.

Reviewer Comment 4

The figure captions should likewise be checked: Figure 5 is described in the text as the ECG–rPPG comparison, but its current caption refers to joint-angle visualization, and Figure 6 should be described as a comparison of aggregate errors rather than an “error distribution” unless an actual distribution is shown.

Response:

Thank you for pointing this out. We carefully checked the figure captions and corrected them to match the content of the corresponding figures. Figure 5 is now explicitly described as the comparison between the ECG reference and the rPPG heart-rate estimates across the 30-second evaluation windows. Figure 6 is now described as an aggregate comparison of the mean absolute error (MAE) across the evaluated rPPG methods, rather than as an “error distribution,” since the figure presents aggregate method-level errors rather than a statistical distribution.

Reviewer Comment 5

the manuscript and response letter should be made fully consistent. The response states that an Ethics and Data Governance section was added, but the current manuscript does not contain the corresponding dedicated section, despite this having been explicitly promised to address consent, secondary data use, data protection, re-identification risk, and possible demographic/skin-tone bias. The authors should also remove the remaining template placeholders and internal references, correct the manuscript numbering and punctuation, and moderate the few residual statements implying clinical decision support or validated clinical utility where only technical feasibility has been demonstrated. These changes appear achievable without additional experiments and would improve the internal consistency and publication readiness of the revised paper.

Response:

Thank you for this helpful comment. We carefully reviewed the manuscript and response letter to ensure full consistency. A dedicated “Ethics and Data Governance” section has now been added to the manuscript, addressing informed consent, secondary use of data, data protection, pseudonymization, encryption, access control, data retention and deletion, re-identification risks associated with derived data, and potential demographic and skin-tone bias in camera-based physiological estimation.

We also removed the remaining template placeholders and internal drafting references and corrected the manuscript numbering, terminology, and punctuation. In addition, statements that could imply clinical decision support, clinical validation, or established clinical utility have been moderated. The manuscript now consistently describes CareSync/T-Rehab as demonstrating technical feasibility rather than validated clinical effectiveness or clinical decision support.

These revisions have been made without additional experiments and ensure that the manuscript and response letter are consistent with respect to the scope, evidence, ethics, governance, and limitations of the presented work.

Author Response File: Author Response.pdf

Reviewer 2 Report

Comments and Suggestions for Authors

satisfied

Author Response

Comment:

satisfied

Response:

Thank you for your positive feedback. We appreciate the reviewer’s assessment and are pleased that the revisions satisfactorily addressed the concerns raised.