Metadata Framework to Support Deployment of Digital Health Technologies in Clinical Trials in Parkinson’s Disease
2. Metadata and Standardization of Drug Development Tools: Learning from Neuroimaging
3. Proposed Metadata Framework for DHTs
- Measurement Device and Hub metadata,
- Sensor and signal metadata,
- Participant/Population metadata,
- Analysis metadata,
- Experimental metadata and
- Contextual metadata
3.1. Pre-Specification of Metadata
3.2. Application-Independent Metadata
- Measurement device and hub: Comprises metadata that uniquely identifies the:
- Measurement device used for data collection, including its brand, model, serial number (medical device UDI where available), hardware and firmware version. This metadata needs to allow the device location on the body and orientation to be recorded. Furthermore, by tracking individual device ID, any change in performance over time or repairs can be associated with the data.
- Hub: Since many wearable measurement devices (e.g., smartwatch) work in combination with a separate device (e.g., smartphone or more generically, a data hub) in order to interconnect with a remote database and potentially also to perform other functions such as pre-processing and authentication, the application-independent metadata also includes metadata to uniquely describe the hardware/software of the hub which the measurement device connects.
- The file format and technical aspects of the data storage and transfer (compression, encryption).
- Metadata version. The metadata framework needs to be able to be refined so it is important that there is a metadata version associated with the device collecting data.
- Sensor and Signals: is the description of the types of data collected including the modality (e.g., accelerometer, EEG/electroencephalogram, ECG/electrocardiogram, PPG/photoplethysmogram), the recording mode and any calibration of the sensor performed prior to deployment in each study, data rates and timing. A single DHT may generate multiple signals with distinct metadata, for example, a DHT might include an accelerometer, gyroscope, magnetometer and PPG sensor, each operating at different acquisition frequency and with different timing information. The metadata framework supports this through a single device supporting multiple sets of sensor metadata. The signals can cover traditional wearable sensor signals, but may also be used for environmental context signals, such as the ambient temperature where subject is located, whether the subject is indoors or outdoors (this could be a binary signal), which room in their home they are located in (the signal would be a number specifying the room identity) and whether they are alone in that room or accompanied.
- Participant/Population: We propose that the application-independent metadata has a single element of participant/population metadata, namely a unique identifier that can be linked to this subject data in the application-dependent metadata.
- Analysis metadata: The application-independent metadata needs to describe any generic analysis performed in the device itself (e.g., the device might output step count or heart rate variability), which we refer to as “pre-processing” to distinguish from endpoint-specific analysis that is application dependent.
- Experiment metadata: We propose that the only application-independent metadata element for the experimental metadata is an experiment identifier. The details of the experiment being performed are application dependent.
3.3. Application-Dependent Metadata
3.3.1. Implementation of Application-Dependent Metadata
- Subject metadata: The application-independent metadata only includes a subject unique identifier (UID). The application-dependent metadata includes the relevant demographics and associated health information (e.g., medical history) relevant to the clinical study concerned, the inclusion and exclusion criteria and any comorbidities relevant to this study.
- Analysis metadata: The data analysis is in many cases very specific to the clinical trial design. We refer to this application-dependent analysis as “endpoint analysis” to distinguish from the generic pre-processing described in the application-independent metadata. All relevant software versions and selectable parameters must be clearly defined.
- Experiment metadata: This describes the clinical trial cohort in which a given subject is enrolled, the clinical site, any clinical trial questionnaire or human (e.g., physician) observation metadata, and a reference to the applicable protocol and its version number. In particular, this metadata needs to include details of any active tests and passive monitoring involved, and the details of the active test.
- Contextual data: A description of the environment of data collection (e.g., clinic, home) and properties of the environment (such as ambient temperature, noise level and light level) if available should be included in the metadata.
3.3.2. Challenges of Application-Dependent Metadata
- Variability in metadata requirements across clinical applications and sensor modalities: For example, we may be interested in monitoring gait in patients with Parkinson’s disease. In one specific clinical trial, we may want to evaluate a patient’s gait using a wrist-worn wearable device in a clinical setting during the performance of a 6-min walk test. In that instance, it may be necessary to record, as metadata, the actual length of the lab or walkway that the patient is using for the test and whether the test was performed with or without caregiver support. This application-dependent metadata would not be required if one would like to evaluate the same patient’s gait at home. Similarly, in some clinical trials, data might be acquired continuously (passively), and for other applications, data would be collected when the subject is prompted to perform a task or complete a PRO (active). For active tasks, the application-dependent metadata would need to include a description of the prompt or simultaneous PRO to fully describe the data collection. The metadata framework proposed here incorporates the necessary detail in the experiment metadata portion of the application-dependent metadata.
- Variability in metadata requirements across different stages of PD: Severity of disease would also have a significant impact on the metadata that should be recorded within a particular trial. If studying individuals with probable PD in the pre-manifest stage of the disease, there may be minimal motor symptoms, and as such subjects may often engage in vigorous activities such as running that would be captured by a continuously recording activity monitor. This would not be the case for patients with advanced disease, who may struggle to safely and independently navigate their own homes. Thus, if we were to devise a measure of “average daily activity” it would greatly vary across these two populations. In the case of pre-manifest PD, a study might measure the amount of time of moderate-to-vigorous activity per week and in the case of advanced PD, a study might seek to measure any and all activity. In the former group, we would need to capture extraneous factors that may have impacted a patient’s ability to perform vigorous activity: if we are monitoring a golfer who usually plays 2–3 times/week, a month-long weather pattern may substantially alter their activity levels. In the latter group, these factors may not be as relevant. The flexible design of the application-dependent metadata format in the proposed framework allows this variability to be described in the experiment metadata and Analysis metadata.
- Data pre-processing: Another challenge we face is that DHTs do not always provide ready access to the raw data, as we are used to collecting from research-grade clinical equipment. Additionally, even if there is access to some version of the raw data, these data often vary greatly across devices, based on the manufacturer or even the version of a specific device. Smartwatch actigraphy devices that generally report step count over a defined epoch often claim to also output raw data that we hope to use for clinical research. Indeed, the term “raw data” is seldom the output of the analogue to digital converters (ADCs) in the sensor, but normally has filters or data compression applied and is often the output of a software interface (API) provided by the manufacturer. Data streams with such differences may not be used interchangeably. The application-independent metadata in our proposed framework addresses this challenge by including both sensor metadata fields that records software and hardware versions and the Device ID. The Device section of the application-independent metadata in the framework therefore uniquely identifies the type of pre-processed data available, even when the details of the pre-processing are not provided by the manufacturer. Given precise specification of this device metadata, lab-based experiments can be used to test whether the data output from different hardware/software versions of the same device are sufficiently similar to be combined in a particular context.
- Data analysis: In addition to pre-processing on the wearable hardware itself, DHTs involve analysis to calculate measurements of interest from the sensor data. This analysis is normally done after pre-processing, and on one or more separate devices such as a mobile phone app, a home hub, or cloud server. Data analysis software can evolve and be changed, and indeed some of this type of analysis software is “self-learning” and changes how it works in use. The proposed framework addresses this challenge through a detailed description of all pre-processing in the application-independent metadata, including details of hardware and software versions of any smartphone app or hub that is used as an intermediary between a data collection device and the data analysis platform. The application-dependent metadata in our framework then uniquely identifies the data analysis software, including its version number that is applied to this input.
- Controlling environmental sources of variability: The environment of data collection is an important source of variability. For example, a clinical trial subject’s mobility may depend on the ambient temperature as well as their symptoms and treatment, and behaviours and activities are influenced by whether the clinical trial subject is on their own or in the same room as a family member or care partner. The way a subject walks may also depend on whether they are inside or outside their home, or which room in the house they are in. A timed up and go task will be influenced by the height of the chair from which the subject stands up. Increasingly, clinical trials are capturing information about these environmental factors. In our framework, such environmental data is captured as a separate sensor (e.g., temperature) but other environmental context would more appropriately be captured in an experiment metadata portion of the application-dependent metadata, for example whether a particular assessment is being performed in the clinic or at home, and whether it is done as part of a prompted task or passively.
4. Use Case Example—Tremor in Parkinson’s Disease
Informed Consent Statement
Data Availability Statement
Conflicts of Interest
- Warmerdam, E.; Hausdorff, J.M.; Atrsaei, A.; Zhou, Y.; Mirelman, A.; Aminian, K.; Espay, A.J.; Hansen, C.; Evers, L.J.W.; Keller, A.; et al. Long-Term Unsupervised Mobility Assessment in Movement Disorders. Lancet Neurol. 2020, 19, 462–470. [Google Scholar] [CrossRef]
- Jack, C.R.; Bernstein, M.A.; Fox, N.C.; Thompson, P.; Alexander, G.; Harvey, D.; Borowski, B.; Britson, P.J.; L Whitwell, J.; Ward, C.; et al. The Alzheimer’s Disease Neuroimaging Initiative (ADNI): MRI Methods. J. Magn. Reson. Imaging 2008, 27, 685–691. [Google Scholar] [CrossRef] [PubMed][Green Version]
- Brazma, A.; Hingamp, P.; Quackenbush, J.; Sherlock, G.; Spellman, P.; Stoeckert, C.; Aach, J.; Ansorge, W.; Ball, C.A.; Causton, H.C.; et al. Minimum Information about a Microarray Experiment (MIAME)—Toward Standards for Microarray Data. Nat. Genet. 2001, 29, 365–371. [Google Scholar] [CrossRef] [PubMed]
- Badilini, F.; Young, B.; Brown, B.; Vaglio, M. Archiving and Exchange of Digital ECGs: A Review of Existing Data Formats. J. Electrocardiol. 2018, 51, S113–S115. [Google Scholar] [CrossRef] [PubMed]
- Brinkmann, B.H.; Bower, M.R.; Stengel, K.A.; Worrell, G.A.; Stead, M. Large-Scale Electrophysiology: Acquisition, Compression, Encryption, and Storage of Big Data. J. Neurosci. Methods 2009, 180, 185–192. [Google Scholar] [CrossRef] [PubMed][Green Version]
- Rübel, O.; Dougherty, M.; Prabhat; Denes, P.; Conant, D.; Chang, E.F.; Bouchard, K. Methods for Specifying Scientific Data Standards and Modeling Relationships with Applications to Neuroscience. Front. Neuroinformatics 2016, 10, 48. [Google Scholar] [CrossRef] [PubMed][Green Version]
- Teeters, J.L.; Godfrey, K.; Young, R.; Dang, C.; Friedsam, C.; Wark, B.; Asari, H.; Peron, S.; Li, N.; Peyrache, A.; et al. Neurodata Without Borders: Creating a Common Data Format for Neurophysiology. Neuron 2015, 88, 629–634. [Google Scholar] [CrossRef] [PubMed][Green Version]
- IEEE 1752.1-2021—IEEE Approved Draft Standard for Mobile Health Data. Available online: https://standards.ieee.org/standard/1752_1-2021.html (accessed on 28 June 2021).
- OHDSI—Observational Health Data Sciences and Informatics. Available online: https://www.ohdsi.org/ (accessed on 15 September 2021).
- Badawy, R.; Hameed, F.; Bataille, L.; Little, M.A.; Claes, K.; Saria, S.; Cedarbaum, J.M.; Stephenson, D.; Neville, J.; Maetzler, W.; et al. Metadata Concepts for Advancing the Use of Digital Health Technologies in Clinical Research. Digit. Biomark 2019, 3, 116–132. [Google Scholar] [CrossRef] [PubMed]
- BEST (Biomarkers, EndpointS, and Other Tools) Resource—NCBI Bookshelf. Available online: https://www.ncbi.nlm.nih.gov/books/NBK326791/ (accessed on 7 September 2021).
- Califf, R.M. Biomarker Definitions and Their Applications. Exp. Biol. Med. 2018, 243, 213–221. [Google Scholar] [CrossRef] [PubMed]
- Roussos, G. Identifying and Characterising Sources of Variability in Digital Outcome Measures in Parkinson’s Disease. NPJ Digit. Med. 2021. in preparation. [Google Scholar]
- Stephenson, D.; Alexander, R.; Aggarwal, V.; Badawy, R.; Bain, L.; Bhatnagar, R.; Bloem, B.R.; Boroojerdi, B.; Burton, J.; Cedarbaum, J.M.; et al. Precompetitive Consensus Building to Facilitate the Use of Digital Health Technologies to Support Parkinson Disease Drug Development through Regulatory Science. DIB 2020, 4, 28–49. [Google Scholar] [CrossRef] [PubMed]
- Apple Watch for Parkinson’s. Available online: https://watchpdstudy.org (accessed on 28 June 2021).
- Jagust, W.J.; Bandy, D.; Chen, K.; Foster, N.L.; Landau, S.M.; Mathis, C.A.; Price, J.C.; Reiman, E.M.; Skovronsky, D.; Koeppe, R.A. The ADNI PET Core. Alzheimer’s Dement. 2010, 6, 221–229. [Google Scholar] [CrossRef] [PubMed][Green Version]
- Jagust, W.J.; Landau, S.M.; Koeppe, R.A.; Reiman, E.M.; Chen, K.; Mathis, C.A.; Price, J.C.; Foster, N.L.; Wang, A.Y. The Alzheimer’s Disease Neuroimaging Initiative 2 PET Core: 2015. Alzheimer’s Dement. 2015, 11, 757–771. [Google Scholar] [CrossRef] [PubMed][Green Version]
- FDA. Grants Accelerated Approval for Alzheimer’s Drug. Available online: https://www.fda.gov/news-events/press-announcements/fda-grants-accelerated-approval-alzheimers-drug (accessed on 1 September 2021).
- FDA. Clinical Trial Imaging Endpoint Process Standards Guidance for Industry; Food and Drug Administration: Silver Spring, MD, USA, 2018; p. 31. [Google Scholar]
- Obuchi, S.P.; Kawai, H.; Garbalosa, J.C.; Nishida, K.; Murakawa, K. Walking Is Regulated by Environmental Temperature. Sci. Rep. 2021, 11, 12136. [Google Scholar] [CrossRef] [PubMed]
|Metadata Name||Pre-Specified Value?||Application-Dependent?||Metadata Item APDM||Metadata Item WATCH-PD|
|Model||Yes||No||Opal||iPhone and Apple Watch|
|Hardware version||Yes||No||Opal v.2||Apple Watch Series 5|
Apple iPhone 11
|Firmware version||Yes||No||20190315||Apple Watch Series 5|
Apple iPhone 11
|Body site||Yes||No||Each wrist||Most affected wrist (Apple Watch)|
|Orientation||Yes||No||Sensors are positioned with device’s charging ports oriented toward more distal body locations||Watch orientation set during initial setup, with watch ‘crown’ positioned facing the hand. Watch on most-affected side|
|Hub/App||Yes||No||Mobility Lab Hub||BrainBaseline version ‘WATCH-PD’|
|Sensor and Signal|
|Sensor type||Yes||No||Accelerometer, Magnetometer, Gyroscope, Barometer||Accelerometer, Gyroscope|
|Calibration||Yes||No||Opal calibration process||N/A|
|Units||No||No||os (gyro); m/s2 (accel); pT (mag); mPa (bar)||os (gyro); m/s2 (accel)|
|Filename of data file||No||No|
|Data rate||Yes||No||128 Hz from all sensors||100 Hz accel and gyro in clinic|
50 Hz accel in home-base passive monitoring
|Timing||Yes||No||Continuous data collection from all sensors during clinic assessment||Continuous data collection during clinic assessment.|
7 day home based data collection
|Multiple data streams||Yes||No||Accelerometer [xyz], Magnetometer [xyz], Gyroscope [xyz], Barometer||Accelerometer [xyz], Gyroscope [xyz]|
|Yes||Yes||Inclusion criteria from WATCH-PD protocol||Inclusion criteria from WATCH-PD protocol|
|Comorbidities||No||Yes||Not available||Not available|
|Data pre-processing||Yes||No||Not available||Not available|
|Endpoint analysis||Yes||Yes||WATCH-PD SAP v1.0||WATCH-PD SAP v1.0|
|Protocol||Yes||Yes||WATCH-PD Protocol v3.0||WATCH-PD Protocol v3.0|
|Questionnaires and scales||Yes||Yes||MDS-UPDRS 2.10||In-clinic:|
Single item 7-point Likert response indicating current tremor severity
|Clinical assessments/Human observation||Yes||Yes||MDS-UPDRS 3.15, 3.16, 3.17; RUE 1 and LUE 2 assessments only||In-clinic:|
MDS-UPDRS 3.15, 3.16, 3.17; RUE and LUE assessments only
N/A—No human observation during home assessments
|Clinical follow-up frequency as a comparator for digital measures||Yes||Yes||Baseline, 1 month, 3 month, 6 month, 9 month, 12 month||Baseline, 1 month, 3 month, 6 month, 9 month, 12 month|
|Clinical correlations to digital measures||Yes||Yes||Sensor collection simultaneous with clinical assessments described above||In clinic, sensor collection simultaneous with clinical assessments described above|
|Active test details||Yes||Yes||Sensor collection simultaneous with clinical assessments as per protocol.||In clinic, sensor collection simultaneous with clinical assessment as per protocol.|
|Passive monitoring||Yes||Yes||N/A||At Home, 7 days passive monitoring.|
|Environmental context (carer, temperature, indoor/outdoor, …)||Not available||Yes||Not available||Not available|
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.
© 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/).
Hill, D.L.; Stephenson, D.; Brayanov, J.; Claes, K.; Badawy, R.; Sardar, S.; Fisher, K.; Lee, S.J.; Bannon, A.; Roussos, G.; et al. Metadata Framework to Support Deployment of Digital Health Technologies in Clinical Trials in Parkinson’s Disease. Sensors 2022, 22, 2136. https://doi.org/10.3390/s22062136
Hill DL, Stephenson D, Brayanov J, Claes K, Badawy R, Sardar S, Fisher K, Lee SJ, Bannon A, Roussos G, et al. Metadata Framework to Support Deployment of Digital Health Technologies in Clinical Trials in Parkinson’s Disease. Sensors. 2022; 22(6):2136. https://doi.org/10.3390/s22062136Chicago/Turabian Style
Hill, Derek L., Diane Stephenson, Jordan Brayanov, Kasper Claes, Reham Badawy, Sakshi Sardar, Katherine Fisher, Susan J. Lee, Anthony Bannon, George Roussos, and et al. 2022. "Metadata Framework to Support Deployment of Digital Health Technologies in Clinical Trials in Parkinson’s Disease" Sensors 22, no. 6: 2136. https://doi.org/10.3390/s22062136