Improving the Quality and Utility of Electronic Health Record Data through Ontologies

Round 1

Reviewer 1 Report

This withe paper is well written and discuss about a common problem in the biomedical domain: the quality of the information stored in EHRs. 

As it is explained by the authors, the paper is a white paper and intends to present the opinion of the authors. Thus, the evaluation of the scientific methodologies (e.g., research questions, state-of-the-art, analytical analysis, ...) are not applicable for this paper. I focused my review on the description of the problems and on the discussions about potential methods that should be deepen explored to solve those problems. However, I also would like to highlight the serious work of the authors and the extensive use of citations to scientific work in order to justify many of their claims. In white papers, the unbiased interpretation of the cited work is key to guide the readers and this interpretation was well done and helps the reader to understand the context/problem as well as to contextualise the opinion/recommendations of the authors.

I appreciate to read that there are many researchers worry about the practical application (or impact) of their work on the clinical practices. This paper highlights the need to go further than only propose new methods to solve existing problem. The collaboration with other actors of the domain (governments, software developers, hospital managers, etc.) is of high importance and need to be better supported and pushed forward. 

Although the relevance of this work, I have some comments/suggestion that could improve the quality of the discussions that this paper will raise up:

- The analysis started in 2016 and stopped in 2019. A refresh of the existing works on the related topics would be good. 

- In section 2.1., the full potential of EHR has 3 items. The difference between 1 and 2 is not clear. In 1, the HMI takes the time of healthcare professional, while in 2 the HMI is not adapted to fulfil documents. For me, this is the cause-effect problem.  Maybe the authors want to express the problem of “where the information is collected (as for 2) and when it is entered in EHR (as for 1) ?”.  The ultimate goal is that the data acquisition/entry task be part of the routine work of the healthcare professional, without adding extra effort. 

- In section 2.2., the item Data processing and storage list some requirements. These items describe specific issues that can be solved during the data acquisition tasks. The authors don’t highlight how data processing/storage can be improved by ontologies. In the previous item (data acquisition), the authors list the expectations of an ontology-base acquisition process. it is, in my opinion, a clearer way to present potential progress from the adoption of ontologies. However, mixing these two different ways to present the problem is misleading the understanding of this part of the text. In my opinion, the authors should focus on the “expectations” or on the “drawbacks”, but avoid mixing them. One way to do it is to link this list with tables 1 (how to fulfil the EHR requirements) and/or 2 (how to avoid EHR problems). Be clear on where to use the ontologies in order to address both (or at least one of these) issues.

 - In section 2.2., the item Ontology based data query and reuse. I think that query and re-use are two different aspect of knowledge exploitation and should not be put together. For instance, “Push and pull scenarios … “ will improve the query formulation or the re-use of the data? One is potentially related with query enrichment while the other is more related with the analysis of the answers. For the semantic interoperability perspective, the goal is to federate the queries (one query to multiple data sources), or to fusion the answers, or both (as illustrated in Box3)?

 - In section 4.5, it would be useful to clarify the meaning of the term “interface” and use different terms for different situations/meaning, when possible. The reader can get lost when the authors use the same term for: 

 -> the combined use of different standards and technologies (e.g., HL7 FHIR and voice recognition)

 -> the mapping between RUs (ICD and SNOMED CT)

 -> the reuse of terminologies within documents (ICD and guidelines guidelines)

 -> the software interface

 -> etc. 

- In the first paragraph of section 6, the text gives the impression that there no financial support to research on ontology-based EHR. Could you please clarify what kind of grants exist (e.g., in USA and EU) and how much is used for improving biomedical data quality? Or, at least, give some comparative information that justify the need of new (or more) funding.

- In page 21, the authors state that "The fact that EHR systems so dramatically lag behind in usability when compared to similar artifacts in other spheres..." . I don’t know another domain where ontologies are so developed. The biomedical domain was one of the first domain that built and used ontologies in practice. There are more than 800 ontologies published in Bioportal and the data quality and systems interoperability problems are still far from being solved. Where are the practical examples of hospitals that solved their problems using ontologies? If the authors can find examples to support their arguments, it would be easier to convince the readers that it is the right way to go. 

Author Response

Please see the attachment

Author Response File: Author Response.pdf

Reviewer 2 Report

This comprehensive paper calls for a new paradigm for clinical computing to be developed using hidden ontology-based knowledge structures in clinical data warehouses.

It offers insights on how to address several key issues, for example querying ontology-based knowledge structures for patient data using description logic queries and meta query languages.

Overall, the paper is an exciting read, drawing on core insights from practitioners in the field supported with examples of where deployed EHRs have fallen short on purposes and requirements as defined and detailed in Table 1.

The paper meets its core objectives well, to assemble evidence, identify existing problems in the use of EHR with supporting real world examples. It provides timely and practical guidance for the education of relevant stakeholders and the informatics community by signposting specific actions to advance quality of clinical data in future EHR design using ontological approaches.    

Overall, the paper is however lengthy and could be streamlined in parts.  Specifically, in Section 4, I have listed some possible edits that might be worth considering. As an educationalist, I do however think there is a value in presenting the material as is and would consider these edits as suggestions to the authors to decide if they wish to summarise or leave the sections as presented. What is clear on reviewing the various sections is the complexity is detailed well. The authors provide supporting evidence and explanations of real-world challenges experienced in this vast field of health informatics with important recommendations on phased approaches to adopt.

While the paper is complex and covers a broad field of relevant theory, it important to present the material to the reader and support next steps for the CTSA and wider community.

In summary, I think that this is an important paper which includes a detailed but necessary broad scope to address the topic of EHR development with ontology including semantics, interoperability, standards and other key associated topics included.  The glossary and boxes are a good addition to the piece. I found it an interesting and relevant read and would suggest that it meets well its core objectives. As a published paper I will be signposting my students to read it and discuss further in class with a view to progressing proposed recommendations.

Specific edits

Line 426 edit FIHR to FHIR

Line 605 first sentence on clinical decision support is there a reference for this sentence that you might include as its quite a strong opinion

Possible lines identified for editing or summarising with associated references.

Lines 446 to 455

Lines 496 to 508

Lines 513 to 524

Lines 584 to 592

Lines 662 to 629

Author Response

This comprehensive paper calls for a new paradigm for clinical computing to be developed using hidden ontology-based knowledge structures in clinical data warehouses.

It offers insights on how to address several key issues, for example querying ontology-based knowledge structures for patient data using description logic queries and meta query languages.

Overall, the paper is an exciting read, drawing on core insights from practitioners in the field supported with examples of where deployed EHRs have fallen short on purposes and requirements as defined and detailed in Table 1.

The paper meets its core objectives well, to assemble evidence, identify existing problems in the use of EHR with supporting real world examples. It provides timely and practical guidance for the education of relevant stakeholders and the informatics community by signposting specific actions to advance quality of clinical data in future EHR design using ontological approaches.    

Overall, the paper is however lengthy and could be streamlined in parts.  Specifically, in Section 4, I have listed some possible edits that might be worth considering. As an educationalist, I do however think there is a value in presenting the material as is and would consider these edits as suggestions to the authors to decide if they wish to summarise or leave the sections as presented. What is clear on reviewing the various sections is the complexity is detailed well. The authors provide supporting evidence and explanations of real-world challenges experienced in this vast field of health informatics with important recommendations on phased approaches to adopt.

While the paper is complex and covers a broad field of relevant theory, it important to present the material to the reader and support next steps for the CTSA and wider community.

In summary, I think that this is an important paper which includes a detailed but necessary broad scope to address the topic of EHR development with ontology including semantics, interoperability, standards and other key associated topics included.  The glossary and boxes are a good addition to the piece. I found it an interesting and relevant read and would suggest that it meets well its core objectives. As a published paper I will be signposting my students to read it and discuss further in class with a view to progressing proposed recommendations.

Answer: We thank very much the reviewer’s comments and acknowledgement of the importance of this work.

Specific edits

Line 426 edit FIHR to FHIR

Answer: Thank you for pointing out this typo. We have corrected the typo.

Line 605 first sentence on clinical decision support is there a reference for this sentence that you might include as its quite a strong opinion

Answer: We added the references #68,#69 to support our statement. Please see redline Line 616-618.

Possible lines identified for editing or summarising with associated references.

Lines 446 to 455

Answer: We added the references #34-#41.

Lines 496 to 508

Answer: We added the references #48-#49.

Lines 513 to 524

Answer: We added the references #50-#52, and #54.

Lines 584 to 592

Answer: We added the references #64-#67.

Lines 662 to 629

Answer: We added the references #71-#75.