Literature Review of Safety Event Reporting in Observational Studies: Challenges Extrapolating across Comparable Products

Round 1

Reviewer 1 Report

Comments and Suggestions for Authors

Dear Authors,

I have read the manuscript and I send you my comments:

1) text must be revised there are format "error"

2) please clarify in abstract the limits of the study on drug interactions related to paxlovid

Comments on the Quality of English Language

none

Author Response

Response:

We thank Reviewer 1 for their careful review of our manuscript and the comments above.

1) With respect to formatting errors, we have noted this feedback for the journals editorial team and will ask for any errors to be corrected prior to your receipt of the updated manuscript.

2) Regarding limitations, we have expanded the limitations section of the manuscript to note additional aspects of the products selected for this review (i.e. differences in chemical structure and product indication) that would impact the comparability with PAXLOVID.  Additionally, we have stated more explicitly in the conclusions section that the exploratory review approach that was undertaken did not yield the information that was of interest.

Your time in reviewing the updated manuscript is much appreciated and we hope you find the revision acceptable for publication.

Reviewer 2 Report

Comments and Suggestions for Authors

MAJOR COMMENTS

- In the introduction section it is stated that "One method for acquiring proxy information on the safety characteristics of interest during the early postauthorization period would be a real-world literature review on comparator products." This statement is highly confusing: If the aim is to collect safety information for a new drug, why would a real-world literature review on comparator products be helpful to address this aim? A "comparator" is usually something that is used as a treatment alternative for the drug of interest (e.g., representing the current standard-of-care for the indication of interest), and may thus be used as a comparator in a clinical trial. Or do the authors mean "comparable products", and not "comparator products"? Even here, it is uncertain, what "comparable" would mean - from the same class of medications, with similar chemical strcuture, with similar pharmacological properites (e.g., CYP3A4 inhibition), etc. Please clarify.

- It is stated that "SAEs associated with PAXLOVID comparators were evaluated in the postmarketing primary care setting as proxies for SAE incidence among PAXLOVID users." Why do the authors think that SAE frequency for other drugs could be used "as proxies" for Paxlovid? These drugs have different chemical structure, different pharmacological properties, and are used for different indications than Paxlovid. As focus was on SAEs (i.e., all serious events, including those associated with the indication and not thought to be related to exposure) the underlying indication is of high relevance.

- "SAE" is a pharmacovigilance term that derives from spontaneous or solicited reporting of adverse events (related and unrelated) in clinical trials, or observational studies with primary data collection ("field studies"). The study, however, included nearly exclusively observational studies based on healthcare databases (retrospective cohort studies). These type of studies use existing healthcare data to do (pharmaco-)epidemiological studies. By definition, there is thus no AE reporting in these type of studies. It it thus highly confusing and misleading if the authors conclude about these studies: "Most focused on a small, predefined list of SAEs and did not provide insight on the broader range of SAEs that might occur for the evaluated comparators, making it difficult to extrapolate a general profile of SAEs among users of the comparator treatments." As no AE reporting can occur in such type of studies, using these as a source to identify SAE frequencies is inappropriate. It is absolutely predictable that such studies will not be useful to provide proxy information (or contextualization information) for SAE frequencies and SAE spectrum, as it would be observed in primary data collection studies.

- Due to mixing up primary data collection observational studies and healthcare database studies, the following conclusions are also strongly misleading: "Overall, the results of the present literature review indicate that observational research typically analyzes a prespecified short list of SAEs in targeted special patient populations. This suggests that, even for a more limited patient population/more directly comparable product, limitations in the scope of the reported SAEs will occur." The view that observational research typically analyzes a prespecified short list of SAEs is based on the nature of the data source(s) used in the identified studies, not on the way how SAEs are reported and handled in observational studies with primary data collection.

 

MINOR COMMENTS

In the introduction section, it is stated that the aim of the study was "to describe the scope and frequency of SAEs among realworld users of PAXLOVID comparators". What do the authors mean with "scope"?

Methods: Include detailed time point (day) when was the search performed

 

Comments on the Quality of English Language

None.

Author Response

- In the introduction section it is stated that "One method for acquiring proxy information on the safety characteristics of interest during the early postauthorization period would be a real-world literature review on comparator products." This statement is highly confusing: If the aim is to collect safety information for a new drug, why would a real-world literature review on comparator products be helpful to address this aim? A "comparator" is usually something that is used as a treatment alternative for the drug of interest (e.g., representing the current standard-of-care for the indication of interest), and may thus be used as a comparator in a clinical trial. Or do the authors mean "comparable products", and not "comparator products"? Even here, it is uncertain, what "comparable" would mean - from the same class of medications, with similar chemical structure, with similar pharmacological properties (e.g., CYP3A4 inhibition), etc. Please clarify.

We agree that the use of the term comparator was not optimal, and have rephrased throughout the manuscript to better reflect the intent of our review: ‘comparators’ has been replaced by ‘products with similar pharmacological properties to PAXLOVID’. 

The introductory statement has been revised as follows to reflect the update in language and additionally to note the exploratory nature of the review conducted: “One exploratory method for acquiring proxy information on the safety characteristics of interest during the early postauthorization period would be a real-world literature review on products with similar pharmacological properties.”

The limitations section of the manuscript notes that there may be additional other pharmacokinetic and pharmacologic characteristics of the antibiotic and antifungal agents, such as product half-life and mechanism of action, that may limit suitability for comparison to the reference antiviral.

- It is stated that "SAEs associated with PAXLOVID comparators were evaluated in the postmarketing primary care setting as proxies for SAE incidence among PAXLOVID users." Why do the authors think that SAE frequency for other drugs could be used "as proxies" for Paxlovid? These drugs have different chemical structure, different pharmacological properties, and are used for different indications than Paxlovid. As focus was on SAEs (i.e., all serious events, including those associated with the indication and not thought to be related to exposure) the underlying indication is of high relevance.

We have revised the language in the review to refer to safety outcomes of interest rather than SAEs, in response to this comment and the subsequent comment from Reviewer 2 (please see below).  The rationale for reviewing safety outcomes of interest among products with comparable pharmacological properties was to obtain a general characterization of the real-world safety profile of PAXLOVID, in the absence of available long term data collected post-authorization. Therefore, through characterizing the interaction profile of products with comparable pharmacological properties, we can further characterize the Paxlovid safety profile and interaction profile, to understand impact on concomitantly prescribed products. We have expanded the limitations section on exposure definition to note the impact of heterogeneity in chemical structure and product indication on the potential to extrapolate observed safety outcomes to PAXLOVID.  The limitations and conclusion section the manuscript acknowledge that this exploratory effort to extrapolate the review results to PAXLOVID did not yield the data that was of interest. 

- "SAE" is a pharmacovigilance term that derives from spontaneous or solicited reporting of adverse events (related and unrelated) in clinical trials, or observational studies with primary data collection ("field studies"). The study, however, included nearly exclusively observational studies based on healthcare databases (retrospective cohort studies). These type of studies use existing healthcare data to do (pharmaco-)epidemiological studies. By definition, there is thus no AE reporting in these type of studies. It it thus highly confusing and misleading if the authors conclude about these studies: "Most focused on a small, predefined list of SAEs and did not provide insight on the broader range of SAEs that might occur for the evaluated comparators, making it difficult to extrapolate a general profile of SAEs among users of the comparator treatments." As no AE reporting can occur in such type of studies, using these as a source to identify SAE frequencies is inappropriate. It is absolutely predictable that such studies will not be useful to provide proxy information (or contextualization information) for SAE frequencies and SAE spectrum, as it would be observed in primary data collection studies.

We agree and have removed the term SAE from our review methods and results; the term safety outcomes has been applied instead. The methods section defines that the safety outcomes of interest were those that met the ‘serious’ criteria that was applied to the literature review.

- Due to mixing up primary data collection observational studies and healthcare database studies, the following conclusions are also strongly misleading: "Overall, the results of the present literature review indicate that observational research typically analyzes a prespecified short list of SAEs in targeted special patient populations. This suggests that, even for a more limited patient population/more directly comparable product, limitations in the scope of the reported SAEs will occur." The view that observational research typically analyzes a prespecified short list of SAEs is based on the nature of the data source(s) used in the identified studies, not on the way how SAEs are reported and handled in observational studies with primary data collection.

 We have deleted the conclusion text listed above, and have revised the description of the study outcomes as safety outcomes rather than as SAEs to avoid misrepresenting the nature of data collected in the publications reviewed.

MINOR COMMENTS

In the introduction section, it is stated that the aim of the study was "to describe the scope and frequency of SAEs among realworld users of PAXLOVID comparators". What do the authors mean with "scope"?

This section of text has been rephrased as  “to characterize the occurrence of safety events of interest among real-world users of products with similar pharmacological properties to PAXLOVID”.

Methods: Include detailed time point (day) when was the search performed

The date of the search has been added to the methods section.

Round 2

Reviewer 2 Report

Comments and Suggestions for Authors

The overall quality of the manuscript has been improved by a more focused and careful wording. However, I am still confused about the aim(s) of this analysis. As stated by the authors, safety outcomes of interest observed for products with similar pharmacological properties to PAXLOVID were evaluated in the post-marketing care setting as proxies for comparable safety outcomes among PAXLOVID users. In addition, the aim was to acquire "proxy information on the safety characteristics of interest" (i.e., of PAXLOVID). 

It seems that the intention was to use event frequencies/rates of safety outcomes observed during real-world use of these similar products as some kind of background incidence, i.e., an estimation of what is "usually observed" with these type of drugs (?). However, this seems not possible due to the rather special indication of use of Paxlovid (COVID-19). If one (for instance) refers to the frequencies/rates of cardiovascular events, renal failure, hospitalisations etc. in users of antibiotics, it seems medically implausible to indirectly "compare" these rates to what is later observed in real-world patients treated with Paxlovid, due to the completely different indications of use. I am thus still struggling with the overall message of the manuscript - as the main problem is not the way the identified studies are reporting safety outcomes of interest, but the fact that these studies are in principle not useful to understand/interpret/predict safety event reporting for Paxlovid. But this was clear already before the literature review.