1. Introduction
The COVID-19 public health emergency of international concern ended in May 2023 [
1], after originally being declared by the World Health Organization (WHO) in January 2020 [
2]. Despite this, COVID-19 remains a global health threat as it still places a burden on healthcare systems and individuals whose immunity has decreased with time [
3]. As of June 2024, 776,007,137 cases and 7,059,612 deaths have been reported worldwide since the beginning of the COVID-19 pandemic [
4]. Considering that approximately 5,700,000 deaths occurred in the first two years of the pandemic [
4], we can see that the trend has declined considerably, largely due to both the passive and active immunization of the population.
1.1. COVID-19 Vaccines
The first COVID-19 vaccine authorized by the European Medicines Agency (EMA) was the Comirnaty© mRNA vaccine marketed by Pfizer-BioNTech [
5], which received approval on 21 December 2020. Shortly after, three more vaccines were approved: the Spikevax© mRNA vaccine by Moderna [
6], the Vaxzevria© adenovirus vaccine by AstraZeneca [
7] and the Jcovden© adenovirus vaccine by Janssen [
8]. The Nuvaxovid© vaccine by Novavax [
9] followed on 20 December 2021, and the Bimervax© vaccine by HIPRA [
10] on 30 March 2023. However, marketing authorization for Jcovden and Vaxzevria was later withdrawn at the request of their manufacturers.
As of 31 December 2023 (latest WHO report cut-off date available to date), approximately 13,650,000,000 vaccine doses had been administered worldwide, with roughly 67% of the population having received a complete primary vaccination series [
11]. While reinfections occur, particularly during winter months, real-world data demonstrate the vaccines’ effectiveness in reducing hospitalizations and improving overall survival [
12,
13]. However, the clinical benefit of all drugs is always contingent on the occurrence of adverse drug effects (AEs), and COVID-19 vaccines are no exception.
COVID-19 Vaccines’ Safety Profile
Although the safety profile of COVID-19 vaccines is generally acceptable, this does not mean that they are risk-free. In the pivotal clinical trials of the first four marketed vaccines (Comirnaty©, Vaxzevria©, Jcovden© and Spikevax©), which included approximately 130,000 subjects in total, some adverse events (AEs) were reported more frequently in the experimental groups compared to placebo, such as pain at the injection site, redness and swelling, headache, fatigue, myalgia and arthralgia, malaise, nausea, vomiting, lymphadenopathy and fever [
14,
15,
16,
17]. These AEs are mild and commonly associated with the majority of vaccines currently on the market.
Regarding serious adverse events (SAEs), no significant differences were observed in the overall percentage reported between the experimental groups and the placebo group. Nonetheless, if the reported SAEs are studied individually, some imbalances emerge for certain preferred terms for the system organ class (SOC) “nervous system”. Specifically, certain neurological events were reported more frequently with some COVID-19 vaccines, including peripheral facial paralysis, transverse myelitis, multiple sclerosis, Guillain–Barré syndrome and seizures. Additionally, imbalances were noted for other SAEs like venous thromboembolism and embolic stroke [
14,
15,
16,
17]. It should be noted that the results of these safety analyses performed prior to the commercialization of the vaccines included data from patients with an average follow-up of 2 months after having received the complete vaccination regimen. Consequently, these pre-commercialization studies may not have detected adverse effects with longer latency times.
As wider vaccination campaigns progressed, post-marketing surveillance revealed additional adverse effects, further confirming the link between specific vaccines and certain rare events initially identified in clinical trials. Some noteworthy adverse events requiring close monitoring include the following: anaphylaxis [
18], myocarditis and pericarditis (both have been observed in very rare cases—less than 1 in 10,000 people vaccinated—with Comirnaty©, Spikevax©, Jcovden© and Nuvaxovid©, usually within 14 days of the second vaccination in younger males) [
19,
20], thrombosis with thrombocytopenia syndrome (TTS) (around 1000 cases of TTS have been reported following vaccination with Vaxzevria© and Jcovden©) [
21] and Guillain–Barré syndrome (it has been reported as a very rare side effect—less than 1 case in 10,000 people vaccinated—with the COVID-19 vaccines Jcovden© and Vaxzevria©) [
12]. Even after nearly four years of widespread use, the relationship between COVID-19 vaccines and certain adverse events, such as transverse myelitis, aseptic meningitis and peripheral facial paralysis, remains under investigation. This underscores the critical importance of continuous pharmacovigilance efforts to ensure vaccine safety and inform public health recommendations.
1.2. Vaccine Pharmacovigilance
Pharmacovigilance, the systematic monitoring of adverse events following exposure to pharmaceutical products, is a critical component in ensuring the safety and efficacy of drugs, vaccines and other medicinal products [
22]. While randomized clinical trials are widely considered the gold standard for evaluating the safety of novel pharmaceuticals, their design often includes small, homogeneous populations observed over short duration periods, limiting their capacity to detect a broader spectrum of AEs. For this reason, post-market surveillance plays a crucial role in detecting and reporting suspected AEs encountered in routine clinical practice [
23].
Different approaches have been proposed to effectively detect and report AEs. One widely used approach involves causality algorithms, which offer a straightforward and consistent method for assessing AEs. However, their predictive value varies; some algorithms are tailored to specific nosological entities, while others employ a one-size-fits-all approach that can overly limit therapeutic options [
24].
Another strategy involves quantifying reporting rates from both healthcare providers and patients [
25]. These individual reports are crucial because they provide invaluable real-world data, enabling the identification of rare or delayed AEs and ultimately contributing to a more comprehensive understanding of medication safety. This is particularly true for large-scale databases like EudraVigilance (European Union Drug Regulating Authorities Pharmacovigilance), which plays a vital role in pharmacovigilance at the European level.
1.3. EudraVigilance
EudraVigilance is the system for managing and analyzing information on suspected adverse effects of medicines that have been authorized or are being studied in clinical trials in the European Economic Area [
26]. The information collected is compiled in an electronic database of public access, which enhances the safe and effective use of medicines by enabling the following:
The electronic sharing of individual case safety reports (ICSRs) among the EMA, national competent authorities, marketing authorization holders and clinical trial sponsors within the European Economic Area.
The early identification and assessment of potential safety signals.
Improved product information for medicines approved in the European Economic Area.
The main objective of EudraVigilance is the early detection of possible safety signals from marketed drugs for human use through continuous monitoring and evaluation of potential safety issues in relation to reported adverse effects. The European Medicines Agency (EMA) and national competent authorities are responsible for regularly reviewing and analyzing EudraVigilance data to detect safety signals. The Pharmacovigilance Risk Assessment Committee evaluates the safety signals detected in EudraVigilance and may recommend regulatory action as a result. EMA also publishes annual reports to provide a summary of the EudraVigilance-related activities the Agency undertakes within the European medicines regulatory network and with stakeholders [
26]. EudraVigilance data are publicly accessible, enabling the conduct of observational studies, including disproportionality analysis, to investigate potential links between adverse effects and specific medications.
1.4. Disproportionallity Analyses
Disproportionality analyses are observational studies that constitute valid methods for the generation of hypotheses aimed at detecting “safety signals”. Safety signals are possible existing relationships between a drug or medical device and an adverse effect generated through the analysis of information extracted from databases containing reports of possible adverse events. By quantifying the degree to which a drug–adverse effect pair occurs disproportionately (i.e., with a higher frequency than would be expected if there was no relationship between that drug and the development of the AE under study), it may be hypothesized that the adverse effect is indeed causally related to the study drug [
27].
It should be noted that the real number of patients exposed to the drug being studied is unknown, since only those cases that have been reported in the database as possible adverse effects are taken into account. Therefore, disproportionality analyses cannot quantify or estimate the real risk between a drug and an AE, and they do not allow causality to be inferred, their sole purpose being to provide a knowledge base on which to subsequently develop analyses with more robust methodologies that do allow the generated hypothesis to be further investigated [
28].
One of the statistical methods used to try to investigate the possible relation between an adverse effect and a drug in disproportionality studies is the Proportional Reporting Ratio (PRR), which is the method used in the present study.
1.5. Rationale and Objectives of This Study
Since the commercialization of COVID-19 vaccines, several cases of various neurological entities in patients with no relevant medical history exposed to different COVID-19 vaccines have been published in the literature [
29,
30,
31,
32]. Similarly, in September 2021, a case of aseptic meningitis after the administration of the first dose of Comirnaty© was detected through the Hospital Pharmacovigilance Program by Laboratory Signals (PFVHSL for its acronym in Spanish) and reported by our Pharmacovigilance Unit in the La Paz University Hospital in Madrid, Spain [
33]. In the immediate aftermath of the COVID-19 outbreak, as the SARS-CoV-2 vaccine rollout reached its peak, pharmacovigilance schemes garnered heightened significance due to concerns from healthcare professionals and public skepticism. This led to an unprecedented increase in both the detection and reporting of vaccine-related AEs in an effort to determine whether the benefits outweighed the risks, ultimately shaping public healthcare policy [
34].
The objectives of this study are two-fold:
Firstly, we evaluated potential safety signals linking COVID-19 vaccine administration to several neurological adverse effects (aseptic meningitis, Guillain–Barré syndrome, polyradiculoneuropathies, multiple sclerosis, transverse myelitis and neuromyelitis optica) using disproportionality analysis of available EudraVigilance reports. This analysis aims to identify signals warranting further investigation with robust methodologies.
Secondarily, given the high proportion of adverse events reports from non-healthcare professionals, particularly during the initial phases of vaccine rollout, we evaluated the influence of the reporting source on disproportionality study results. We hypothesize that reports from non-health professionals, often concerning mild adverse events, may dilute the signal for serious and specific AEs by increasing the denominator in PRR calculations. This analysis will help evaluate the potential impact of the reporting source on pharmacovigilance data interpretation.
2. Results
A total of 4,159,820 reports for COVID-19 vaccines and 114,025 for the 47 comparator vaccines were compiled. A decline in the number of reports submitted to EudraVigilance can be observed over the years, with a particularly notable drop in 2023. A total of 4,159,820 reports for COVID-19 vaccines and 114,025 for the 47 comparator vaccines were compiled. In 2021 1,825,990 reports for COVID-19 vaccines and 35,375 reports for the comparator vaccines were registered; whereas in 2022, the numbers were 1,804,279 and 30,654, respectively, and in 2023, they were 529,551 and 47,996, respectively (
Figure 1).
Of the reports collected for the control vaccines: 71, 20, 95, 5 and 25 duplicate cases were excluded.
Figure 1 shows the flowchart of neurological AEs reported following COVID-19 vaccinations and comparator vaccinations in EudraVigilance.
The percentage of AE reports submitted by non-healthcare professionals was consistently higher for COVID-19 vaccines compared to control vaccines. In 2021, 57.33% of COVID-19 vaccine reports were from non-healthcare professionals, compared to 33% for control vaccines (
p < 0.001). This trend continued in 2022 (59.4% vs. 36.5%,
p = 0.001) and 2023 (42% vs. 24.36%,
p = 0.006) (
Figure 2).
2.1. Primary Objective: Relationship Between COVID-19 Vaccines and Neurological Adverse Effects
To evaluate the possible relationship between the COVID-19 vaccines and the different neurological adverse effects under study, we calculated the PRR. We examined each vaccine separately and then combined to assess overall trends. Since EudraVigilance data are organized by calendar year, we calculated PRRs for 2021, 2022 and 2023 individually and then for the combined period from January 2021 to December 2023, allowing us to track any changes in reporting trends over time.
2.1.1. 2021
In 2021, none of the neurological adverse effects obtained a PRR ≥ 2, the threshold necessary to suggest a potential neurological safety signal. The only vaccine that obtained a PRR significantly higher than 1 was Jcovden© for polyradiculoneuropathy with a PRR of 1.52 (IC 95% 1.29–1.80). It is important to note that while Nuvaxovid© was authorized in the EU during this period, no adverse effects associated with it were reported in EudraVigilance, resulting in a PRR of 0 for all neurological conditions (
Table 1).
2.1.2. 2022
Similarly to 2021, in 2022, no vaccine achieved a PRR ≥ 2. Notably, adverse effects associated with Nuvoxavid© were reported in 2022, as shown in
Table 2, although not for all the neurological conditions under investigation.
2.1.3. 2023
In 2023, several significant safety signals (PRR ≥ 2) emerged between specific COVID-19 vaccines and neurological adverse effects. Jcovden© exceeded the significance threshold for polyradiculoneuropathy [PRR 5.4, IC 95% 3.98–7.32)], multiple sclerosis [PRR 2.72, IC 95% (1.08–6.87)], transverse myelitis [PRR 4.68, IC 95% (1.02–21.35)] and neuromyelitis optica [PRR 7.79, IC 95% (3.5–17.37)]. In addition, both Spikevax© and Comirnaty© showed significant safety signals with multiple sclerosis [PRR 2.50, IC 95% (1.70–3.68), and PRR 2.33, IC 95% (1.68–3.24), respectively] and transverse myelitis [PRR 3.50, IC 95% (1.66–7.50) and PRR 3.58, IC 95% (1.85–6.93), respectively] (
Table 3). These findings highlight a potential increased risk of these neurological events following vaccination with these specific vaccines and warrant further investigation.
2.1.4. Total Period (January 2021–December 2023)
Despite the significant signals observed in 2023, an analysis of the combined three-year dataset revealed no vaccine with a PRR exceeding the threshold of 2 (
Table 4). This is likely due to the higher number of cases reported in 2021 and 2022, which heavily influenced the overall PRR calculations, diluting the impact of the 2023 findings. Essentially, the larger volume of data from the earlier years, where no significant signals were found, pulled the overall PRR values down.
2.2. Secondary Objective: Relationship Between Percentage of Events Reported by Non-Healthcare Professionals and Proportional Reporting Ratio
Given the high proportion of adverse effects reported by non-healthcare professionals and the low PRRs observed, we investigated whether the reporting source influenced the PRR values. Specifically, we examined the relationship between the percentage of reports from non-healthcare professionals and the calculated PRRs.
To assess if reports from non-healthcare professionals might obscure potential links between vaccines and specific adverse effects, we used Spearman’s correlation coefficient (SCC; ρ). This analysis compared the percentage of non-healthcare professional reports with the case/no-case ratio (calculated by dividing the number of each specific neurological adverse event by the total number of adverse events) for each neurological adverse effect associated with both COVID-19 vaccines and control vaccines across 2021, 2022 and 2023 (
Table 5).
With the 30 paired observations, the calculation of SCC was −0.4683 (p = 0.009), which indicates a moderate negative correlation between the two variables being analyzed. This suggests that a higher percentage of reports from non-healthcare professionals tends to be associated with lower PRR values. While this relationship is not extremely strong, it is sufficient to highlight a notable trend between non-healthcare professional (non-HCP) reports and the Proportional Reporting Ratio (PRR). This finding suggests that an increased volume of non-healthcare professional reports could potentially dilute the PRR, potentially impacting the reliability of signals in pharmacovigilance systems. This emphasizes the need for careful interpretation of signals in systems with a high proportion of non-healthcare professional contributions, as these reports might differ in content or accuracy compared to those from healthcare professionals. However, this relationship is not perfectly linear, indicating that other factors likely contribute to this trend. Further investigation is needed to understand the complex interplay between the reporting source and PRR calculations.
3. Discussion
After examining the results of the PRR calculation of 4,159,820 COVID-19 vaccine reports and 114,025 comparator vaccine reports from EudraVigilance, the only vaccine with a PRR greater than 2 was Jcovden© in relation to polyradiculoneuropathy and neuromyelitis optica, and only in the analysis of the 2023 data. Notably, no significant PRR was observed for any COVID-19 vaccine across the three-year period when considering all neurological effects under study.
Since the marketing authorization of the COVID-19 vaccines, polyradiculoneuropathies such as Guillain–Barré have already been described and appear in the summary of product characteristics (SmPCs) for both Jcovden© (as seen in the 2023 results of this study) and Vaxzevria©; and despite not appearing in the SmPCs of any of the COVID-19 vaccines, cases of optic neuritis have been described in the literature in previously healthy persons vaccinated with Comirnaty©, Spikevax© and Jcovden© [
35] (as also shown in the 2023 results of the present study). However, it is striking that in this disproportionality study, no high PRRs were obtained for any of the COVID-19 vaccines for transverse myelitis, when it is an effect that has been described in the literature for almost all COVID-19 vaccines [
36] and already appears in the Jcovden© and Vaxzevria© SmPCs. These results are probably related to the notable fact that in these three years of data collection, almost 98% of the reports collected were for COVID-19 vaccines (4,159,820), and of these, 56.28% were made by non-healthcare professionals, compared to the 30.29% of reports made by non-healthcare professionals for control vaccines (
Figure 1 and
Figure 2). This phenomenon likely reflects the extensive vaccination campaigns’ widespread information dissemination regarding potential vaccination risks and heightened public health awareness during the pandemic, all contributing to an unprecedented influx of adverse event reports. This influx may have diluted the signal for specific neurological events within the large volume of reported data.
A comprehensive analysis of adverse event reports reveals a predominance of mild effects associated with COVID-19 vaccines, typically characterized by post-vaccination symptoms such as fever, injection site pain, myalgia, and headache. Conversely, reports for comparator vaccines exhibit a higher proportion of moderate to severe adverse effects. This disparity may be attributed to the lower percentage of reports submitted by non-healthcare professionals for comparator vaccines or their longer market presence, leading to a large denominator in PRR calculations and a reduced case/no-case ratio for COVID-19 vaccines. This reduction may obscure potential signals between specific adverse effects and COVID-19 vaccines. These findings are consistent with the data obtained in this study. For example, in 2023, an increase in the PRR is observed across almost all COVID-19 vaccines, coinciding with a decrease in the percentage of AEs reported by non-healthcare professionals.
The findings observed from the analysis of EudraVigilance in the present study align with trends observed in VAERS and the Yellow Card scheme. Both systems have reported a predominance of mild adverse effects for COVID-19 vaccines, such as fever, injection site pain and headache [
37,
38]. Both systems have observed that reports for comparator vaccines tend to include a higher proportion of moderate to severe adverse effects. This may be influenced by factors such as the longer market presence of comparator vaccines and differences in reporting behaviors among healthcare professionals and the general public [
39,
40].
When examining the trends in both the total number and percentage of AEs reported by non-healthcare professionals and comparing them with the evolution of the PRRs, a certain signal is observed. This signal is further supported by the calculated Spearman’s correlation coefficient, which, with a result of −0.4683 and a p-value of 0.009, indicates an inverse relationship between these two variables. Nonetheless, it is important to remember that correlation does not imply causation; thus, while the two variables are related, further investigation using additional statistical tests or data exploration is necessary to understand the underlying dynamics and potential causal factors.
Strengths and Limitations
The present disproportionality study has a robust information base, as the reports that have been collected for the analysis come from EudraVigilance, which has a very large number of reports from all countries in the European Economic Area. Therefore, its strength comes from its ability to harness a vast amount of data from over a million reports, which enhances its statistical power and robustness. This extensive dataset can allow us to identify potential safety signals and patterns that may not be evident in smaller studies. Additionally, the large sample size employed increases the likelihood of capturing rare adverse events, providing a more comprehensive understanding of the relationship between medications and reported outcomes.
On the other hand, while leveraging a wide dataset of over a million reports and thus possessing great statistical power, this study comes with inherent limitations characteristic of observational studies. Despite its ability to identify potential signals between variables, it cannot establish causality; observed correlations may be influenced by confounding factors or biases inherent in the reporting system. Additionally, the analysis relies heavily on the accuracy and completeness of the data collected, which can vary significantly across reports. Variability in reporting practices, differences in the population studied, and the potential for underreporting or overreporting of certain events can further skew results, which has possibly occurred given the reporting differences between COVID-19 vaccines and the rest of the vaccines in the studied period. This suggests that reports made by non-health professionals are not as accurate as those made by health professionals, who also tend to report more complicated and therefore more serious nosological entities. Moreover, another limitation of using a database such as EudraVigilance is that a unique report may contain multiple preferred terms describing the same AE. The search would therefore return several results for the same adverse event reported, resulting in an overestimation of some adverse events. There is no simple way to solve this problem, since, as explained above, the sheer number of reports makes it impossible to discard possibly duplicate reports. Consequently, while the findings can offer valuable insights, they should be interpreted with caution and complemented by further research to validate the signals identified.
5. Conclusions
While a PRR exceeding 2 was observed only for Jcovden©, Spikevax© and Comirnaty© in 2023, with no vaccines exceeding this threshold in the combined three-year analysis, this may be attributed to the high proportion of AE reports submitted by non-healthcare professionals for COVID-19 vaccines in 2021 and 2022. This is supported by the observed inverse correlation between the percentage of reports from non-healthcare professionals and the case/no-case ratio. Future replications with more homogeneous data may yield more accurate results.
This finding underscores the critical importance of high-quality, homogeneous data for robust PRR estimation, particularly for rare adverse events such as neurological events. Moreover, despite this study’s obvious limitations, its low-cost and straightforward methodology makes it very cost-effective, enabling researchers to quickly analyze trends and generate hypotheses for further investigation.
Ultimately, this study emphasizes the need for continuous improvement in data quality within pharmacovigilance systems, like Eudravigilance. By enhancing data quality and addressing potential biases, disproportionality analyses can serve as a valuable first step in identifying potential safety signals and guiding more in-depth research to optimize public health outcomes.