Conceptual and Methodological Barriers to Understanding Long COVID

Abstract

This article focuses on individuals with Long COVID after contracting SARS-CoV-2. Although some patients have complications such as diabetes mellitus or tissue damage to the heart or lungs as a result of SARS-CoV-2 infection, our research focuses on individuals who have persistent symptoms that are not consistent with major organ dysfunction. The current article reviews methodological and conceptual issues that need to be considered in the development and use of a case definition for Long COVID and discusses the significance of appropriately phrasing questions assessing symptoms, specifying thresholds for when to count a symptom, determining how many symptoms should be required for a diagnosis, assessing symptoms over time, using multiple assessment modalities, and differentiating symptomatology from functionality. Dealing with these issues, particularly triangulating data from multiple sources, allows investigators to develop a more reliable and valid way to assess Long COVID.

1. Conceptual and Methodological Barriers to Understanding Long COVID

An estimated 95% of persons aged ≥16 years had SARS-CoV-2 antibodies from previous infection or vaccination by 2022 [1]. By Feb. of 2024, 17.1 to 18.2% of U.S. adults reported having experienced Long COVID [2]. Those with a more severe initial infection have a higher risk of developing Long COVID, but Long COVID can develop even after a mild infection [3]. Risk factors for Long COVID include SARS-CoV-2 viremia, Epstein–Barr viremia, specific autoantibodies, type II diabetes, obesity, elevated blood pressure, chronic lung disease, and depression [4]. However, there is little agreement on how to assess, diagnose, and treat people with persisting symptoms of Long COVID. There is not even a consensus on what to call this recently recognized illness [5] as some refer to it as post-COVID-19 condition or Postacute Sequelae of SARS-CoV-2 Infection (PASC).

Long COVID complications present a broad and inconsistent spectrum of symptoms, and challenges in defining Long COVID include the timeframe of symptom onset or presentation. As one example, the World Health Organization [6] developed a case definition for what they refer to as “post-COVID-19 condition”. Their definition states that it “occurs in individuals with a history of probable or confirmed SARS-CoV-2 infection, usually 3 months from the onset of COVID-19 with symptoms that last for at least 2 months and cannot be explained by an alternative diagnosis”. However, the scientific rationale for this length of time and duration is unclear.

Others have focused on defining a specific post-viral symptom, such as fatigue. Sandler et al. [7] defined post-COVID fatigue in the following way: “a dominant symptom; chronic; disabling to an extent that it interrupts all or a majority of normal activities (such as work/school attendance, social activities, etc.); persistent for 6 months or more (3 months in children/adolescents); and emerged during confirmed acute COVID-19 (i.e., with a positive severe acute respiratory syndrome coronavirus 2 [SARS-CoV-2] test), without symptom-free interval since onset”. Such a definition, while specific, focuses just on one of potentially hundreds of possible symptoms and does not address the issue of some symptoms emerging days or months after the infection as well as the fluctuating nature of symptoms and the issue involving co-mingling of functional limitations, which will be addressed later in this article.

There are currently multiple definitions that have been proposed for Long COVID and there is a high priority to reach a consensus on a case definition [8]. Case definitions are crucial in science, especially for syndromic illnesses that lack a diagnostic test, as they allow a diagnosis for the syndrome so that scientists can study those with the syndrome versus those without. If difficulties occur in arriving at a reliable case definition, complications emerge in estimating prevalence, discovering biomarkers, and evaluating treatment approaches. This has occurred for the post-infectious illness that is known as Myalgic Encephalomyelitis/Chronic Fatigue Syndrome (ME/CFS). When criterion variance occurs, patient heterogeneity makes it difficult to identify biomarkers, and healthcare workers have difficulty distinguishing these illnesses from solely psychiatric reasons [9].

Jason and Islam [10] recently provided the rationale for a five-axis system for diagnosing Long COVID (see Supplementary Materials). Axis 1 delineates the designation of COVID variations, such as Omicron, Beta, Gamma, Delta, or Alpha, as well as the supporting evidence (positive antigen results, positive PCR tests, etc.). The amount of time since infection is included in Axis 2. Collateral damage to various organ systems (respiratory, neurological system, metabolic, cardiovascular, gastrointestinal, etc.) is the subject of Axis 3. Patients with Long COVID fall into two categories: those who have lung or other organ damage (e.g., fibrosis and subsequent pulmonary vascular damage due to the acute respiratory infection or post-intensive care syndrome), and those without known organ damage. Functional impairment, classified as mild, moderate, or severe, is the focus of Axis 4. Patients can be categorized into meaningful groups with the use of straightforward rating scales, and the functional distinctions within these groups have a significant impact on the services that they require. COVID-19-generated symptoms are identified by Axis 5, and each symptom is assigned a frequency and severity rating. Furthermore, using psychometrically validated questionnaires is essential to increase the likelihood that the symptoms will be evoked similarly by different investigators, thus leading to fewer issues with data interpretation and comparison.

Over the past few years, we have used this designated five-axis system to classify patients with Long COVID, and some of the lessons we have learned are laid out below, particularly in terms of phrasing questions, specifying thresholds for when to count a symptom as significant, determining how many symptoms are required for a diagnosis, assessing symptoms over time, using multiple modalities to understand this complex disease, and differentiating symptomatology from functionality.

Our current study involves comparing a group of college-age students who had contracted SARS-CoV-2, half of whom recovered and half of whom did not. We then matched these young adults with another sample of individuals who had developed Infectious Mononucleosis, of which half had recovered and half had not. Below, we discuss issues involving the identification of SARS-CoV-2-infected youth who were provided a screener, a more intensive structured survey, a medical examination, and a qualitative interview. Our study aims to determine immunologic and metabolomic risk factors for the development of Long COVID following SARS-CoV-2 infection and ME/CFS following Infectious Mononucleosis. In contrast to healthy controls, we hypothesize that altered host immune and metabolomic profiles are associated with post-viral fatigue following primary Epstein–Barr virus and SARS-CoV-2 infection. Our qualitative in-depth interviews explore perceptions of major life changes, fluctuations in student and employment status, changes in somatic symptoms, changes in coping strategies, stigma experienced, and the evolution of support systems, to determine which, if any, of these factors are also important for the development and maintenance of Long COVID and ME/CFS.

2. Phrasing Questions

Structured interview schedules ensure that researchers and clinicians obtain the same diagnostic data from interviews, and its implementation minimizes differences in the way clinical data are obtained [11]. Most Long COVID researchers have identified symptoms using questionnaires with unclear psychometric qualities. The process of developing surveys and questions has also received little attention from researchers. Our group has learned several lessons in crafting questions to assess Long COVID. For example, it is important to carefully determine whether patients understand whether their current symptoms are due to SARS-CoV-2 infection. Our initial screener asked patients: “Do you currently have ongoing symptoms that manifested since you contracted COVID-19?” Although this seems to be an obvious question to ask to determine whether individuals should be screened into a Long COVID study, we found that some patients who initially answered “no” to this question later changed their answer after we asked them about different current symptoms which helped them make a connection between current symptoms and their exposure to SARS-CoV-2. For example, during the screening, a respondent answered “no” to having ongoing symptoms following SARS-CoV-2, but, during the medical examination, the student mentioned he had both fatigue and muscle pain, and, when asked questions about specific symptoms since being infected with SARS-CoV-2, he also recalled an episode of brain fog that lasted 3 days and trouble focusing in class and remembering what was said. He also experienced post-exertional malaise for more than 24 hours. Had we used his response to the first screening question to make a diagnosis, it would have been inaccurate, as this person did have Long COVID.

It is also important to be careful about the time frame of symptoms. To give an example, during the administration of a structured questionnaire, we asked participants, “How certain are you that your symptoms have been due to COVID-19”? Respondents had the following choice points: “Not at all certain”, “A little bit certain”, “Somewhat certain”, “Very certain”, and “Extremely certain”. We realized that some patients were thinking about the symptoms that occurred after contracting SARS-CoV-2, whereas other patients were thinking about the current time frame. It was only by altering the question from “your symptoms” to “currently experienced symptoms” that we were able to gather information about current symptomatology rather than what had occurred during their acute initial infection.

Asking patients whether they have Long COVID or ME/CFS can be unreliable. In a study of patients with Long COVID, Jason and Dorri [12] found just 71% of respondents who reported having ME/CFS had met the criteria for ME/CFS, whereas 40% of those who claimed not to have ME/CFS satisfied the criteria. Stated differently, a large number of people with Long COVID who think they also have ME/CFS do not fit the case criteria for ME/CFS, and a further group with Long COVID do not know that they meet the criteria for ME/CFS. Furthermore, it has been found that ninety to ninety-five percent of individuals who meet the criteria for ME/CFS are unaware that they have the condition, according to evidence from adult and pediatric community-based epidemiology studies [13,14]. Based on these findings and experiences from the current ongoing study, due to participants not knowing the symptoms of ME/CFS or Long COVID, it is not sufficient to just ask patients whether they have ME/CFS or Long COVID, as most do not know what symptoms are in the established ME/CFS and Long COVID case definitions. In addition, the fact that many medical personnel are not acquainted with the case criteria for ME/CFS or Long COVID complicates their ability to diagnose patients accurately.

3. Specifying Thresholds for Symptoms to Be Counted

Numerous somatic symptoms are prevalent in community and clinic samples, but they often occur at low levels, not meeting a minimum threshold of what might be considered a burden [15], so counting them as symptoms can result in introducing trivial symptoms into Long COVID symptom counts. This becomes even more problematic when considering that most existing Long COVID studies have focused on the occurrence of individual Long COVID symptoms [16]. More confusion occurs when occurrence measures are referred to as frequency measures. A survey might ask respondents whether they experience unrefreshing sleep, and perhaps 35% of the sample responds affirmatively. This is not a frequency-of-time measure but an occurrence measure, as it only assesses the “yes/no” occurrence of unrefreshing sleep in the sample. If each respondent was asked the percent of time unrefreshing sleep occurred, such as if it occurred a little of the time, half the time, most of the time, or all of the time, that is a frequency measurement. If a problem such as headaches occurs frequently, but is mild, it should probably not be considered significant or a burden and not be counted as part of a Long COVID case definition. Similarly, if a severe migraine only occurs once every few months, it also should not meet the threshold for a symptom of Long COVID. Many participants in our study who had been exposed to SARS-CoV-2 did have some persisting symptoms at the frequency of “a little” of the time and of “mild” severity. When participants only had these types of low-frequency and -severity ratings, they were not considered to have Long COVID; however, had we just asked about the occurrence of these infrequent and mild symptoms, such symptoms would have been counted toward being diagnosed with Long COVID. Using both frequency and severity ratings has helped differentiate Long COVID from other related illnesses. For instance, Oliveira et al. [17] discovered that the frequency and severity evaluations of “unrefreshing sleep” and “flu-like symptoms” were the best discriminators of ME/CFS versus Long COVID.

While severity and frequency scores help separate patients from controls, it is also critical to specify thresholds. Watson and others [18] dynamically adjusted the criterion for symptoms by the frequency and severity ratings recorded in a sample of patients with ME/CFS and controls. Patients with ME/CFS were consistently discriminated from controls using a framework that included a frequency of “half the time” and a severity of “moderate” [19]. We have used such thresholds in our work on our current study of determining whether symptoms meet the burden threshold for being counted for a diagnosis of Long COVID.

4. How Many Symptoms for a Diagnosis

Another critical issue is determining the number of symptoms needed to diagnose Long COVID. One of our participants following SARS-CoV-2 infection had fatigue at a frequency of most of the time, and a severity of moderate. This symptom was significant and a burden; yet, as this was her only symptom, she would not have been given a Long COVID diagnosis if multiple symptoms were required. Most criteria imply or state that multiple symptoms are required to be present. For example, in a recent Long COVID case definition developed by the RECOVER group [20], of their 12 symptoms that differentiated those who were infected versus those not infected, there was a requirement that a score of 12 or greater was needed to be classified as Long COVID-positive. A person with just a high frequency and high severity score for fatigue would not be diagnosed with Long COVID. Even someone experiencing severe palpitations, dizziness, and gastrointestinal symptoms would only receive a score of 4 on their diagnostic system, not 12, meaning they would not fulfill the requirements for Long COVID. Also of interest, ninety-three people in their sample who had symptom ratings below twelve (therefore not meeting their Long COVID diagnosis) indicated they were in fair or poor physical condition overall.

One severe symptom can disable a person, and diagnostic systems should allow such individuals to be included within a Long COVID classification system. In our work with Long COVID patients, we have found that even one symptom that is severe or very severe and occurs most of the time or all the time should make the person eligible for a Long COVID diagnosis. Developing criteria that suggest that scientists exclude severely impacted patients from meeting Long COVID criteria can have significant negative patient and healthcare consequences.

5. Multiple Assessments over Time

After infection, the proportion of adults and adolescents who meet the case definition for post-viral illnesses will decline over time. For instance, ME/CFS decreased from 35% at six weeks to 27% at three months, 12% at six months, and 9% at twelve months in a prospective study of adult patients with acute infections with Epstein–Barr virus (glandular fever), Coxiella burnetii (Q fever), or Ross River virus (epidemic polyarthritis) [21]. When adolescents with infectious mononucleosis were studied by Katz et al. [22], the percentage of adolescents who fulfilled the criteria for ME/CFS likewise declined with time, from 13% at 6 months to 7% at 12 months to 4% at 24 months. Although there are contradictory estimates of the progression of Long COVID over time, COVID symptoms have also been found to diminish progressively over time [17,23,24]. If longitudinal samples include individuals with symptoms of 6 months or less, more of them will likely improve than those who have several years of illness duration, and this time factor needs to be considered in research with post-viral illnesses.

In addition, richer and more accurate characterizations of the patient’s illness might occur if the research involves patients with data collected from more than just one assessment time. As our data collection involved both a screen, a survey, a qualitative interview, and a medical examination at different time points, we were able to utilize multiple vantage points for triangulating data from the participants, and we believe this resulted in more valid characterizations of the respondents’ symptoms. For most cases, all data collected were consistent across time regarding their Long COVID status, but, in some cases, data collected at one time point varied with what was found at another time point. By collecting data over time, we were able to make judgments of how individuals might have interpreted the questions differently, so we could be more confident in making the best decisions about their Long COVID status.

6. Multiple Assessment Modalities

With complex diseases such as Long COVID, there are many benefits to employing a multidisciplinary team of investigators, including from fields such as medicine, clinical psychology, computer science, immunology, infectious diseases, metabolomics, neurology, and statistics. This type of collaboration allowed us to view data using multiple methods including network analysis, data mining, random forest supervised machine learning, logistic regression, receiver operating characteristic curves, and general systems theory. We collected data from a comprehensive physician examination that included an assessment of orthostatic intolerance (postural orthostatic tachycardia syndrome or orthostatic hypotension). Our blood samples allowed for the testing of plasma cytokines, lymphocyte counts, natural killer cell count and activity, T-cell subsets, autoantibodies, and metabolomic analysis. We also used a severity of mononucleosis scale to assess the severity of Infectious Mononucleosis in our ME/CFS study. Family history data allowed us to determine if genetic factors might play a role in etiology. We also used validated self-report questionnaires to measure symptoms, physical limitations, and psychological variables.

Patients were also able to provide information about the contextual issues surrounding their symptoms with the use of qualitative data. As an example, for one young adult, data collected in the screening, survey, and medical examination were unclear as to whether her symptoms were independent of her SARS-CoV-2 infection; yet, during the qualitative interview, we learned that the respondent felt that her symptoms were caused by this infection.

7. Differentiating Symptomatology from Functionality

Several investigators [7] have tried to define a Long COVID symptom as meeting certain functional criteria involving disability and the interruption of normal activities. The World Health Organization’s [6] definition of the post-COVID-19 condition is that symptoms “generally have an impact on everyday functioning”. The U.S. Department of Health and Human Services [25] defines Long COVID as “multisystemic; and may present with a relapsing–remitting pattern and progression or worsen over time, with the possibility of severe and life-threatening events even months or years after infection”.

This trend to co-mingle symptoms and functional status in post-viral case definitions is problematic, as symptoms and the physical limitations of the illness should be independent constructs and a pre-determined reduction in functioning should not be required to have Long COVID. In other words, some patients have multiple frequent and severe symptoms but minimal limitations in their daily living but they may still have Long COVID. To summarize, many patients with Long COVID have annoying and persisting symptoms but still have the energy to engage in all their pre-COVID activities.

Other physical illnesses do not require patients to document impairments in previous levels of functioning to be diagnosed. In contrast, mental disorders often require a substantial reduction in functioning in their case definitions [26]. It is of interest that most ME/CFS case definitions require patients to have a substantial reduction in functioning [27]. Just having high levels of symptoms is not in itself enough to receive an ME/CFS diagnosis, and this co-mingling of symptoms and functionality has unintentionally contributed to delegitimizing their symptoms, as significant symptoms are not enough to be diagnosed. Therefore, the application of substantial limitations to Long COVID diagnostic efforts may create conceptual confusion regarding the case definition and potential stigma for patients. The five-axis rating of symptoms reviewed in the introduction keeps as two separate domains the frequency/severity of symptoms and functional limitations. It is possible to create scales that measure functional limitations imposed by symptoms [28], but it is important to not co-mingle these two domains in the Long COVID case definition.

8. Discussion

As mentioned in this article, many Long COVID questionnaires measure just the occurrence of somatic symptoms. In addition, it is not uncommon for studies to have both occurrence measures for some symptoms, and frequency or severity measures for others [20]. In addition, most investigations have not used psychometrically sound questionnaires so it is more likely that the symptoms will not be elicited reliably by different investigators, thus reducing potential problems for interpreting and comparing the data.

Some of the problems that have been reviewed in this article can be observed in a recent influential RECOVER article by Thaweethai et al. [20], who proposed a new method for diagnosing Long COVID. They identified 12 symptoms that differentiated those who were infected versus not infected with SARS-CoV-2. A symptom score was provided for each symptom, and individuals who scored 12 or greater were classified as Long COVID-positive. The occurrence of smell/taste was the best discriminator and was associated with 8 points. Therefore, if a respondent had this symptom, it counted for 8 of the needed 12 points to meet the criteria for Long COVID. In other words, the occurrence item “Loss of or change in smell or taste” was the most important symptom, accounting for 8/12 or 66% of the variance identifying Long COVID individuals. Their study found that, among those who were ultimately classified as Long COVID-positive, 41% had this symptom. However, Dorri and Jason [29] looked at the same item, “loss of or change in smell and/or taste,” in another Long COVID data set utilizing frequency and severity techniques to quantify this symptom instead of merely rating occurrence. Dorri et al. [29] found just 12.6% of patients satisfied the requirements for this symptom, using standard ME/CFS criteria, which require symptoms to occur at least half the time and to have a severity of at least moderate. Clear frequency and severity criteria caused this important symptom to drop from 41% in the RECOVER study to just 12.6% in Dorri et al. [29] study. Additional evidence is also available on this issue. While, at the beginning of the pandemic, smell and taste loss were considered common symptoms of COVID-19, by analyzing data from more than 7 million COVID-19 patients across the US, Reiter et al. [30] found the risk of losing your sense of smell and taste from the most recent COVID-19 Omicron variants was only 6–7% of what it was during the early stages of the pandemic, so these symptoms are no longer common symptoms of infection.

Another example from the Long COVID criterion studied by Thaweethai et al. [20] of “sleep disturbance” can be instructive. Participants who reported that they were currently experiencing any of the following symptoms on the symptom questionnaire were asked to complete an eight-question survey that served as their criteria for “sleep disturbance”: “Stopping breathing during sleep or sleep problems (such as snoring, trouble falling asleep, nighttime awakenings, or trouble staying awake during the day) 3 or more times a week”. Only 32% of those who were identified as Long COVID-positive had symptoms related to sleep disruption, so it was not counted as a key symptom of Long COVID. When a more accurate term “unrefreshing sleep” was used in another study [29], 78% of Long COVID participants satisfied the requirements of occurring for at least half the time and of at least moderate severity. Two other studies using this phrasing yielded similar results [17,31]. The term “unrefreshing sleep” has been used for decades in ME/CFS research, and, had it been used in the Thaweethai et al. [20] study along with frequency and severity data, unrefreshing sleep might have been selected by these investigators as one of the key symptoms differentiating those from those who were not infected. In summary, our understanding of critical Long COVID symptoms is affected by whether imprecise occurrence measures are employed or more specific frequency and severity measures and thresholds are used, as well as the care in phrasing the symptoms.

How questions are asked is worth examining in more detail in another area of post-viral research. In the 1990s, researchers found that the somatic symptoms included in the ME/CFS criteria were highly prevalent in the general population [32], which led to the conclusion that occurrence measures in the ME/CFS field were too imprecise. In response, researchers shifted to evaluating how frequently the symptoms in the ME/CFS case criteria occurred. However, research found that patients with Major Depressive Disorder experienced fatigue just as frequently as those with ME/CFS. Stated differently, it is not possible to distinguish levels of fatigue between ME/CFS and Major Depressive Disorder with just frequency measures. However, when measures of severity were introduced, those with ME/CFS had significantly higher levels of fatigue severity than patients with Major Depressive Disorder [33], and, with other frequency and severity symptom measures, it was possible to differentiate between ME/CFS and Major Depressive Disorder with a 100% accuracy [34]. Therefore, by assessing frequency and severity, it was possible to make this important diagnostic differentiation between ME/CFS and a psychiatric condition.

As indicated in this article, another serious diagnostic issue involves requiring functional limitations for a diagnosis of Long COVID. Physical illnesses do not require patients to demonstrate impairments in levels of functioning to be diagnosed. However, the Diagnostic and Statistical Manual of Mental Disorders primarily uses a substantial reduction criterion for diagnosing many mental disorders [35]. When applied to a diagnosis of ME/CFS, it requires patients to have functional impairment [36,37], and this substantial reduction criterion has been stigmatizing to individuals with ME/CFS [27]. In the same way, requiring patients with Long COVID to evidence significant declines in functioning may unwittingly stigmatize them. Individuals who continue to experience SARS-CoV-2 symptoms should have their functional limits independently evaluated, as some will have functional limitations and others will not.

In our current study, we have found the usefulness of the five-axis system for defining Long COVID [10]. As previously mentioned, Axis 1 comprises identifying the COVID-19 infection variation and the kind of infection record. The time passed from infection, or the time elapsed since being sick or affected would be included in Axis 2. Individuals who have experienced symptoms for fewer than 28 days may be diagnosed with Acute COVID-19. Individuals who continue to exhibit symptoms for more than four weeks may be considered for a Long COVID diagnosis. The nature of medical collateral damage to different organs is the focus of Axis 3. Functional impairment is the fourth axis and patients can be classified as mild, moderate, or severe. Axis 5 comprises the identified symptoms, which can be obtained from surveys employing validated questionnaires (see Supplementary Materials).

Our article suggests that the diagnosis of Long COVID is more complicated than asking patients if they have this disease, and there are validity issues involved with asking a long set of occurrence-of-symptom questions at one time point. When more refined questions are asked, incorporating both frequency and severity ratings, and, when multiple perspectives are incorporated into the diagnostic process, more valid assessments of Long COVID status are possible. Our article has focused more on self-report data but bringing into the diagnostic process a host of biological variables [38,39,40] and a medical examination will lead to more accurate decisions. Learning lessons from decades of research with past post-viral illnesses will ultimately enhance our efforts in understanding how to diagnose Long COVID [8,15].