Media release
From:
Flawed body of research indicates true ‘long COVID’ risk likely exaggerated
Overly broad definitions and lack of comparator groups have distorted risk, say researchers
Leading to increased public anxiety and healthcare spend; misdiagnoses; diversion of funds
Major flaws in the current body of published research on ‘long COVID’ have likely greatly exaggerated the true risks of developing the condition, suggests an analysis published in BMJ Evidence Based Medicine.
Overly broad definitions, a lack of appropriate, or any, comparison groups, among other things, in studies looking at the incidence, prevalence, and control of the condition—epidemiology—have distorted the risks, say the researchers.
This is further compounded by inclusion of poorly conducted studies into systematic reviews and pooled data analyses that end up overstating the risk yet again, they add.
The likely consequences of this include, but aren’t limited to, increased public anxiety and healthcare spend; misdiagnoses; and diversion of funds from those who really do have other long term conditions secondary to COVID-19 infection, suggest the researchers.
Many after effects of COVID-19 infection include post-ICU syndrome—a constellation of health issues that are present when the patient is in intensive care and which persist after discharge home—and shortness of breath following pneumonia. Trouble is: these are common to many upper respiratory viruses, point out the researchers.
None of the working definitions of ‘long COVID’ used by influential health bodies, such as the US Centers for Disease Control and Prevention, the World Health Organization, the UK National Institute for Health and Care Excellence (NICE), Scottish Intercollegiate Guidelines Network (SIGN), and the Royal College of General Practitioners requires a causal link between the virus responsible for COVID-19 (SARS-CoV2) and a range of symptoms.
Not only should comparator (control) groups be included in ‘long COVID’ studies, when they often aren’t, but they should also be properly matched to cases, ideally by age, sex, geography, socioeconomic status and, if possible, underlying health and health behaviours, which they rarely are, say the researchers.
During the early stages of the pandemic, when SARS-CoV-2 testing wasn’t widely available, studies were more likely to include a non-representative sample of SARS-CoV-2-positive patients by including fewer patients with mild or no symptoms.
This is known as sampling bias, which occurs when certain members of a population have a higher probability of being included in a study sample than others, potentially limiting the generalisability of a study’s findings, explain the researchers.
“Our analysis indicates that, in addition to including appropriately matched controls, there is a need for better case definitions and more stringent [‘long COVID’] criteria, which should include continuous symptoms after confirmed SARS-CoV-2 infection and take into consideration baseline characteristics, including physical and mental health, which may contribute to an individual’s post COVID experience, “ they write, adding that the umbrella term ‘long COVID’ should be jettisoned in favour of different terms for specific after effects.
While the results of high quality population studies on ‘long COVID’ in adults and children have been reassuring, they point out, the body of research “is replete with studies with critical biases” they add, setting out common pitfalls.
“Ultimately, biomedicine must seek to aid all people who are suffering. In order to do so, the best scientific methods and analysis must be applied. Inappropriate definitions and flawed methods do not serve those whom medicine seeks to help,” they insist.
“Improving standards of evidence generation is the ideal method to take long COVID seriously, improve outcomes, and avoid the risks of misdiagnosis and inappropriate treatment,” they include.
Notes for editors
Analysis: How methodological pitfalls have created widespread misunderstanding about long COVID doi 10.1136/bmjebm-2023-112338
Journal: BMJ Evidence Based Medicine
Expert Reaction
These comments have been collated by the Science Media Centre to provide a variety of expert perspectives on this issue. Feel free to use these quotes in your stories. Views expressed are the personal opinions of the experts named. They do not represent the views of the SMC or any other organisation unless specifically stated.
Professor Paula Lorgelly, Faculty of Medical and Health Sciences and School of Business, University of Auckland
If the pandemic has taught us one thing it is the value of hindsight. Our responses during a pandemic to testing, isolation, and lockdown has been latterly criticised. What is often forgotten with hindsight critique is that these responses were during a pandemic, informed by the best available evidence and performed at haste.This commentary raises some valid questions about the scientific evidence regarding the prevalence of Long COVID, and much of the critique is valid although the authors fail to acknowledge that much of the Long COVID research is from studies set up to understand COVID infections during that acute pandemic period, and now subsequently have been utilised to understand the long tail of the pandemic. They are informative nonetheless and this paper – which takes aim at the epidemiological studies of Long COVID by describing best practice – is useful for new studies going forward, but should not take away that Long COVID is debilitating for many and the risk is real, just like there is a risk for any acute respiratory infection to have lasting symptoms.
"The authors first question the definition of Long COVID, including a lack of confirmatory evidence that a COVID infection is the cause. Early in the pandemic there was no testing, and currently – due to changes in isolation requirements – there are limited RATs or PCR testing. Confirming Long COVID symptoms as caused by COVID-19 would require serology testing; this is not routine in New Zealand so would require formal surveillance. The Ministry of Health failed to launch their much-awaited surveillance surveys, and while individuals can pay for serology (antibody) testing, at $100 it is unaffordable for many. Given many individuals with Long COVID have had repeat infections, confirmatory tests would not be able to confirm the infection that caused Long COVID, and antibodies also wane so testing would not necessarily identify an infection sometime in the distant past.
"The Long COVID Registry Aotearoa does not require testing nor even a formal diagnosis, any individual who self-reports long COVID symptoms is able to join. The registry is not set up to estimate the prevalence of long COVID, but instead estimate the burden – the impact on health, quality of life and finances. The authors of this paper argue that we need to improve evidence generation in order “to take Long COVID seriously, improve outcomes, and avoid the risks of misdiagnosis and inappropriate treatment”. A better estimate of prevalence could indeed help, but given patients are being gaslit and struggling to get diagnosed with little treatment options available, and certainly no specialist clinics in Aotearoa, understanding the impact of Long COVID beyond its prevalence will allow the Ministry of Health, healthcare providers and other agencies to take Long COVID seriously.
Professor Michael Baker, Department of Public Health/Te Tari Hauora Tūmatanui, University of Otago, Wellington
I agree with the first message of this paper, which is that we need well designed studies to provide a valid measure of the long-term effects of acute COVID-19 infection (Long COVID).
"Such studies need robust case definitions, adequate duration of follow-up, and suitable comparison groups. Now that the pandemic has been with us for more than three years, we should expect such high-quality studies. Ultimately, these studies should provide us with consistent evidence about key aspects of the epidemiology of Long COVID – including its prevalence, severity, and duration. The published literature is not yet giving us this clear consistent picture.
“The second message of this paper appears to be that there is a negligible risk of Long COVID, based on the selection of papers they have quoted. That message is concerning and does not fit with mainstream scientific evidence, nor the experience of the large population of people living with Long COVID and the clinicians caring for them. There is now overwhelming evidence that SARS-CoV-2 carries a significant risk of long-term effects. This evidence doesn’t just come from epidemiological studies, but also from studies looking at the severe and lasting pathological changes that occur following SARS-CoV-2 infection.
“Even if the risk of Long COVID from a single infection is now relatively low with Omicron subvariants, and with vaccination fortunately pushing it down even further, it remains a serious problem that we need to manage. COVID-19 has infected most of world’s population and is continuing to cause multiple reinfections for many. Consequently, the population level impact of infection is large and growing. There is also the potential that some of the long-term effects of infection with this virus may take years to manifest themselves, as we have seen with other viral infections.
“One remarkable feature of this paper severely damages its credibility. It includes a section titled The most well-designed studies provide reassuring estimates. In it the authors include just two studies to support that sweeping statement: one was in children, and in the other they reported and commented on the analysis of findings for people younger than 50 years. In doing this highly selective ‘mini meta-analysis’ they are contradicting the core message of their paper. This very biased treatment of the subject suggests that these authors have an underlying view of Long COVID, rather than the evidence-informed perspective they are promoting.
Dr Mona Jeffreys, Associate Professor (Research) in Epidemiology, Te Hernega Waka - Victoria University of Wellington
The paper is a careful assessment of the pitfalls of epidemiological studies. It is worth remembering that some of these were conducted during a time of significant upheaval.
"The authors make good points regarding the lack of consistency in case-definitions, and other potential sources of bias. It is worth noting that all definitions, including the one used in Aotearoa New Zealand, are broadly consistent, other than the American one (CDC).
"These are useful reminders about how to carry out careful epidemiological investigations.
"However, the authors do not appear to have been informed by the Myalgic Encephalomyelitis/ Chronic Fatigue Syndrome literature; they appear to prefer to focus on a set of pre-defined, persisting, single post-COVID symptoms, rather than considering Long COVID as a term for a syndrome or collection of symptoms which can fluctuate in presence/ absence as well as severity.
"There is no evidence for the “unintended consequences” listed in the key messages box, these are not mentioned in the paper, and appear to be the authors’ impressions, rather than evidence-based statements.
"Our lowest possible estimate of the proportion of adults who have symptoms of Long COVID in Aotearoa New Zealand (from the pre-Omicron period) is 2.5%. Even given the possibility of Omicron and/or vaccination having half the risk of Long COVID, this translates to nearly 25,000 people.
Emeritus Professor Warren Tate FRSNZ CNZM, Department of Biochemistry, University of Otago
There is no doubt many people have developed a serious post condition following their infection with SARS-C0V-2. The numbers are large because of the unprecedented SARS-Cov-2 infection numbers worldwide - now approaching 800 million. The BMJ paper highlights an important problem as we move forward to understand better the extent and long-term effects of the mixture of conditions classified as Long COVID.
"The World Health Organization moved quickly to establish a clinical case definition for what was known to be a heterogeneous group, at the very least containing people with ongoing organ damage, and groups with a classic post viral fatigue condition reminiscent of Myalgic Encephalomyelitis/Chronic Fatigue Syndrome (ME/CFS). Despite this, it was surprising that supposedly well researched papers have come out with a huge range of 10-50% of those with COVID, although more recently it has dropped in some publications below 10%, with variants like those of the Omicron group causing a lower incidence.
"Hence, there has been a pressing need for better definition with precise controls and diagnosis of the Long COVID condition and in this sense the BMJ paper is a timely reminder. Particularly pressing is to have clear definition of the clinical phenotypes. A very recent paper defined four clinical phenotypes, a post viral fatigue syndrome like ME/CFS, a respiratory syndrome, a pain syndrome, and a neurosensory condition in a ratio of ~4:2;2;1. So, the need as outlined in this BMJ paper for better definitions and precise controls so targeted help for families, communities, and countries can be better defined, is timely.
"The call not to divert precious funding towards Long COVID and away from other needs - while having vaildity - ignores the fact that for decades the collective group of post viral syndromes (ME/CFS) has been poorly resourced, and the affected largely ignored. This new interest in Long COVID has benefits for those people who have been living with such debilitating conditions for many years. But the BMJ paper principle is important in that these affected people may have even less support if the focus is totally on Long COVID. But it would be a tragedy if those affected with Long COVID become another subgroup of the ‘missing millions’.
Associate Professor Amanda Kvalsvig, epidemiologist in the Department of Public Health, University of Otago Wellington
he authors of this study usefully point out that because the umbrella term Long COVID is a broad definition, it's challenging to design population studies that give a precise measure of Long COVID prevalence in a population. The discussion about pitfalls in study design is good. But this paper is highly selective in terms of the studies that the authors choose to highlight.
"The authors have omitted to mention that alongside population surveys there is an extensive and active field of research to understand the impacts of this virus on the body. The science about longer-term effects of Covid-19 infection is now very much better understood than it was even a year ago. The research shows that COVID-19 is a multisystem disease that can cause microclots, changes to the immune system, viral persistence in tissues, and other effects even in mild cases; this major paper in Nature Reviews Immunology provides a detailed and recent overview. These effects are the basis for well-described longer-term effects of COVID-19 such as impaired brain function, extreme fatigue, and stroke. These tissue-level effects can also be silent (causing no symptoms) but they are known risk factors for heart disease and other conditions, raising concerns for population health in the future, with potentially large numbers of people experiencing poor health in years to come.
"Even the research that is mentioned in this article is not as reassuring as the article seems to suggest:
- Antonelli et al. reported lower risk of Long COVID for Omicron compared with Delta infection, depending on age and time since vaccination. But they noted that Omicron variants have caused far higher case numbers and the conclusion of their article is that 'future numbers with Long COVID will inevitably rise.'
- Studies that use serology to identify past infection have an important limitation that some people (including children) don’t produce long-lasting antibodies despite having had the infection, leading to unreliable results in Long COVID research.
- Studies using ONS data are not designed to detect and measure the longer-term effects that are described in other research.
"All pandemics and major epidemics cast a long shadow of chronic disease that can continue to emerge decades after the initial emergency phase. In 2023 we are still experiencing high levels of COVID-19 infection and reinfection in Aotearoa New Zealand. Reducing COVID-19 infection rates is still the only reliable way to reduce the longer-term impacts on population health. We have many effective ways to do this, in particular improving indoor air quality. It would be good to see the New Zealand Government acknowledging this challenge and taking active steps to protect New Zealanders’ future health.