EXPERT REACTION: Modelling predicts 80 per cent adult COVID-19 vaccination won't be enough

We need to be extremely cautious about making policy decisions based on just a single model. Such models are extremely sensitive to their inputs and assumptions, and can easily provide misleading results. We need to remember that last year many of the early pandemic models incorrectly predicted that COVID-19 if left unchecked would reach an early peak within just a few months and would then spontaneously disappear due to herd immunity. Many governments including UK, Sweden, Brazil made major decisions based on those modelling results with disastrous consequences, collectively resulting in over a million avoidable deaths across those countries who did not implement any early control measures based on misplaced trust in those models. We don’t want to see a repeat of this in Australia.

The Doherty model is based on assumptions that the two current vaccines provide high levels of sustained protection against delta virus transmission, assumptions poorly supported by available data. At best, at their peak these vaccines might provide 50% protection against transmission but this then appears to rapidly wane even while protection against serious disease is relatively maintained.  Factor this real world data on the delta virus and efficacy of these vaccines into the model and a population vaccination coverage of at least 90-95% is more likely to be needed to reduce the likelihood of future uncontrolled outbreaks if other control measures are relaxed too early.

Another ongoing issue is the repercussions of long Covid, and whether the current vaccines will reduce this risk. One of the surprising features of long Covid is that it does not appear to correlate with severity of disease, with many sufferers being young patients who have only had mild infections.  If current vaccines only reduce disease severity and do not prevent infection then they may not reduce the incidence of long Covid – more epidemiological data is urgently needed on this question.

The argument that some lives will need to be sacrificed in the interests of avoiding future lockdowns presupposes that a final solution is in sight, such that these lives will not be lost in vain. However scientific evidence suggests that while current vaccines are beneficial they do not by themselves represent a final solution, and better alternatives are still needed.

What is needed is acceptance that the pandemic still has a way to go, the ultimate solution is still not available and hence we should continue to apply policies that prevent/stall unnecessary deaths, while investing all available resources into more quickly finding the ultimate solution, at least one aspect of which might be safer and more effective vaccines that provide better protection against the variants and have more impact on reducing transmission than current candidates.

The basic principal of modelling is that any prediction made is only as good as the assumption and data that are fed into the model. If unrealistic assumptions are made, vastly unrealistic predictions will follow. This is why when reviewing model results, experts will always first look at the model’s assumption and algorithm before looking at the model output.

The modelling by the experienced Doherty team, and the calculations presented by Hyde and colleagues today, use very different methods and make very different assumptions. The Doherty example formally models both viral transmission in the Australian population, likelihoods of disease and health systems capacities.

It used both published international studies as well as the best available Australian data to derive parameters and inputs into the model. As a result of this careful construction and parameterisation, while still modelling different scenarios, the Doherty example is likely to give realistic predictions of what could occur under different vaccination levels. The calculations by Hyde et al and the assumptions that go into them are much more simplistic and a lot less realistic.

A key difference between the two, is the reproduction numbers used for the modelling. Hyde et al use an R0 of 6, based on international data. This is the rate at which the delta virus spreads when there are no restrictions in place and people mingle freely and in close contact. In Australia, with the current regulatory settings in place, this will only ever occur in an a close household setting (e.g. between parents and their children), not between households and in the general population.

What is a more accurate reflection of the viral transmission is the effective reproduction number (R eff) i.e. how many subsequent infections are actually observed in real-life. This is why the Doherty modelling is modelling the transmission potential (TP) of the virus under different control settings. This is why the Doherty team, even though they actually assume a higher R0 for delta than Hyde et al (i.e. ~8), set the baseline transmission potential of the delta virus under basic public health and social measures and real-world ‘test, trace, isolate, quarantine‘ at 3.6.

As the Doherty modelling convincingly shows, vaccination levels and other public health measures further reduce the transmission potential of the virus, making it possible to constrain viral transmission with fewer – but very importantly not without any – public health measures in place.

Releasing and publicising results that are based on overly simplistic concepts and unrealistic assumptions, at a time when an overstretched and exhausted Australian public needs reassurance that the very significant efforts they are asked to sustain will lead to an eventual positive outcome, is not only unhelpful but is in my opinion irresponsible.

Australia needs to review more than one modelling scenario – this set of models produced by Hyde et al adds to the existing models regarding COVID in Australia. We need to discuss many (Doherty, Burnet, Grattan, McBride, Hyde and more). This is not just for academic interest – but instead because it is a crucial time in Australia’s pandemic response.

This is a crucial time in the pandemic and Australia must not squander its opportunity, keeping restrictions, while awaiting the population to be covered by vaccination – because the alternative (re-opening when vaccinations are too low) risks a ‘let it rip’ scenario through the unvaccinated which may overwhelm health services requiring further lock downs – which may not be as helpful as they are now because at that point the ‘cat would be out of the bag’, so to speak.

With modelling, one must pay detailed attention to the assumptions being used as they can change the outcomes of the modelling drastically.

One key assumption is that of R0 [reproductive number - a measure of how contagious the strain is]. In this set of models the authors assumed R0 to be 6 for Delta, this is in keeping with US CDC reporting earlier this month. 

It is useful that these models by Hyde and team have been run with various scenarios of vaccination (by age, by vaccine type, with/without children vaccinated) as this provides us greater knowledge regarding potential scenarios and therefore provides further discussion points for decision-making.

It is notable in these models that, vaccinating children and adolescents prevents infections and deaths in all age groups. It is also notable in these models that even vaccinated people will die, if restrictions are relaxed too soon with insufficient vaccination coverage.

It is notable from these models that the authors recommend greater than 90 per cent of the entire population be vaccinated before exposing Australians to freely circulating COVID virus in the community without strong restrictions.

This alternative OSF modelling predicts the likely morbidity and mortality rates from prematurely relaxing lockdowns and/or fully reopening. The modelling assumptions are valid/cogent, the predicted outcomes are realistic, and within the boundaries and context of the Grattan, Burnet and Doherty models.

The Doherty model is based on more assumptions than less assumed in this OSF model, which explains the discrepancy between the two (less assumptions means the model is more realistic). 

I make the following observations on actions that can be taken (or discussions that can be had) on the aggregate likelihoods predicted across all models. 

1) Fully reopening should only be considered when herd immunity (95%) has been achieved, which means some degree of public health restrictions must be enforced until 95%, given the R=6 virulence and predicted morbidity and mortality rates. 

2) Each state/territory will reach the 80% - 90% vaccination coverage on different dates, for instance ACT is first up for 80% on 16 Oct, while WA is last in line on 25 Nov, this level of detail (or granularity) should be factored into decisions on which restrictions can be relaxed and when. International travel and self-quarantine should also be aligned with this state-level coverage (it is not a one size fits all). 

3) All models state that high mortality will occur among vulnerable vaccinated people – a formal definition of “vulnerable vaccinated” along with a breakdown of this group by location and morbidity/hospitalisation requirement should be matched with vaccination coverage at LGA or state level so that contact tracing for the Delta variant can be repurposed to primarily focus on these vulnerable individuals and communities.

For most, mathematical modelling feels like a black box, not easily subjected to scrutiny. The Doherty institute has just released the outcome of complex up to date modelling. 

In Australia: How many competing models do we need? Multiple variables need to be accurately measured or estimated. If just one is wrong, what confidence can be put in the predictions?

The value of modelling depends on multiple scientific disciplines from epidemiology and psychology to mathematics, history and ethics. It is not an easy task.

This model shows similar results to other models from the Grattan and Burnet Institutes. Modelling depends on assumptions used, and the considerations incorporated in the model, so results can vary.

From our own modelling (unpublished), I would concur that Australia would need at least 80 per cent of the whole population vaccinated, and may still need some restrictions such as masks. We can learn from the UK, the US and Israel which all lifted restrictions between May-June 2021 with vaccination rates around 60 per cent of the whole population. All three countries saw a resurgence of Delta, with severe school outbreaks, paediatric ICUs overflowing, many kids hospitalised etc. We should take these lessons on board.  

Some strategies that will help with safer re-opening include ventilation in business and schools (testing and using air purifiers, for example), a communication campaign on shared air and ventilation, vaccination of children 12 and up, and measures to protect health workers and the health system.

A surge could decimate the health workforce and have flow-on effects for all healthcare, so it is in all our interests to make sure our health workers do not get infected. They should all have fit-tested N95s and a third dose booster vaccine as a priority. I am hearing from country hospitals that they have run out of PPE and staff are being asked to get their own. Our country hospitals and regional communities are especially vulnerable.

There is a lot we can do to protect health workers, the health system, the community and children if opening up at the current proposed levels.

The latest modelling regarding the effects of opening up the community too early, or indeed getting to the point of treating the virus like ‘the flu’ is very timely if not sobering.

It needs to be noted that all theoretical modelling is based on a series of assumptions which drives the model of which the output is a ‘scenario’ of what is likely to occur. It therefore is used as a guide.

However these models are sophisticated, and irrespective of which way you cut it, our future path living alongside the SARS-CoV-2 virus will not be the same as living with the flu. The results from the latest modelling clearly demonstrate the need for significant continued social adaption plus also major increases in our healthcare capacity

New modelling described by Hyde, Parslow, Grafton and Kompas for COVID-19 vaccination coverage required before public health measures can be relaxed requires careful consideration.

Several parameters are covered in the study and these need to be interpreted in relation to several assumptions that are made and listed in the supplementary material of the publication. Assumptions are made based on the overseas vaccine experience where it is difficult to separate the effects of vaccination from widespread COVID-19 infection and associated acquired immunity.

Currently no correlate of protection (e.g. specific level of neutralising antibody and/or T cell response required for protection) is known for any of the COVID-19 vaccines. Measurement of antibody titres and/or T cell numbers following vaccination has not been considered to any real extent in Australia and is needed to determine what level and response is required for protection.

Without some idea of whether vaccination induced an appropriate response, and in how many people, will make specific modelling of protection following COVID-19 vaccination difficult.

The lack of such information also makes it hard to determine to what extent booster vaccination is necessary and in which part of the population. Vaccination itself does not equal immunisation.

Presently, this is more likely going to be needed for the elderly and those who are immunocompromised due to medical conditions and/or medications which suppress the immune response. Further clinical data will be needed to determine if the current modelling accurately represents the level of vaccination required before public health measures can be relaxed.

Vaccination remains the best defence against the severe effects of COVID-19 infection to help prevent hospitalisation and complications such as Long COVID, and this is still the most important message for the public.

This group have used the herd immunity threshold for a population to determine the final size of the epidemic. This is unusual, and it should be noted that they do not seem to have a dynamic transmission model.

As they have put it, they have made estimates based on a lower limit of final size, this should also be interpreted as meaning - assuming the epidemic is over and no dynamic changes in policy occur during the epidemic. Once a final size is estimated they then used the published infection fatality rates and other severe outcomes to estimate the total number of hospitalisation and deaths by age group.

The Doherty modelling [used by the government] is very reliant on the assumed effective reproduction number, the relative effects of the virus on children (borrowed from the values for the original Wuhan strain) and the impact of public health measures (I think assumed to be equal across age groups). So I am not surprised that other groups are finding vastly different results. 

Models are mostly saying similar things but interpretation is the key difference. My group has shown that the models are very sensitive to assumptions. Here are some from The Doherty (baseline) model:

- That the Delta variant has the same child infection sparing properties that the original strain had

- That we can keep the effective reproduction number to 3.6

- That we can apply public health measures evenly across all age groups and populations

If any of these assumptions change by about 10 per cent, then children are likely to drive COVID-19 infections and we will see quite a different epidemic to that predicted by the Doherty.

There will be many more children infected, and the infection will spread faster and cause more overall infections.

Another feature of Delta that is not yet completely understood but very concerning is that it may be more severe (according to Fisman et al in Canada). 

I do not have a problem with opening up at 70 per cent vaccine coverage but we need to do so understanding what this is likely to look like and just how uncertain we are about the potential for hospitalisations and deaths.

Further, we should not do it until all people have had the opportunity to be vaccinated, and vaccinating younger age groups (down to 12+) could make a big difference in outcome.