Covid-19: Lockdown evaluation model was flawed, claims analysis

Researchers from the Imperial College London designed a model to examine the effect of different measures that were put in place to prevent the spread of the coronavirus.

However, the model had fundamental shortcomings and cannot be used to draw the published conclusions, claimed Swedish researchers from Lund University, and other institutions.

The analysis, published in the journal Nature, stated that the results from Imperial indicated that it was almost exclusively the complete societal lockdown that suppressed the wave of infections in Europe during spring.

The study estimated the effects of different measures such as social distancing, self-isolating, closing schools, banning public events and the lockdown itself, the analysis added.

First author of the analysis, Kristian Soltesz, associate professor in automatic control at Lund University, said: “As the measures were introduced at roughly the same time over a few weeks in March, the mortality data used simply does not contain enough information to differentiate their individual effects.”

He added: “We have demonstrated this by conducting a mathematical analysis. Using this as a basis, we then ran simulations using Imperial College’s original code to illustrate how the model’s sensitivity leads to unreliable results.”

The researchers intended to analyse the Imperial College model as the study explained almost all of the reduction in transmission during the spring via lockdowns in 10 of the 11 countries modelled. The exception was Sweden, which never introduced a lockdown.

“In Sweden, the model offered an entirely different measure as an explanation to the reduction ― a measure that appeared almost ineffective in the other countries. It seemed almost too good to be true that an effective lockdown was introduced in every country except one, while another measure appeared to be unusually effective in this country,” Soltesz noted.

Soltesz, however, mentioned that it is entirely plausible that individual measures had an effect. But, the model could not be used to determine how effective they were.

“The various interventions do not appear to work in isolation from one another, but are often dependent upon each other. A change in behaviour as a result of one intervention influences the effect of other interventions. How much and in what way is harder to know, and requires different skills and collaboration,” says Anna Jöud, associate professor in epidemiology at Lund University and co-author of the study.

Analyses of models from Imperial College and others highlight the importance of epidemiological models being reviewed, according to the authors.

“There is a major focus in the debate on sources of data and their reliability, but an almost total lack of systematic review of the sensitivity of different models in terms of parameters and data. This is just as important, especially when governments across the globe are using dynamic models as a basis for decisions,” the authors pointed out in their analysis.

“With a lot at stake, it is wise to be humble when faced with fundamental limitations. Dynamic models are usable as long as they take into account the uncertainty of the assumptions on which they are based and the data they are led by. If this is not the case, the results are on a par with assumptions or guesses, concluded Soltesz.

COMMENT NOW