Perils of cross-country Covid-19 comparisons
Rachel Irwin,
Researcher,
Lund University,
Sweden
During the past year, the international media environment has been filled with articles and social media posts about which countries are doing best or worst in the pandemic. Cross-country comparison is especially problematic, as countries have different ways of collecting and reporting data which, in turn, need to be interpreted in relation to country contexts. Sweden has been particularly been singled out for its approach, and much of this coverage has been misleading and lacking context. Here are a few lessons, based on how Sweden has been portrayed in the international media environment (1).
First, avoid “cherry-picking” data. Depending on the choice of comparator country, one can prove the success or failure of different types of social distancing measures, colloquially called “lockdowns.” Sweden has more deaths per capita than other Nordic countries but fewer than Spain or the United Kingdom, which imposed various “lockdown” measures; this is not at all enough information to draw conclusions on public health measures. It may also seem obvious to compare Sweden to other Nordic countries, but this is just as problematic as assuming, for example, that all Baltic countries are the same. Also, depending on context, the social pressure to abide by recommendations can be just as effective as the threat of a fine. However, context makes it difficult to compare the outcomes of compulsory measures in one location with those of voluntary measures in another.
Second, country-level data is misleading and an outbreak in one part of a country does not mean a virus is running rampant through the whole nation. This was particularly striking in the so-called “first wave” of the pandemic when Stockholm had much higher morbidity and mortality rates, in comparison to other parts of the country which were more similar to much of Norway and Denmark (to use a faulty comparison!). Even within a single region, deaths are unequally distributed by geography, class, ethnicity, gender and age; however, it is not always possible to disaggregate data due to patient confidentiality or policies around the collection and reporting of ethnicity.
Third, be specific about which policy is being evaluated, and at what level. Sweden has several levels of government (national, regional, and municipal) with different abilities to set and implement policies: guidelines may be set at a national level but implemented at a local level. Also think about whether you are evaluating the content of the policy or its implementation.
Fourth, we must evaluate polices from the perspectives of the diverse groups that experience them. Sweden is not homogenous and the ‘success’ or ‘failure’ of polices depends on one’s perspective, often shaped by living conditions, geography, class, relationship status, age and other demographic factors. Both voluntary and compulsory social distancing measures exacerbate inequalities, requiring a complex conceptual model of their effects on health, equity and society.
Fifth, when to evaluate? Different countries and regions have always been at different phases in their pandemics. The choice of when to evaluate policy is often a political one – and necessary in a democracy – but evaluating too soon can lead to ambiguous results, often at the taxpayer’s expense.
Sixth, familiarity with a country’s context and language goes a long way in improving the quality of policy evaluation. Within the international media coverage, it was common to see public health experts, with little connection to Sweden, making inaccurate statements in the media, often based on misunderstandings about the country’s complex system of public administration and politics, or with the assumption that Stockholm represents the whole country. Moreover, one reason why there were not more so-called “lockdown measures” is because many of these are complicated to implement under Swedish law, a nuance that was missing from much of the reporting.
Overall, asking ‘Does lockdown work?’ is useless. Rather, ask what specific measures were put in place, at what phase in the pandemic, and in what combination? How do these measures and their impact differ by sub-region or group? What are the (non-health) contextual factors that impact outcomes? How do different policies affect health and wellbeing? And will we ever be able to fully disentangle the consequences of the pandemic and the measures taken?
It should go without saying that the news and social media is not the most appropriate forum for the complex task of health policy evaluation. Rather, it is a complex undertaking that relies on a good understanding of a country’s context and a wide range of combined expertise, not least clinical, epidemiological, social, historical and legal.
(1) The article is based on research funded by the Swedish Research Council, grant number 2018-05266
Expert article 2972
> Back to Baltic Rim Economies 3/2021
To receive the Baltic Rim Economies review free of charge, you may register to the mailing list.
The review is published 4-6 times a year.