Masks, transmission, AstraZeneca recommendations: Scientific evidence mostly emerges from a continuous process. Which can be problematic in a crisis situation.
In normal times, scientific evidence serves as the basis for new hypotheses and innovations, and ideally also catalysts for political decisions. The detailed processes behind delivered data was mostly reserved for scientists and possibly politicians or other interested parties.
During the corona crisis, we are all suddenly confronted with it and it can be confusing.
Scientists are now increasingly in the spotlight and huge political decisions are made on a daily basis that have a major impact on our lives - based on existing or non-existent scientific evidence.
Questions such as; How is the virus transmitted? More about aerosols, or more about smear infection? Are masks useful, or is this not yet scientifically proven? Why wasn't AstraZeneca initially recommended for older people and now it's the opposite?
In the grip of the pandemic, it is difficult to keep track of everything and to understand all of the scientific developments.
In this article, we would like to use specific examples from the Covid pandemic to illustrate how scientific evidence is created in order to better classify scientific statements and (apparent) contradictions - and what is particularly important to researchers, politicians, journalists and the public in this time of crisis.
How is scientific evidence created?At the beginning there is always a question:
When the new coronavirus spread in the first few months of 2020, a key question was: How is SARS-CoV-2 transmitted? If it is mainly smear infections, then hand and surface disinfection would be a decisive protective measure. If the virus spreads via droplets, masks could offer protection. If aerosols are used, interiors would have to be avoided or at least well ventilated, in addition to the masks. Since it was a new virus, scientists didn't yet know the answer.
From literature research to hypothesis
"At the beginning of the pandemic there was no evidence, we could only speculate on the basis of what we know about other similar viruses," says Rudi Balling. Scientists did what they always do when a new question arises: They combed through the scientific literature and formulated hypotheses - assumptions that then have to be proven or refuted through observations or experiments. “We assumed that it works like a normal cold, ie mainly via smear infections and droplets. Back then I still opened all the doors with my elbows because I wanted to avoid contaminating my hands,” remembers Balling.
About systematic data collection to new knowledge
“It took a long time to find out that the main routes of transmission are not smear infections, but a mixture of droplets and aerosols. This evidence only came through systematic data collection,” continues Rudi Balling. Here in Luxembourg, he said, the Large Scale Testing program and contact tracing contributed to this finding, even if collecting data from clusters and transmission routes was not easy. And of course there were observations and correlation studies in other countries that captured connections. "That takes time and requires statistically robust case numbers," he says.
We all know the political decisions that were based on this new evidence only too well: Keep your distance and avoid contacts by closing restaurants, cafes, shops, schools, fitness studios ... and the tiresome mask requirement.
Evidence as a continuous process
Scientific evidence is often just a consensus that prevails among the majority of scientists at time X. It is therefore not uncommon for scientists to change their minds or positions, because, "Scientific evidence is a continuum; it is constantly being supplemented, refined and sometimes refuted by new findings," explains Rudi Balling.
What is a normal process for scientists can be very confusing for politicians and the general public. There was a lack of understanding, for example, when the mask requirement was introduced in the corona pandemic. At the beginning it was said: masks are useless. Then suddenly: masks bring something. The reason for this change of course was not a change of heart on the part of scientists or politicians, but rather two things: new evidence on transmission routes and the consensus that the benefits of face masks as one of several protective measures could outweigh the inconveniences or the risk of possible harm by wearing a mask.
"The role of aerosols is also still a topical research topic," emphasizes Rudi Balling. “Recent studies have shown that they can stay in the air for hours. Architects and engineers are now also participating in this research and are investigating, for example, how aerosols spread in high-rise buildings.
What difficulties does the corona crisis pose for scientists?
The factor of time and uncertainties
It takes time for evidence to crystallize. Usually researchers have this time too. In the current crisis, however, they are constantly running out of time. Politicians and the general public sometimes have to make decisions based on evidence that is uncertain or incomplete. Decisions in the corona crisis often have to be made with an assessment of a certain risk.
“Scientists can say 'I don't know,'” says Rudi Balling. Ignorance is a large part of their everyday life and they are constantly looking for new evidence, new knowledge, the truth. They communicate and discuss possible uncertainties, as well as strengths and weaknesses of their studies, not only among themselves, but also increasingly in public in the current crisis. Rudi Balling says: “Politicians often don't have this feeling of uncertainty and still have to make decisions. And the population looks at both, and so fear arises. This can quickly lead to distrust."
According to Rudi Balling, we are faced with a dilemma in the current situation: “Science takes time, but political decisions cannot wait. And the population also has no time and feels like a victim. "
Data protection and transparency
Another problem that plagues not only Luxembourg researchers is access to data from health authorities. For modeling the number of cases, high-quality and well-annotated data from test results, contact tracing and clinics, for example, are very important, says Rudi Balling.
"At the beginning there were simply no digitized pipelines for data exchange with the health authorities," explains Rudi Balling, "but even today it is still difficult for our researchers to get reliable data." One reason for this is data protection.
Here, Balling points out another ethical dilemma: “It is one thing, how far data protection can go in normal times. But should the same measure apply to times of crisis? Or do you have to make small compromises there in order to be able to find solutions to the crisis more quickly? "
Exchange with politicians
In general, Balling would like more exchange between science and politics - in both directions. "More exchange with politicians, feedback and transparency would be important in order to create a basis of trust and to work together optimally," he says.
In this context, he cites the evidence on the role of children in the pandemic as an example. In the beginning it was not clear: are they at risk? Are they carriers? Together with the pediatrician Isabel de la Fuente from the Kannerklinik at the CHL, Balling summarized the scientific evidence on the topic in a policy brief in May 2020. Her conclusion at the time: Children can become infected and also transmit the virus, but are often asymptomatic and far less at risk of severe disease than adults and especially older people. "Reliable data on the prevalence, herd immunity or the dynamics of the COVID-19 pandemic in children in Luxembourg were not available," emphasizes Balling. However, these are important as a basis for political decisions in the area of child health care.
The scientist and the pediatrician therefore recommended that a child prevalence study be carried out in Luxembourg. But that was not done then. To this day, Rudi Balling doesn't know why. "It would be helpful for us to know what scientific evidence is reflected in political decisions."
Though information can be dense, scientists can also improve their communication. The AstraZeneca vaccine certainly offers enough material for discussion. As a reminder: after the clinical study was briefly interrupted twice, the vaccine received its provisional approval, but was initially not recommended for people over 65 years of age. Then older people were vaccinated with it anyway, and now some countries no longer vaccinate it or only offer the vaccine to people over 60 - because in younger people there were very rare cases of sinus vein thrombosis after vaccination.
"Here we always acted according to the principle of caution," says Balling. It was right to stop the study or vaccinations. To investigate whether the observed relationships can be attributed causally to the vaccination and to re-evaluate the benefit-risk of the vaccination.
"Communication failed", Balling admits. “In terms of scientific evidence, nothing went wrong,” he assures, “but the reasons for the individual decisions were not communicated convincingly.” It is understandable that there is mistrust among the population. Balling sees it as a dilemma for communication: "On the one hand, it is good and important to communicate transparently, on the other hand, very few laypeople can weigh up the risks, which can then lead to fear of the vaccination."
What mistakes can be made in evaluating scientific evidence?
No matter how well or poorly scientists communicate, it is not always easy for 'regular' people to interpret their communications. Here are the most common mistakes that can be made when evaluating scientific evidence.
"Finger pointing" and the prevention paradox
“You should be careful not to point your finger at someone. We all make mistakes,” emphasizes Balling. He cites an example: “Models of the Covid-19 Task Force in Luxembourg were mostly correct with their predictions of the number of cases. But once they were wrong, in the second wave in July 2020 - the number of cases did not rise as predicted. That was enough for people to say: Everything they predict is wrong."
But even if they are correct, people like to say: yes, that's how it was predicted. Or the so-called prevention paradox occurs. Containment measures are being taken and the population is changing their behavior - the number of cases remains stable or even declines. And people then say: It didn't turn out as bad as the researchers predicted.
Not enough evidence-based risk communication
Let's stay with the example of the modeling. “The scientists of the task force communicated their case numbers for a while with the respective uncertainties, which corresponds to a probable, optimistic and pessimistic scenario. They also presented a short-term and a long-term projection. "Modeling epidemiological scenarios is very difficult, there are many factors that can influence the evolution of the number of cases," says Balling, "This is particularly true for long-term projections." Help politicians make their decisions.
According to Balling, it is imperative that journalists communicate the uncertainties of the available scientific evidence and they should also be able to provide a risk assessment on a common sense level. “What does a 10% probability mean that the pessimistic scenario will occur? What if that happens? How do I prepare this? When is it reasonable to close schools? Of course, this has a lot to do with the respective culture of the country, but it is a problem when the media are always on the lookout for headlines and are driven by click rates."
In the social networks it is worse, says Balling: “Some influencers are paid per click. There are those who publish the same message from Retraction Watch over and over again every three months because it brings them good click rates. That is an ethical problem. "
Editor's note: Retraction Watch is a blog that reports on scientific publications that have been withdrawn - whether by the authors themselves or the journals in which they were published.
In the Corona crisis, politicians have to make decisions based on scientific evidence almost every day. Which is often still very heterogeneous or fraught with uncertainties. “But politicians sometimes also look for what fits well into their party political agenda,” Balling says. He is referring to a project in which a lot of energy was invested last summer: the development or adaptation of a corona tracing app for Luxembourg. Ultimately, however, the government decided against it - perhaps also because a contact tracing app doesn't fit well into a liberal concept, thinks Rudi Balling.
Too much focus on individual reports and rare events
Another common mistake: According to Balling, is that both politicians and journalists are too distracted by individual reports on rare events.
As an example, he first cites the Kawasaki-like syndrome in infected children, which caused a stir in May 2020 shortly before the schools reopened. “It is a very rare systemic disease that can occur worldwide in children with viral infections such as Covid-19 in around 9 out of 100,000 cases and can usually be treated well with medication. According to Isabel de la Fuente, a paediatrician at the CHL Kannerklinik, around five cases of Kawasaki syndrome occur in children with flu infections in Luxembourg each year. Last year there were fewer Kawasaki cases from flu, but about the same number from Covid-19, all of which could be treated. In the media, however, the few Kawasaki cases in Luxembourg received a relatively large amount of attention."
A more recent example is the very rare thrombosis associated with low blood platelets, which was mostly observed within two weeks of being vaccinated with the AstraZeneca vaccine - especially in women under 60 (1 to 2 cases per 100,000 women in Germany). “Initially, there are only reports of individual cases; sound evidence is only created through systematic data collection. The aim is to record how frequent an event is and whether there are causal connections," according to Balling.
A comparison must be made: Does the observed rare event occur more frequently among vaccinated people than it would occur purely by chance under normal circumstances (i.e. not European and British drug approval authorities have now concluded that the thromboses observed are a “possible” and “extremely rare” side effect of vaccination with the Astrazeneca vaccine and therefore included in the list of possible side effects in the product sheet.
They estimate that, on average, about one person for every 200,000-250,000 people who are vaccinated could develop such a rare thrombosis, but they still rate the benefits of the vaccine as greater than possible risks - except for those under 30, who are the risk of a severe course of Covid-19 disease is extremely low.
Equate recommendations with scientific evidence
Is a conclusion from a European Medicines Agency (EMA) scientific evidence? “No,” says Rudi Balling, “the EMA recommendations are not evidence, but they are usually based on evidence and thus form trustworthy care for the population.” Institutions such as the EMA or STIKO could only make brief recommendations. Politicians then have to make decisions that both protect minorities and satisfy the majority. “How do I protect the individual, but at the same time also the entire population? This is often an ethical dilemma. Dialogue with ethics experts can be very useful here,” says Balling.
Scientific evidence is findings from research. With the help of systematic methods and critical evaluation, scientists try step by step to find answers to certain questions - or at least support for their hypotheses. This process takes time and multiple studies, so scientific evidence is often slow to emerge. And this evidence can be continuously expanded, supplemented and refined - and under certain circumstances also refuted again.
Science often moves in a gray area at a point in time X. Scientific evidence is often fraught with uncertainty and cannot always provide definitive answers.
This can be a problem in public debates that (should) be based on scientific evidence. Scientists cannot always say whether something is black or white. However, political and individual decisions are often viewed or presented as black or white on the basis of assessments provided by scientists.
It would be important for scientists to communicate more clearly what is known and what is not, and the uncertainties associated with the latest evidence.
It would be important for politicians, journalists and individuals to always look at all available information and weigh decisions based on probabilities. But this is exactly what many non-scientists often find difficult.
In both cases, science communication can help build bridges and provide classifications.
Authors: Michèle Weber (FNR)
Editors: Jean-Paul Bertemes (FNR), Sabine Schmitz (LCSB)