1st April 2021
By Dr Suzie Cro (Research Fellow and Clinical Trials Statistician) and HEALTHY STATS public involvement group members Ania Henley (chair), Joanna C, Manos Kumar, Paul Hellyer and Yasmin Rahman.
Right now, in Europe and across the world, vaccines for COVID-19 are being rolled out. You may have already received or be expecting a vaccination offer soon. They have been made possible due to the fast and impressive actions of innovators within the healthcare field.
Vaccines are widely thought to be our main route out of the COVID-19 pandemic, and public trust and understanding around the safety and effectiveness of the vaccines is paramount in supporting the required uptake in order to offer societal protection.
Robust and rigorous clinical trials have helped to prove that these new vaccines meet the high safety and effectiveness standards set by medicines regulators such as the European Medicines Agency (EMA). Results from these trials have been widely reported pretty much everywhere, including within mainstream media. As a clinical trial statistician, I was interested to find out what members of the public thought about the results that were reported, and whether they had been fully understood.
On 29th January 2021, the HEALTHY STATS public involvement group met online to discuss the clinical trial results reported for two COVID-19 vaccines:
- The Pfizer/BioNTech COVID-19 vaccine1
- The Oxford/AstraZeneca COVID-19 vaccine2
The HEALTHY STATS public involvement group is part of the National Institute of Health Research (NIHR); a United Kingdom government agency which funds research into health and care. With this group, we are aiming to improve the information reported from clinical trials on the health benefits of new treatments to patients. Members include public partners between the ages of 20 to 70 of mixed ethnicities and gender.
These are the key insights I learnt from our discussions:
Not everyone understands what vaccine effectiveness means for reducing symptoms of COVID-19 infection
On 18th November 2020, Pfizer/BioNTech announced that their vaccine was ‘95% effective’. Shortly after, on 23rd November, Oxford/AstraZeneca released their results, reporting that their vaccine was up to ‘90% effective’. This, however, was if a half dose/full dose vaccine combination was administered. If a full dose/full dose combination was administered, it resulted in 62% effectiveness. The overall effectiveness across both regimens was 70%.
Overall, the group were impressed with the high vaccine effectiveness numbers reported. However, not everyone fully understood exactly what 95% or 90% effective meant.
So what exactly does ‘95% effective’ mean? If you take two equally large groups of people with diverse characteristics; one group is vaccinated and the other is not: we would expect there to be 95% fewer cases of symptomatic COVID-19 in the vaccinated group.
The discussions highlighted that better communication around what vaccine effectiveness means is required. The public would then better understand how well the vaccines work and how at risk they still are from any mild/moderate/severe symptomatic COVID-19 illness following vaccination. It is important that individuals do not misinterpret how effective the vaccines are for reducing the symptoms and the high levels of protection they provide because it might lead them to question whether to have the vaccine.
More broadly, beyond COVID-19, this emphasised the need for companies to better explain the results of their research in lay language that everyone will understand, and this in turn will increase the trust in published results. This should be factored into good publication practice as a core principle.
Number of severe COVID-19 cases and deaths during the trial are important public information
The numbers of severe cases and deaths from COVID-19 within the trials and how these differed between the treatment groups were of key importance to the public involvement group.
In the Pfizer/BioNTech trial, from the day of being vaccinated, there were nine cases of severe COVID-19 in the placebo group (the group who did not receive the actual vaccine within the trial to act as a comparison to the group that did) and one in the vaccine group. Whereas in the Oxford/AstraZeneca study, from 21 days after being vaccinated, all 10 severe COVID-19 cases and one death were in the placebo group. Therefore, there was either no, or an extremely low rate of, hospitalisation or death in those with severe COVID-19 after vaccination and no deaths occurred as a result of the vaccine in either study.
Should statistics from different trials be compared?
Some members of the public involvement group initially felt the statistics could be compared. This revealed that the differences between the trials aren’t widely appreciated. The two trials included different populations of people: Pfizer/BioNTech included participants aged 16yrs+ from USA, Argentina, Brazil, South Africa, Germany and Turkey; while the Oxford/AstraZeneca included participants aged 18+ from UK and Brazil. Different types of content for the placebo were used: Pfizer/BioNTech used a saline solution placebo; Oxford/AstraZeneca used a saline solution or meningitis vaccine. Illness was measured by Pfizer/Biontech from seven days after the 2nd vaccine, whereas Oxford/AstraZeneca from 15 days after the 2nd vaccine.
Direct comparison of results from different trials can be misleading as the numbers don’t factor in these differences. Public sources of information, for example the media, should avoid comparing results from different trials to ensure people do not incorrectly perceive that one vaccine works better than the other. Standardisation across trials would help but is not always possible due to diversity within recruited populations.
Safety is paramount to the public
It was equally important for the public involvement group to know the vaccine was both safe and effective. Whilst initial reactions on the vaccines effectiveness were good, there were concerns about the lack of reported information on side effects and how severe these might be.
Only recently we saw the impact of blood clotting concerns on the vaccine outreach programmes across many countries, so it is clear that transparency and pragmatism must be front and centre when discussing potential side effects. Regulators carefully examined such concerns and were clear that there was no evidence of increased blood clots or any other serious adverse events in the vaccine groups of the clinical trial.3
The clinical trials have shown both vaccines are safe. The public involvement group wanted to see more information on safety communicated publically so that individuals can make better informed decision for themselves. This might help reduce the historical mistrust about fears on new healthcare interventions.
Who have the vaccines been tested on?
The group also wanted to know more about the differences of those given the vaccines in the trials, to know how appropriate the results were for them individually. For example, Pfizer/BioNTech indicated that their trial included ‘42% with diverse backgrounds,’ but knowing the ages and ethnicities of participants in trials is important to the public.
Whilst information on participant characteristics and further safety data is available in scientific papers, the group said further work is required to make this information more digestible and available to the public. This is important in ensuring the public trust that their gender, age, and ethnicity is represented and that they can therefore agree that the data applies to them. This could overcome many of the challenges associated with perceived lack of vaccine belief amongst minority communities.
Why does it matter?
Never before have the results of clinical trials been so eagerly awaited. I was personally delighted when the results of these trials were released, showing overwhelming evidence of effectiveness and safety. However, vaccine (as well as general medicine) uptake, is intrinsically dependent on public understanding, trust and confidence and our way out of this pandemic relies on communication of scientific information like never before. It is no longer a ‘nice to have’ but a necessity as the public are called to receive their vaccinations, and make their own decisions about whether to accept. Clear and understandable information relating to the vaccines, and the vaccine trials, is crucial at this time in point, and will help citizens to make better, informed decisions which will ultimately lead to better outcomes for all.
European Medicines Agency. 2021. EMA recommends first COVID-19 vaccine for authorisation in the EU – European Medicines Agency. [online] Available at: <https://www.ema.europa.eu/en/news/ema-recommends-first-covid-19-vaccine-authorisation-eu> [Accessed 30 March 2021].
ac.uk. 2021. Oxford University breakthrough on global COVID-19 vaccine | University of Oxford. [online] Available at: <https://www.ox.ac.uk/news/2020-11-23-oxford-university-breakthrough-global-covid-19-vaccine-0> [Accessed 30 March 2021].
European Medicines Agency. 2021. COVID-19 Vaccine AstraZeneca: benefits still outweigh the risks despite possible link to rare blood clots with low platelets – European Medicines Agency. [online] Available at: <https://www.ema.europa.eu/en/news/covid-19-vaccine-astrazeneca-benefits-still-outweigh-risks-despite-possible-link-rare-blood-clots> [Accessed 30 March 2021].