How we know vaccines are safe and effective

How often have you read the entire leaflet before taking a paracetamol or giving it to your child? Have you ever wondered where that information comes from, and how do we know it is reliable?

All new vaccines or drugs have to be approved for human use before they can be provided to those who need them. The pathway from development to approval is rigorous. Vaccines or drugs need to be carefully tested for safety (to ensure that they cause no harm) and efficacy (to evaluate if they actually work) before making them available for the general population. The testing is done first in animals, and if this proves successful, the product can then be tested in humans (clinical trials).

There are three phases of clinical trials, known as phases 1, 2 and 3. All the phases must be justifiable and are strictly regulated for scientific value, rigour and to ensure compliance with ethical values governing research in human volunteers (see below for more detail).

Random allocation

The only way to know if a vaccine is safe and if it can prevent a disease is to conduct large trials that measure how often people who got the vaccine experience side effects and how many are affected by the disease. The best approach, which provides the most reliable results, is to randomly (by computer) allocate trial participants into one of two groups, so that one group receives the vaccine and the other receives a placebo, such as salt water or a different vaccine.

It is also important that the study volunteers, the health care workers and scientists who conduct the trial, and laboratory personnel who perform tests all do not know which trial participants received the placebo or the vaccine. This “blinding” ensures that the conduct of the trial is not affected by biases or changes in behaviour that could come from knowing which group a participant was allocated to.

The name for this type of trial is a placebo-controlled, blinded, randomised trial. These trials need to include a sufficiently large number of persons from the same community to be able to see if there are any differences between the safety and efficacy results between the vaccinated and the placebo groups.

Phase 1 trials

Phase 1 trials are usually the first time the vaccine is given to human volunteers, primarily to assess safety in a small group (10–100) of healthy adults. Safety follow-up may be for up to 12 months, to check for short- and long-term side effects. Initial data about the immune responses that the vaccine induces (immunogenicity) may be collected.

Phase 2 trials

If phase 1 testing shows the product is safe, phase 2 trials may commence, typically enrolling larger numbers (100–1000) of volunteers, to measure safety and immunogenicity. Phase 2 trials also allow comparison of different doses and/or formulations so that the best one can be tested in the follow-up phase.

In phase 2, participants may also include a broader age range, or volunteers who may have other conditions or co-morbidities, such as diabetes, hypertension or even be immunocompromised. Some phase 2 trials can start to assess efficacy, although these trials are usually not large enough to provide definitive results on how well the product works. Still, such trial may provide a “proof of concept” to inform further testing. 

Phase 3 trials

If phase 2 results show the vaccine is safe and induces a good immune response, phase 3 trials in tens of thousands of volunteers can be conducted, usually at multiple sites around the world. The main objectives of phase 3 trials are to measure efficacy (how well the product works) and to gather further safety data, for example on rare side effects.

Accelerated clinical development 

Usually it takes 10–15 years to move a new vaccine candidate or drug through these three phases before it is licensed. Given the enormous cost of these trials, they usually do not happen concurrently as funders would wait for results of each phase to decide whether pursuing the testing in the next phase is warranted.

In addition, large scale production of the vaccine usually starts after successful testing in phase 3 trials. Given the global scale and severity of the Covid-19 pandemic, we have seen unprecedented co-operation, collaborations, accelerated planning and investment in the different phases, so that preliminary data from phase 3 trials have been generated within a year from the discovery of the virus and manufacturing scale-up has been undertaken while the trials are still ongoing.

Standards and regulations for clinical trials

Some people express concern that clinical research involving new vaccines and drugs could be biased or manipulated by those who have economic interest in selling the product, changing or falsifying the results to make them look better, to increase profitability. While there are examples of such abhorrent and unethical behaviour in the past, ever more stringent regulations and structures are in place to protect against this today.

The trial design features described above (controlled, blinded, randomised) are the first layer of protection against such biases, but a number of additional standards and regulations for clinical trials, described below, provide further protection. In addition, compensation of researchers who conduct the clinical trials is not affected by whether the product is found to have efficacy or to be safe or not.

All clinical trials are conducted under very strict conditions, in a manner that is decided in all its detail before the study starts. The exact trial design is written down in a document, the study protocol, which describes all activities and measurements performed in the trial as well as how the results will be analysed and reported.

Trials must be registered

All trials must be registered on a publicly accessible clinical trial registry, such as the South African National Clinical Trials Register (SANCTR) or clinicaltrials.gov. The trial protocol and associated documents have to be reviewed and approved by a number of independent committees, including scientific review boards, ethics committees and the regulatory authority (SAHPRA in South Africa), before the study starts. These committees and authorities have to be independent and cannot be involved in the study itself.

The trial protocol is also typically provided, as supplementary material in an appendix, when a clinical trial is published in a scientific journal (discussed below), so that the reader can cross-check and review the conduct of the trial.

Rights and safety of volunteers

The rights and safety of the volunteers are always of the utmost importance and all trials must adhere to established international ethical and scientific quality standards for the design, conduct, recording and reporting of clinical research involving the participation of human participants. The conduct of the trial is pre-defined and cannot be changed. In addition, independent organisations routinely audit the research teams that conduct trials, their premises and laboratories, to ensure that standards and regulations are adhered to.

Possible side effects

Data about any possible side effects and efficacy are collected in a standardised and predefined manner for a certain amount of time in all participants. In the case of vaccines, the number of people who get infected or get sick from the disease are counted, as are the number of severely ill participants, those who require hospitalisation, or died.

Once all the data have been collected and recorded in a database that cannot be modified (the database is locked), the blinding is broken to reveal who received the vaccine and who received the placebo. If the proportion of participants who got sick from the disease is significantly lower in the vaccinated group compared to those who received the placebo, then the vaccine is considered protective, or efficacious.

The vaccine efficacy, i.e. how well the vaccine protects, is therefore a measure of how big the difference is between the number of people who get sick between the two groups. The side effects are also compared between the two groups, to determine if the safety of the vaccine is acceptable. Normally, vaccines induce mild side effects such as pain at the injection site, some fever or muscle pain (like flu), which are completely normal signs that our body is reacting to the vaccine and generating an immune response.

In exceptional circumstances, such as for vaccines to prevent Covid-19, a preliminary analysis may be conducted while the trial is still ongoing to provide an indication of whether the vaccine is safe and efficacious. Despite the unprecedented speed at which vaccines to prevent Covid-19 are being tested in clinical trials, all the standards and regulations mentioned above still apply.

Publication and dissemination of scientific results

To share the findings from a trial, it is necessary to write a research article that describes the background and problem that the drug or vaccine seeks to address, the objectives of the study, the methods used to perform the trial, the results of the trial as well as the interpretation of the findings.

Such articles can be immediately posted to freely accessible online repositories as “pre-prints”, which makes the information immediately available to anyone. However, pre-print articles have not been reviewed, scrutinised or verified by experts in the field. 

Publication of research articles in specialised scientific journals is subject to a process of “peer-review”, where the manuscript is evaluated by editors of the journal and then sent to several independent experts in the field, who were not involved in the study, to assess the validity and potential impact of the results.

For articles that report the results of clinical trials the peer-review process also ensures that the researchers adhered to the clinical trial protocol, that the trial was conducted in a rigorous and ethical manner, that the analyses of data are appropriate and valid and that the interpretation of the findings is sound. This is a critical part of the scientific process and ensures that flawed or erroneous studies are not published in the scientific literature.

During the last year, a number of Covid-19 studies reported as pre-prints (for example, some studies on effects of hydroxychloroquine) were amplified and sensationalised by mainstream media, public figures and even some scientists before they had passed peer-review, only to be shown as flawed or erroneous. 

Licensure of vaccines

Authorisation of a product for human use requires careful review of all available safety, efficacy and immune response data, trial reports, and dossiers on the development of the product. These are submitted by the manufacturer to a country’s regulatory authority (SAHPRA in South Africa) for evaluation of all available evidence.

Licensure is dependent on clear demonstration that the benefits of new medicine (i.e. efficacy) outweighs the risks of administering the medicine. Since vaccines are given to healthy people before they are infected with a germ, the benefits should markedly exceed risks.

On the other hand, for drugs that are given to people who are already sick, more side effects may be acceptable, if the overall benefits are significant, for example preventing death, reducing hospitalisation or more severe symptoms. SAHPRA has committed to expediting (putting to the top of the list) review of all Covid-19 vaccines and treatments. 

Emergency use of a medicine

In an emergency situation, be it for an individual or a community, applicants can apply for use of an unregistered medicine or vaccine to the Section 21 Unit. This may for example be for a life-saving cancer treatment for an individual using a new product not yet licensed in South Africa, or, for example, for a make of vaccine not registered in South Africa when the usual source becomes unavailable on a long-term basis. This was, for instance, necessary to ensure that the BCG vaccine (the vaccine for TB) could be provided during a global BCG shortage in recent years. 

Based on reasonable quantity and quality of data from ongoing phase 3 trials, the first Covid-19 vaccines were approved for emergency use less than a year after SARS-CoV-2 was identified. In South Africa, SAHPRA has recently approved the AstraZeneca-Oxford vaccine for emergency use.

Post-licensure surveillance of medicines

Even if a regulatory authority’s decision to allow marketing of a new product is based on randomised controlled trials involving thousands of people, these cannot possibly represent every single individual in a population, and there could be side effects so rare that they did not occur, or were not detected, during clinical trials.

For these reasons, when a new vaccine or medicine starts to be used in the general population, there are surveillance systems in place to pick up any concerning side effects that did not occur during the research studies and to continue evaluating safety and efficacy.

In addition, once it is given to the general population, the vaccine or drug may work less well compared to what was observed in strictly monitored clinical trials. This is known as “effectiveness”, as compared to the measured “efficacy” in trials. Data on the effectiveness of Covid-19 vaccines are starting to emerge in countries where the vaccines have been rolled out, such as Israel, Brazil, the USA and the UK. 

All the available information concerning a vaccine or a drug’s use, as well as possible known side effects, are summarised in the leaflet included in every medicine’s packaging. The information included there comes from years of research and is constantly updated if new information becomes available once the medicine is in general use.

Written by Dr Benjamin Kagina, PhD, Senior Research Officer, Research in Evidence-Based Vaccinology; Associate Professor Elisa Nemes, PhD, Clinical Immunologist; Dr Michèle Tameris, MBChB, Senior Clinical Researcher; Dr Virginie Rozot, PhD, Research Officer, Clinical Immunologist; and Professor Thomas Scriba, PhD, Clinical Immunologist.