“Over half of new cancer drugs ‘show no benefits’ for survival or wellbeing,” The Guardian reports. That was the finding of a study looking at the evidence supporting new cancer drugs approved between 2009 and 2013 by the European Medicines Agency (EMA).

The study found only half of drug approvals had clear evidence showing they either prolonged people’s lives, or improved their quality of life. That’s not the same as saying these drugs would not help anyone. But research presented at the time of the drugs’ approval, and gathered in the three to eight following years, did not show that they worked better than existing treatments in terms of prolonging or improving quality of life.

The study raises questions about whether medicines regulators should be stricter about the type of evidence they accept when allowing drugs to be marketed. This is especially relevant in the field of cancer treatment (oncology) where a course of treatment with new drugs can cost tens of thousands of pounds.

European regulatory approval is only part of the process in the UK. New medicines are assessed by the National Institute for Health and Care Excellence (NICE). NICE looks more closely at the evidence to see whether drugs give value in terms of improving patient outcomes and quality of life before making a recommendation for it to be prescribed on the NHS.

While the issue of whether these new drugs “work” or not is still a matter of debate, the study highlights the fact that when it comes to medication, “new” doesn’t automatically mean “better”.

Listen to the researchers talk about the study here.

Where did the story come from?

The study was carried out by researchers from Kings College London, London School of Economics and Political Science, Riga Stradins University in Latvia, and the London School of Hygiene and Tropical Medicine. It was published in the peer-reviewed British Medical Journal and is free to read online.

Most of the UK media reported the study accurately.

Somewhat ironically, many of the newspapers reporting on the lack of evidence for these new drugs have previously run articles criticising the NHS for not funding these drugs.

What kind of research was this?

This was a cohort study, which examined evidence submitted to the European Medicines Agency which led to approvals of cancer drugs.

The researchers wanted to see:

what types of studies were being accepted as evidence
how many drug approvals were supported by clear evidence of improvement in length or quality of life
how many drugs approved without this evidence had evidence published after approval
if the evidence around living longer or improved made meaningful difference to patients in real terms

What did the research involve?

Researchers searched for all cancer drug approvals made by the European Medicines Agency (EMA) from 2009 to 2013. They retrieved the European Public Assessment Report (EPAR) for each approval – the document, which summarises the evidence the EMA used to decide to approve the drug. They extracted data about study type and survival and quality of life.

They then searched for studies published since the drug was approved, up to March 2017. Where drugs did show a benefit for survival or quality of life, they used a widely accepted scale to assess how clinically important these results were.

They classified the studies as randomised controlled trials (the most reliable type of study) or non-controlled trials (where there is no control group to compare the effects of the new drug).

They looked at whether researchers measured length of life or quality of life as a primary outcome.

Because studies that show benefits in long-term survival take a long time, researchers often measure secondary outcomes (surrogates) to give a quicker estimate of whether a drug works. These include whether a tumour is shrinking and how fast the disease grows or spreads. While these measures may still be useful, they don’t necessarily translate into longer or better lives for patients.

Three researchers worked on extracting data, and cross-checked each other’s work. Drugs were judged to show evidence they extended life if the trial included overall survival as a primary or secondary endpoint, and showed a difference between the new drug and the control group.

Researchers judged drugs to show improvement in quality of life when there was a difference between the new drug and control group on any item or subscale of a recognised quality of life scale.

They used the European Society for Medical Oncology’s Magnitude of Clinical Benefit Scale (MCBS) scoring system to grade trial results for whether they were clinically meaningful. For example, a drug that extended expected survival time for a terminal cancer by 12 months would be regarded as clinically meaningful.

What were the basic results?

Researchers found 48 cancer drugs had been approved for 68 uses.

At the point where the drugs were approved:

for 24 drug uses (35%), evidence showed that the drug prolonged life
for seven drug uses (10%), evidence showed that the drug increased quality of life
for 39 drug uses (57%), there was no evidence that they either prolonged life, or increased quality of life
In the follow-up period after approval (3.3 to 8 years), new evidence showed that three of the 39 drug indications did increase length of life, and five improved quality of life. This meant that, overall, 35 of 68 drug approvals made by the EMA (51%) had evidence to show improved length or quality of life.

Looking at the figures more closely:

For those drugs that had the evidence available at the time of approval, improvement in length of life ranged from 1 month to 5.8 months. The average improvement in length of life was 2.7 months.
Only two of the 26 drugs shown to extend life also showed improvements in quality of life.

How did the researchers interpret the results?

The researchers say their results show that “European regulators commonly accept the use of surrogate measures of drug benefit as primary endpoints,” in trials submitted as evidence for drug approvals. They say the European Medicines Agency’s standards are “failing to incentivise drug development that best meets the needs of patients, clinicians and healthcare systems.”

They say their analysis shows that “critical information about the outcomes that matter most to patients” might never be gathered, once a drug is approved for use. They say the EMA should “reconsider” its standards.

Conclusion

Most of us assume that when a drug has been approved by a regulator for use, that means it has been shown to work. This study suggests that is not necessarily the case, or that even if it works they might not make a meaningful difference.

The absence of evidence about the two outcomes that matter most to patients and their families – how long they will live, and how good their quality of life will be during that time – from half of the cancer drugs approved during a five-year period, is worrying. Patients cannot be expected to make informed decisions about which treatments to take, without good quality information on these outcomes.

It can be difficult to carry out the best medical research that recruits enough people and follows them for long enough to get all the evidence needed for the drug, particularly for rare cancers.

That’s why people have come to accept the use of surrogate outcome measures, to make research more feasible and get new drugs to people with potentially incurable cancers more quickly in cases where time, or lack of it, is of the essence.

But if surrogate measures are accepted at the time when drugs are approved, it is essential that information about survival and quality of life is collected and published in the following years.

There are, however, some limitations to this study which should be noted:

Researchers didn’t look at how suitable trial designs were. For example, new drugs might be compared to an ineffective or minimally effective drug, rather than to the best care otherwise available. This means that the drug benefits could have been further overestimated.
Researchers only looked at the key trials assessed by the regulators. There may be other trials, published or unpublished, which showed different results.
The studies included in the EPAR assessment reports used varying methods to demonstrate quality of life or length of life.
Some EPAR assessments did not make it clear whether the evidence for the drug showed a true improvement in length or quality of life. In these cases the researchers looked to the EMA’s conclusions or favoured the drug giving them “the benefit of the doubt”. This too may have led to an overestimation of effect.
Overall, the report suggests that regulation of new drug approvals needs to be tighter. As said, drug approval does not automatically mean that it will be recommended as a first-choice option by medical guidelines. NICE looks closely at the evidence to see whether the drug gives value in terms of making meaningful improvements to patient outcomes and quality of life before recommending its use.

Anyone worried about the evidence behind a cancer treatment they are being offered, or are taking, can talk to their cancer specialist and ask them to explain what difference it has been shown to make.