You may have seen press coverage of Dr. Anthony Fauci announcing preliminary findings from a clinical trial of remdesivir in critically ill COVID-19 patients. This was one of several investigations looking at the role of this antiviral medication. The FDA today approved the medication by moving at self professed ‘lightning speed’ (I wasn’t aware that the FDA was capable of such agility). These results have not yet been published in a peer-reviewed journal.

When critiquing any clinical trial result, there are several things that need to be evaluated. The first is biologic plausibility or, in other words, does it make sense for the medication to work? For this remdesivir certainly seems like a good candidate. It is an antiviral medication developed by Gilead Sciences and works by inserting itself into viral RNA chains, causing their premature termination. COVID-19 is an RNA virus. Remdesivir was originally developed to treat Ebola but did not produce survival benefits seen with two other drugs, and was dropped as a therapy. A different application of a medication that was ineffective in its initial development application has been welcome news to Gilead and its stockholders.

The second aspect to look at is the study design for which the NIH study (to which Dr. Fauci refers), also checks out. Researchers performed a randomized, placebo-controlled trial (which is the gold standard of clinical investigations) in nearly 1100 patients hospitalized with COVID-19 with lung involvement, including those requiring supplemental oxygen or mechanical ventilation.

But it is here where things go a bit off the usual course. Dr. Fauci reported an “interim analysis” which is exciting insofar as it found that recovery was 31% faster with remdesivir than placebo (11 vs. 15 days). Further, the mortality rate was 8.0% with remdesivir treated patients and 11.6% with those receiving placebo (p=0.059). This sounds promising, but frankly is not a slam dunk.

A clinical trial is concurrently watched by a behind the scenes data safety and monitoring board. This is an independent group of clinicians, statisticians and patient advocates who evaluate both patient safety and treatment effectiveness during the trial. They only intervene if the medication is clearly causing harm to patients or if it is so obviously beneficial that continuing the trial would be unethical to those receiving placebo.

We generally define statistical ‘significance’ by a p-value and, by convention, this value should be less than 0.05. Or, in actual English, the likelihood of seeing these results simply by chance is less than 1 in 20 (5% or 0.05). There is no magic to this number, but it is helpful conceptually. A mortality rate of 8.0% with remdesivir and 11.6% with placebo does not attain this level of statistical significance as its reported p-value is 0.059. This is a pretty unusual time to stop the trial, because generally a board would recommend stopping only if the p-value was ≤ 0.0002 (1 in 5000 likelihood by chance) with one-third of the study data available, or ≤ 0.012 (1 in 83) with two-thirds available.

Lastly, results presented by Dr. Fauci do not square with a smaller randomized placebo-controlled trial of remdesivir from China published the same day. It including 240 adults in China with severe COVID-19 and showed no difference in the time to clinical improvement. In fairness, the study was cut short as it aimed to enroll roughly 450 patients, but recruitment was stopped early as the outbreak came under control in Wuhan. This left the study ‘underpowered’ – or in other words it lacked a sufficiently large sample size to answer the research question of interest.

The fact that we have a medication that may (emphasis on may) be helpful in severe COVID-19 cases is welcome news. But we have missed a huge opportunity to learn more. Specifically, we have no information on which patients the drug could help. Are older or younger patients likely to benefit? How do different pre-existing illnesses tip the risk-benefit ratio? Instead, we got what Memorial Sloan-Kettering Cancer Center physician and epidemiologist Peter Bach calls “the bare minimum of information” — just enough to drive the drug over the regulatory hurdles needed for approval. Now there are ethical barriers to any future placebo-controlled studies which could advance our knowledge. So rushing remdesivir through its early trials might help some patients in the short term, but may indeed may be detrimental to others over the months to come.