22 April 2020 Blog Post: Los Angeles County Seroprevalence Survey. Please Make It Stop

Honestly, I now spend more time battling misinformation and poorly constructed clinical studies than I do actually communicating useful information. Earlier this morning I wrote about flaws in commercially available antibody testing. I deliberately did not write about Stanford’s misleading Santa Clara study as this has been covered nicely elsewhere (see: Feud over Stanford coronavirus study: The author owe us all an apology)

But now in Los Angeles we have our own misleading study to discuss. Yesterday morning, a study out of the University of Southern California on a sampling of residents in Los Angeles reports that hundreds of thousands of adults here may have already been infected. As of Monday, Los Angeles County had recorded fewer than 13,000 cases. From this study in Los Angeles, the estimated infection rate ranged from 2.8% to 5.6%. The Los Angeles Times then immediately reported that this figure is 55 times higher than the reported official case count.

So apparently the new model in COVID-19 science is to release study reports to journalists without publishing the methodology of the study. Or having the study peer reviewed.

So the first methodologic critique is study sampling. According to the press release, USC researchers and Public Health officials conducted drive-through antibody testing April 10th and 11th at six sites. Participants were recruited via a proprietary database that is representative of the county population. The database is maintained by LRW Group, a market research firm. According to their website – LRW is “an analytics-driven marketing services company powered by data, sophisticated analytics, and deep human understanding.” And not a single epidemiologist in senior management. So who precisely was recruited and, of those, how many agreed?

Second, the USC study used a test manufactured by Hangzhou Alltest Biotech which is the same test used in the Stanford study critiqued above. An independent study(Lassauniere et al. from Statens Serum Institut and Copenhagen University Hospital in Denmark) found this to be the least reliable immunoassay of 9 that they evaluated. It is so unreliable that the UK, who had ordered 3.5 million of the tests, will not even use it. The Danish study suggested a a specificity of 87%. Researchers did not test sensitivity further because it performed so poorly. A separate study reported testing sensitivity of 88.7% and specificity of 90.3%. The Alltest Biotech testing kit was shown to cross react with antibodies to Influenza A, Influenza B, adenovirus, and dengue.

So lets work the numbers. The USC/LA County survey included 863 adults in Los Angeles County. They estimate 4.1% of the county’s adult population has antibody to the virus which means that they had 35 positive tests. Using the published sensitivity and specificty numbers of the Alltest Biotech kit, we can then accurately and mathematically critique this study.

Now, the test performs well when it is negative as its negative predictive value is 99.5% (95% confidence interval: 98.6% to 99.8%). This is the probability that subjects with a negative screening test truly don’t have the disease.

This test performs terribly when it is positive as its positive predictive value is 72.1% (95% confidence interval: 62.6% to 79.9%). This is the probability that subjects with a positive screening test truly have the disease.

This potential error in the test can easily dominate the results, especially in such a small sample size.

Statistician John Cherian of D. E. Shaw Research, a computational biochemistry company, made his own calculations given the test’s sensitivity and specificity — and conservatively estimated the proportion of truly positive people in the Stanford study to range from 0.2% to 2.4%. I would suspect that Los Angeles County has similar true seroprevalence rates. But faulty underlying data on 863 people in a County of over 10 million hasn’t stopped wide and sweeping conclusions.

Dr. Sooj (USC Professor and lead author) – “The estimates also suggest that we might have to recalibrate disease prediction models and rethink public health strategies.”
(I disagree)
 
Dr. Simon (Chief Science Office at LA County) – “Though the results indicate a lower risk of death among those with infection than was previously thought, the number of COVID-related deaths each day continues to mount, highlighting the need for continued vigorous prevention and control efforts.”
(I agree with the second half of his statement but it is frankly dangerous to communicate a lower mortality rate giving people a false sense of security).
 
Dr. Ferrer (Director LA County Health Department) – “These results indicate that many persons may have been unknowingly infected and at risk of transmitting the virus to others. These findings underscore the importance of expanded polymerase chain reaction (PCR) testing to diagnose those with infection so they can be isolated and quarantined while also maintaining the broad social distancing interventions.”
(So based on the results from this testing modality you are suggesting we use another testing modality to find out what is really going on?)
 
I think we need some sort of cease fire on misleading COVID-19 studies. At a minimum, they need to be submitted to a peer-reviewed journal and critiqued before issuing a press release.

𝗦𝗶𝗴𝗻 𝗨𝗽 𝗳𝗼𝗿 𝗢𝘂𝗿 𝗡𝗲𝘄𝘀𝗹𝗲𝘁𝘁𝗲𝗿

Dig deeper into the health topics you care about most by signing up for our newsletter.

by submitting this form you indicate you have
read and agree to our Privacy Policy and Terms
of Use. Please contact us to for us for more
information.

Recommended Posts

No comment yet, add your voice below!


Add a Comment

Your email address will not be published. Required fields are marked *