You may have missed the stunning admission from Dr. Deborah Birx yesterday that the White House Coronavirus Task Force is ‘missing’ about 50% of the #COVID-19 testing data. As she stated:

“Well, I’m telling you, I’m still missing 50% of the data from reporting,” she said. “I have 660 (thousand) tests reported in. We’ve done 1.3 million. … So, we do need to see — the bill said you need to report. We are still not receiving 100% of the tests.”

As if the CDC’s inability to obtain data is not maddening enough (it should flow from providers to the local health department to the state health department to the CDC – I dutifully fill out case reports for LA County myself), this lack of data raises the more serious issue of scientific bias. If you are already sufficiently enraged by the CDC’s response to this pandemic, I suggest you stop reading now.

The follow-up question which should have been asked (and would have been asked if there was an Epidemiologist in the room) is “which testing are you missing.” This answer is critical because data may be missing completely at random (MCAR), or missing not at random (MNAR).

So if Dr. Birx is missing 50% of the data and that data is randomly and equivalently distributed across all the testing (MCAR), then conclusions on infection rates, mortality rates, and transmission patterns (“bending the curve”) are still valid. However, if observations are missing preferentially (MNAR) from certain geographic areas, age groups, or are not reported for particular a gender or ethnic group, then any conclusions are biased and rendered useless. Intuitively, in MNAR observed data are of little help, because there is no information to account for the missing data.

This is particularly problematic when making policy and resource decisions based on these flawed data. For instance, a region that has no barriers to reporting to the CDC might show a higher burden of disease than the population at large. Or, it could show a lower burden of disease such that needed ventilators would then be shunted to another region. You just end up guessing.

So while Dr. Birx’s admission that they are missing 50% of collected data on coronavirus testing is stunning, all the more disheartening is that she also does not know which 50% she is missing.