Variation in the false positive rate

There are numerous factors that can contribute to a false positive result.  We understand some more than others.  Error can creep in right from the moment the swab is opened until it passes quality control checks in the lab.  Examples we understand include other RNA being present from a different source and cross contamination. Each source of error may be tiny but there are enough of them that they add up to a significant number.

Contaminant viral RNA can be found because of viruses which many of us carry around obliviously (3). Symptomatic patients may have a cross reaction with another coronavirus which has been reported with SARS 1 (2). Contaminant human DNA from the X chromosome has produced false positive results for other coronavirus PCR testing (4).  (Interestingly, women seem far more likely than men to have tested positive.  It would be good to see how this has changed over time). A common cause of false positive rates in the real world is cross contamination.  However, there is no suggestion of incompetence.  For the kinds of false positive rates that we are getting there is clearly minimal risk of this happening but the risk cannot be reduced to zero.

Through experimentation we can label a particular test with a percentage of false positive results that can be expected.  This is a constant so long as all the variables are constant.  However, that does not happen in real life.  There will be day to day variation but the mean over a longer time period will be the same.

Similarly, the number will be a constant for a specific population but could be different for another population.  For example, a symptomatic population is more likely to carry RNA from other viruses that could cross react than an asymptomatic one.

For the above reasons, the Pillar 4, random testing of the public, can be expected to be lowest and seems to be around 0.05%.  The Pillar 2 testing of patients in the community will include many symptomatic cases and the false positive rate for this testing seems to have settled around 0.8%.  The rate for patients and staff in hospitals will include a smaller proportion of symptomatic patients and the false positive rate falls between the two at 0.4%.  Each of these figures has a range and the overall day to day variation seems to be between 0.2% and 1.5% overall. Patients testing antibody positive in Pillar 3 tand 4 testing are reported in the Pillar 1 and 2 data.  Without being able to see the raw data for PCR testing alone these estimates of false positive rates may be inflated.  Having said that the numbers being identified by Pillar 3 and 4 are relatively low.  

It has been suggested that the discrepancy is largely due to Pillar 4 testing being subject to additional scrutiny. Specifically, requiring more than one positive result before reporting as positive.  I have not found any evidence that that is the case but this would be one way of improving on the false positive rates.  It will not get it to zero though as sometimes the reason for the false positive will be constant for both samples.

Between April and June the PIllar 4 results showed a constant ratio of symptomatic to asymptomatic patients which was used to claim that these must all be true positive cases (1). However, nothing is perfect.  If the ratio of symptomatic patients to asymptomatic patients is the same for true positives and false positives then there will be no difference.  The positive results from Pillar 4 show a certain amount of noise over time until June, whereupon the level is absolutely constant.  This is a good indication that a false positive rate has been reached. Assuming a false positive rate of absolutely zero is not credible.

Extrapolating from the range above, when testing 200,000 patients the false positive results could be anywhere from 400 to 3000 a day.  On 19th August we had probably reported 60,000 false positive results out of a total of 324,000 tests.  By the beginning of December, when seasonal coronaviruses rear their heads, we could have amassed an extra 170,000 or more false positive cases.

.  

References

  1. https://www.medrxiv.org/content/10.1101/2020.07.06.20147348v1.supplementary-material
  2. https://pubmed.ncbi.nlm.nih.gov/26077969/
  3. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2095096/
  4. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7128111/

Published by clarecraigfrcpath

I have been a pathologist since 2001 and a Consultant Pathologist since 2009. I worked for Imperial College Healthcare Trust as a cytopathologist until 2015. Subsequently I was the day to day Pathologist on the cancer arm of the 100,000 Genome Project and more recently I have worked for Panakeia, an AI digital pathology startup as Head of Pathology. I am currently between jobs and doing occasional consultancy work and crunching COVID data.

2 thoughts on “Variation in the false positive rate

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: