Assay Performance

Limit of detection - analytical sensitivity

The limit of detection (LoD) is defined as the lowest concentration at which 19/20 replicates (or approximately 95% of all true positive replicates) are positively detected. Initial LoD estimates were made based on a dilution series of a synthetic SAR-CoV2 construct brought all the way through the process. The preliminary LoD established by the dilution series was confirmed by testing a total of 20 replicates of the Synthetic SARS-CoV-2 RNA Control at the varying copy levels (Table 1).

Table 1. LoD replicate dilutions of COVID construct

The LoD for this assay was determined to be 60 viral copies into RNA extraction (1.6 copies/µL). 19/20 extraction replicates of RNA construct spiked into human cell controls tested at this concentration (19/20) were positive.

Clinical evaluation

30 positive specimens and 28 negative specimens, from upper respiratory swabs with positive SARS-CoV-2 results or negative test results previously tested using the CRSP v1 assay. Additionally, 20 replicates at ~2X of the LoD were also run through the process from extraction to detection. In order to be considered passing, there had to be 95% agreement at 1x-2x LoD, and 100% agreement at all other concentrations and for negative specimens (Table 2). Both evaluations passed.

Table 2. Summary of clinical evaluation sample runs

Dry swab LoD bridging study

Spun polyester swabs were coated in a negative clinical matrix and allowed to dry. A high viral load positive clinical specimen was diluted to create a range of copies/mL at between 1-3X the LoD of the assay and spiked on to each swab. Swabs were processed through the dry swab reconstitution, extraction, and detection protocols. LoD was determined as the level at which 19/20 replicates were successfully called positive (Table 3). The LoD was confirmed at 60 copies/µL.

Conclusion

The Broad CRSP v2 assay (high throughput) with dry swabs as an input is validated with an LoD of 1.6 copies/µL (or 1600 copies/mL). For context, a study published in The Lancet¹ found median viral load in COVID positive patients at presentation to be 158,489 copies/mL.

Clinical sensitivity

The ability to detect the virus is impacted not just by the performance of the test, but also by the timecourse of the infection in an individual. Researchers have shown² that the longer in time after the onset of symptoms a person is tested, the lower the chance of detecting the virus in that person (as they have largely cleared the virus out of their system).
 

Assay sensitivity and specificity

The absolute sensitivity and specificity of the type of testing we and others are doing for COVID-19 diagnostics are difficult to calculate, as there is no gold standard assay against which we can compare results. In the absence of this, the FDA and clinical best practice guidance is to compare the test on clinical specimens with positive and negative results that have been established using another FDA-approved test. To this end, we initially validated our assay with samples that had been run at another laboratory using their FDA-approved test. This validation cohort included 103 positives and 40 negative samples. Further, 20 of the positive samples were diluted to the established limit of detection of the assay (the lowest concentration of virus at which we see 95% accuracy in test results). In all cases, we saw 100% concordance with the established result. Subsequently, we ran 543 cases through our assay and a collaborating clinical labs FDA-approved assay and again saw 100% concordance of results. In additional studies where we have run a range of clinical samples several times internally, we have seen sensitivity and specificity >95% when we are at or above our limit of detection. Studies have now shown that the infectivity of an individual relates to their viral load and that the viral load and infectivity are both highest in the few days immediately preceding and following the onset of symptoms. When we see a low viral load we believe it is most often a person who has been infected but has largely cleared the virus and may therefore be less infectious.

When there are large numbers of positive individuals in a tested population, the main mode of test failure that we are concerned with is False Negatives (a negative test result for a person who does in fact have the virus). When the tested population has a low viral prevalence then we become more concerned with False Positives (a positive result in a person who does not have the virus). Since our validation cohorts did not show evidence of any false positives we have a high degree of confidence that false positives are not a significant source of error in our assay. Based on data from recent weeks in large cohorts where the prevalence is expected to be very low, we validate this observance. In one institution with >5000 tests run between mid-May and mid-June, we find only 3 positives. In each of the individuals who tested positive we have reason to believe that they were in fact exposed to the viral material being tested for (therefore in testing terms, we consider them True Positives). This empirical data would suggest that the assay False Positive Rate can be no higher than 1/5000 (or 0.02%) and may in fact be lower. Given the significant implications of False Positives, especially within a campus setting, we plan to routinely repeat testing of some proportion of positive results to further ensure the very high specificity of our test.