Appendix B

Measures of Diagnostic Performance: Sensitivity, Specificity, and Predictive Value

SENSITIVITY

Sensitivity is a measure of the ability of a diagnostic test, or the application of diagnostic criteria, to correctly detect disease when the disease is present. More technically, Sensitivity is the probability a test is positive when the disease is present. Until there is an external reference standard (e.g., biomarker) for Long COVID (or sub-phenotypes), the diagnosis of Long COVID is based on the patient-reported symptoms as per the 2024 NASEM definition. It is therefore not possible to calculate the sensitivity of the 2024 NASEM definition.

Se represents the diagnostic measure of sensitivity.

D+ represents the presence of disease.

T+ represents a positive test result, or equivalently, that a set of diagnostic criteria, represented by T, are satisfied, as represented by T+.

P[T+|D+] is the probability a test is positive when the disease is present. Equivalently, this is the probability a set of diagnostic criteria are satisfied when the disease is present.

By definition, Se = P[T+|D+].

SPECIFICITY

Specificity is a measure of the ability of a diagnostic test, or the application of diagnostic criteria, to correctly rule out disease when the disease is absent. More technically, specificity is the probability a test is negative when the disease is absent. Similar to sensitivity, it is not possible to calculate the specificity of the 2024 Long COVID NASEM Definition until an external reference standard is identified.

Sp represents the diagnostic measure of specificity.

D- represents the absence of disease.

T- represents a negative test result, or equivalently, that a set of diagnostic criteria, represented by T, are not satisfied, as represented by T-.

P[T-|D-] is the probability a test is negative when the disease is absent. Equivalently, this is the probability a set of diagnostic criteria are not satisfied when the disease is absent.

By definition, Sp = P[T-|D-].

Importantly, the ability to measure sensitivity and specificity depends on an independent truth standard for the presence or absence of the disease in question. When there is no independent truth standard, such as pathology or a definitive biomarker, there is no definitive way to measure sensitivity and specificity. This is the case currently with Long COVID. Given state of knowledge at the time of writing this report, the sensitivity and specificity of the 2024 NASEM Long COVID Definition is unknown.

PREDICTIVE VALUE

Predictive value is a measure of the ability of a diagnostic test, or the application of a set of diagnostic criteria, to correctly detect disease (Positive Predictive Value) or to correctly rule out disease (Negative Predictive Value) when the presence or absence of disease is uncertain. More technically, Positive Predictive Value is the probability the disease is present when a test is positive, and Negative Predictive Value is the probability a disease is absent when a test is negative.

PPV represents Positive Predictive Value.

P[D+|T+] is the probability the disease is present when the test is positive. Equivalently, this is the probability the disease is present when the set of diagnostic criteria are satisfied.

By definition, PPV = P[D+|T+].

NPV represents Negative Predictive Value.

P[D-|T-] is the probability the disease is absent when the test is negative. Equivalently, this is the probability the disease is absent when the set of diagnostic criteria are not satisfied.

By definition, NPV = P[D-|T-].

When applying a diagnostic test, or set of diagnostic criteria, to a group of individuals, the predictive value of the test or set of diagnostic criteria depends on the prior probability of the disease in the group who are tested as well as on the sensitivity and specificity of the test.

P[D+] represents the prior probability of disease in a group who are tested for the disease. This could be considered the background frequency or prevalence of the disease.

P[D-] represents the prior probability of the absence of disease in a group who are tested for the disease. Because a disease is either present or absent, the sum of the probabilities P[D+] plus P[D-] equals 1.0. P[D-] can be represented as (1-P[D+]).

The mathematical relationship between predictive value and the prior probability of disease, test sensitivity, and test specificity is given by Bayes Formula.

Bayes Formula for Positive Predictive Value is:

$P[D^{+}|T^{+}]=\frac{SexP[D^{+}]}{\left(SexP[D^{+}]\right)+\left(1-Sp\right)x\left(1-P[D^{+}]\right)}$

Similarly, Bayes Formula for Negative Predictive Value is:

$P[D^{-}|T^{-}]=\frac{Sex\left(1-P[D^{+}]\right)}{Sex\left(1-P[D^{+}]\right)+\left(1-Se\right)xP[D^{+}]}$

It may be easier to see the relationships among sensitivity, specificity, disease prevalence, positive predictive value and negative predictive value in the form of a 2×2 table.

$Se=\frac{A}{A+B}+P\left[T^{+}|D^{+}\right]$

$Sp=\frac{D}{B+D}+P\left[T^{-}|D^{-}\right]$

$P\left[D^{+}\right]=\frac{A+C}{A+B+C+D}$

$P\left[D^{-}\right]=\frac{B+D}{A+B+C+D}$

$P\left[D^{-}\right]=1-P\left[D^{+}\right]=1-\frac{A+C}{A+B+C+D}=\frac{A+B+C+D}{A+B+C+D}-\frac{A+C}{A+B+C+D}=\frac{B+D}{A+B+C+D}$

The PPV is the fraction of true positives (A) among all positives (A+B), or $\frac{A}{A+B}$ as may be seen directly from the table. The equivalence to the previously given formula can be shown in terms of sensitivity, specificity, and prevalence (that is, the prior probability of disease), as follows:

$PPV=\frac{Se\text{}xP[D^{+}]}{(Se\text{}xP[D^{+}])+(1-Sp)x(1-P[D^{+}])}=\frac{\frac{A}{A+C}x\frac{A+C}{A+B+C+D}}{(\frac{A}{A+C}x\frac{A+C}{A+B+C+D})+(1-\frac{D}{B+D})x(1-\frac{A+C}{A+B+C+D})}=\frac{\frac{A}{A+B+C+D}}{(\frac{A}{A+B+C+D})+(\frac{B}{B+D})x(\frac{B+D}{A+B+C+D})}=\frac{\frac{A}{A+B+C+D}}{\frac{A+B}{A+B+C+D}}=\frac{A}{A+B}$

It may similarly be shown that the previously given formula for negative predictive value in terms of sensitivity, specificity, and prevalence (or prior probability of disease) equals the fraction of true negatives among all who test negative, or $\frac{D}{C+D}$, as represented in the table.

The prior probability of disease strongly influences the predictive value of a diagnostic test or the predictive value of a set of diagnostic criteria. For example, consider a highly inclusive set of diagnostic criteria, that is sensitivity = 0.98, with only middling ability to rule out the disease in question, that is specificity = 0.75. Consider two test populations of 10,000 persons, population A with a very high prior probability of the disease or condition in question, P[D+] = 0.6, and population B with a much lower prior probability of the disease or condition in question, P[D+] = 0.05. The results of applying this identical set of diagnostic criteria in each population would be as follows:

Population A:
Se = 0.98

Sp = 0.75

P[D+] = 0.6

PPV = 5,880/6,880 ~ 0.85

NPV = 3,000/3,120 ~ 0.96

In this case approximately five of six who test positive and 24 of 25 who test negative will be correctly classified with respect to the presence or absence of the disease.

Population B:

Se = 0.98

Sp = 0.75

P[D+] = 0.05

PPV = 490/2,865 ~ 0.34

NPV = 7,125/7,135 ~ 0.99+

In this case, a positive test will correctly classify a patient only in about one third of those with the disease or condition in question, while a negative test will virtually rule out the condition.

One implication is that the application of the same diagnostic criteria when many patients have recently had acute SARS-CoV-2 infection will produce a higher positive predictive value than will result after the acute infection declines in frequency, which means that patients with the same symptoms have a lower prior probability of the symptoms being related to COVID.