Table 1 Modified GRADE for diagnostic accuracy studies
Factors that determine and can decrease the quality of evidence | Explanations and how the factor may differ from the quality of evidence for other interventions |
|---|---|
Study design | Cross-sectional or cohort studies in patients with diagnostic uncertainty and direct comparison of test results with an appropriate reference standard (best possible alternative test strategy) are considered high quality and can move to moderate, low or very low depending on other factors. |
Risk of bias (limitations in study design and execution) | Representativeness of the population that was intended to be sampled. Patient selection: consecutive or random sample of patients enrolled? Case-control design avoided? Did the study avoid inappropriate exclusions? Independent comparison with the reference standard. All enrolled patients should receive the index test and the reference standard test. Diagnostic uncertainty should be given. Is the reference standard likely to correctly classify the target condition? Flow and timing: was there an appropriate interval between index test(s) and reference standard? |
Indirectness Patient population, diagnostic test, comparison test and indirect comparisons of tests | The quality of evidence can be lowered if there are important differences between the populations studied and those for whom the recommendation is intended (in prior testing, the spectrum of disease or co-morbidity); if there are important differences in the tests studied and the diagnostic expertise of those applying them in the studies compared to the settings for which the recommendations are intended; or if the tests being compared are each compared to a reference (gold) standard in different studies and not directly compared in the same studies. Panels assessing diagnostic tests often face an absence of direct evidence about impact on patient-important outcomes. They must make deductions from diagnostic test studies about the balance between the presumed influences on patient-important outcomes of any differences in true and false positives and true and false negatives in relationship to test complications and costs. Therefore, accuracy studies typically provide low quality evidence for making recommendations due to indirectness of the outcomes, similar to surrogate outcomes for treatments. |
Important Inconsistency in study results | For accuracy studies unexplained inconsistency in sensitivity, specificity or likelihood ratios (rather than relative risks or mean differences) can lower the quality of evidence. |
Imprecise evidence | For accuracy studies wide confidence intervals for estimates of test accuracy, or true and false positive and negative rates can lower the quality of evidence. |
High probability of Publication bias | A high risk of publication bias (e.g., evidence only from small studies supporting a new test, or asymmetry in a funnel plot) can lower the quality of evidence. |