Table 3 Modified COSMIN checklist for methodological quality assessment
From: The reliability of WorkWell Systems Functional Capacity Evaluation: a systematic review
Requirements | Excellent | Good | Fair | Poor | |
|---|---|---|---|---|---|
1 | Was the percentage of missing items given? | Percentage of missing items described | Percentage of missing items not described | - | - |
2 | Was there a description of how missing items were handled? | Described how missing items were handled | Not described but it can be deduced how missing items were handled | Not clear how missing items were handled | - |
3 | Was the sample size included in the analysis adequate? | Adequate sample size (≥ 100) | Good sample size (50–99) | Moderate sample size (30–49) | Small sample size (< 30) |
4 | Were the administrations independent? | Independent measurements | Assumable that the measurements were independent | Doubtful whether the measurements were independent | Measurements not independent |
5 | Was the time interval stated? | Time interval stated | - | Time interval not stated | - |
6 | Were patients stable in interim period on the construct to be measured? | Patients were stable (evidence provided) | Assumable that patients were stable | Unclear whether patients were stable | Patients were not stable |
7 | Was the time interval appropriate? | Time interval between test-retest ranges from 3 to 21 days | - | Doubtful whether time interval was appropriate | Time interval between test-retest is less than 3 or more than 21 days |
8 | Were the tests conditions similar for both measurements? e.g., type of administration, environment, and instructions | Test conditions were similar (evidence provided) | Assumable that test conditions were similar | Unclear whether test conditions were similar | Test conditions were not similar |
9 | Were there any important flaws in the design or methods of the study? | No other important methodological flaws in the design or execution of the study | - | Other minor methodological flaws in the design or execution of the study | Other important methodological flaws in the design or execution of the study |
10 | For continuous scores: Was ICC calculated? | ICC calculated and model or formula of the ICC is described | ICC calculated but model or formula of the ICC not described. Pearson or Spearman correlation coefficient calculated with evidence provided that no systematic change has occurred | Pearson or Spearman correlation coefficient calculated without evidence provided that no systematic change has occurred or with evidence that systematic change has occurred | No ICC or Pearson or Spearman correlations calculated |
11 | For dichotomous/ nominal/ordinal scores: Was kappa calculated? | Kappa calculated | - | - | Only percentage agreement calculated |