Malignancy validation in a United States registry of rheumatoid arthritis patients

Rheumatoid Arthritis (RA) is an auto-immune, multi-system inflammatory disease with significant morbidity and mortality. In addition to its articular manifestations, RA has a variety of extra-articular complications, including malignancy [1, 2]. The impact of immunosuppressive treatments used to manage patients with RA may also influence the development of malignancies, as each of these treatments cause perturbations of the immune system that may lead to malignancy [3–6]. Because of the influences of disease activity and therapy on the immune system of patients with RA, as well as the relative infrequent occurrence of many types of malignancy, it can be difficult to assess the impact of a specific medication on the risk of malignancy.

To date, several epidemiologic and pharmacoepidemiology studies have been performed to assess the rate of malignancy in RA patients treated with Tumor Necrosis Factor a inhibitors (TNFi) [7–18]. Sources for these datasets include national registries, prospective cohorts, and administrative databases. Cancer outcomes in these studies have been identified based on national cancer registers, patient self-report, physician report, administrative claims, and medical records. In the United States, common methods for ascertainment of malignancy among RA patients include patient self-report or physician report. Another approach is mining of centralized administrative databases, often using claims data or ICD-9 codes to document presence of malignancy, as in the US, national malignancy registries do not exist. The Surveillance Epidemiology and End Results (SEER) registry does capture data on incident malignancies in the US, but it only collects data for a handful of states as well as some additional, separate urban areas. There are also state specific cancer registries, however, between the overall number of RA patients and the overall low rate of malignancy among RA patients, they often do not have the power necessary for pharmacoepidemiologic studies. In addition, individual patient information is not available in many of these registries to permit linkage to observational registries of other diseases. Outside the United States, in addition to the above methods, many countries have national cancer registries where reporting of all incident malignancies to a central database with confirmation is mandatory.

To classify incident cancers in a large cohort of patients with RA where a cancer registry was not available, primary records were requested to validate the malignancy. We developed an adjudication process and tested the accuracy of physician-reported incident cancers, using pertinent medical records as the gold standard.

Reasons why cases were excluded (N = 57) from categorization as incident malignancies are shown in Figure 1. The most common reason for exclusion was determining, usually via source documents, that there was insufficient data to determine whether an incident malignancy had occurred (43 cases, 75.4% of excluded, 9.3% overall). Six cases were excluded as they were prevalent, but not incident malignancies (10.5% of excluded, 1.3% overall). Five cases were excluded as on corroboration, they were confirmed to not be malignancies (8.8% of excluded, 1.1% overall). Lastly, 3 cases were excluded as they were submitted twice (5.3% of excluded, 0.7% overall). All 234 definite malignancies had corroborating data to confirm date of onset and histology. Of the 69 probable malignancies, 36 had no corroborating records submitted, but had an incident date and history on the TAE form. Of the 101 possible malignancies, 79 had no records submitted. Of the possible malignancies, 60 had a histology submitted but no date of the malignancy. 13 had a date of the malignancy but no histology, and 28 had both no histology and no date.

Accurate classification of incident malignancies is essential for pharmacoepidemiologic studies, as well as to compare rates between different cohort studies and assess the relationship between disease characteristics and treatments with the development of cancer. Patient self-report may be an imperfect method for ascertaining incident malignancies, as the difference between a benign mass and true neoplasm is not always clearly communicated or understood. Even physician report may be unreliable, as the treating rheumatologist may not have the primary information about an incident malignancy. Administrative databases based on claims data are also not ideal, as the methodology used to define malignancy can cause the positive predictive value to vary widely [21, 22]. We attempted to validate physician report of incident malignancy based on physician questionnaires and adverse event forms using a gold standard of pertinent medical records review in a large cohort of patients with RA. We were able to confirm approximately two-thirds of reported incident malignancies. We found that having a positive report of incident malignancy on the adverse event forms increased the PPV only slightly compared to questionnaires alone. However, the PPV of the physician report alone may not be sufficiently high enough to be relied on for epidemiologic purposes. Hence, use of non-validated reports of incident malignancy might overestimate the true incidence in patients with RA and might bias the results of pharmacoepidemiologic studies assessing causal associations with specific treatments.

Our second notable finding was the exclusion of 14 malignancies (3.0%) because they were either duplicate entries, established cancers that were not incident, or proven by corroborating documents to not be malignant. An additional 43 cases (9.3%) had insufficient data and could not even be deemed possible malignancies. Clearly some of those are incident cases, although it is not clear how many. In addition, 101 cases (21.9%) could at best be classified as possible incident malignancies (either histology report or date of diagnosis missing). The accuracy of classifying cancer in a study assessing the risk of different treatments is important. If the possible malignancies are dismissed as non-cancers, it might artificially create a sense of decreased risk for a given medication, when at least some of those cases are likely to be true malignancies. Alternatively, including all possible malignancies might change the perception of the risk/benefit ratio and prevent patients or physicians from choosing a medication that could have benefit. The first approach would provide greater specificity; however, this is at the sacrifice of sensitivity. Assessing both rates would provide a range within which the true value most likely exists, but also illustrates the degree of uncertainty generated by physician report and even follow-up adverse event forms.

Our third significant finding was that despite follow-up with the primary treating rheumatologist, almost a third of the time, we could not obtain source documents to corroborate the malignancy. In some cases, this may be due to lack of diligence by the site. However, in most cases, this was because records truly could not be obtained. As a result, since 2008 the registry now requests source documents at the time the cancer is initially reported, to avoid requesting records of events from multiple years earlier. This proactive approach has improved the proportion of cases with source documents.

The methodology defines the classification of the malignancies, as it does depend on appropriate documentation submitted from the patient’s rheumatologist. The use of appropriate specialists for specific malignancies, such as a dermatologist for a skin cancer or a urologist for a genitourinary cancer, was deemed appropriate as for those malignancies where that specialist will treat the patient directly, as opposed to having an oncologist manage treatment. All ‘possible’ malignancies were classified as such because of some inadequacy of the requisite data to have confidence that it was truly an incident case.

The type of dataset itself also can impact the rate of malignancy found in a given population. The intensity of evaluation is much higher in a clinical trial, and virtually all cases of malignancy during the follow-up period would be expected to be reported. However, Phase II and III randomized controlled trials only follow patients with a single intervention for a finite period, and thus cannot offer the kind of data that can be possible in a long-term disease registry. It is possible that the intensity of surveillance itself can impact clinical care and outcome (Hawthorne effect), but it is still unclear if it will affect the overall incidence of a specific comorbidity such as malignancy in patients with RA.

The strengths of this analysis include the use of a very large dataset which includes information gathered from both providers and patients in the context of an established infrastructure which permitted the identification and follow-up of all reported cases. This allowed us to procure source documents in most cases. In addition, we used a standardized record review process with multiple adjudicators. All of these steps increased the validity of the results.

This study did have certain weaknesses as well. Although standard questions on the development of new malignancies were included in both the physician and patient questionnaires at each visit, it is still possible that some malignancies were not reported. As previously noted, the absence of a national cancer registry in the US makes independent assessment of new malignancies extremely difficult if not impossible. As such, we cannot calculate a negative predictive value of the default evaluation of “no malignancy.” Individual state cancer registries are available, are sponsored by both the NCI and the CDC, and their reliability continues to improve, though they are not reflective of nationwide rates, as the SEER data is. However, identification of individual patients requires the use of personal health information (PHI), and the CORRONA consent does not permit the use of PHI to cross-link with other registries. In addition, despite repeated attempts to obtain source documents, in many cases none were available. This led to the classification of many malignancies as “possible,” even while reported independently on the primary questionnaire from either the patient or treating physician. Certainly many of these cases likely were true malignancies. We believe that this problem is likely to be endemic to all observational registry studies in the absence of a national cancer registry. The response rate for pertinent records did improve when cases were reported to CORRONA more recently. However, there was no significant difference in rate of excluded cases (either not an incident malignancy or confirmed to be not a malignancy) from earlier versus more recent cases. A cancer registry where all malignancies are reported and validated, as is frequently done outside the United States, is superior to our methodology. However, information from within the United States is still of great value, as drug utilization patterns in the US are quite different from European registries where penetration of biologic agents is significantly greater than in Europe[23, 24]. We therefore believe that it is critical to appropriately analyze data on major comorbidites from a US source.

In conclusion, our process of confirming malignancies started with the identification of possible cases reported at the time of a routine clinic visit. It was further refined through subsequent hierarchical steps which included a targeted adverse event form and subsequent review and ranking of available source documents by a team of physicians. As a result of these rigorous adjudication steps, we found that routine reports of malignancy by patients and physicians were not always accurate. The implications of over and underreporting in large disease registry may be epidemiologically significant. We believe that it would be ideal if uniform standards for reporting of these events could be adopted in observational disease registries where cancer registries are not available.