This article has Open Peer Review reports available.
Internet hand x-rays: A comparison of joint space narrowing and erosion scores (Sharp/Genant) of plain versus digitized x-rays in rheumatoid arthritis patients
© Arbillaga et al; licensee BioMed Central Ltd. 2002
Received: 30 November 2001
Accepted: 30 April 2002
Published: 30 April 2002
The objective of the study is to examine the reliability of erosion and joint space narrowing scores derived from hand x-rays posted on the Internet compared to scores derived from original plain x-rays.
Left and right x-rays of the hands of 36 patients were first digitized and then posted in standard fashion to a secure Internet website. Both the plain and Internet x-rays were scored for erosions and joint space narrowing using the Sharp/Genant method. All scoring was completed in a blind and randomized manner. Agreement between plain and Internet x-ray scores was calculated using Lin's concordance correlations and Bland-Altman graphical representation.
Erosion scores for plain x-rays showed almost perfect concordance with x-rays read on the Internet (concordance 0.887). However, joint space narrowing scores were only "fair" (concordance 0.365). Global scores demonstrated substantial concordance between plain and Internet readings (concordance 0.769). Hand x-rays with less disease involvement showed a tendency to be scored higher on the Internet versions than those with greater disease involvement. This was primarily evident in the joint space narrowing scores.
The Internet represents a valid medium for displaying and scoring hand x-rays of patients with RA. Higher scores from the Internet version may be related to better viewing conditions on the computer screen relative to the plain x-ray viewing, which did not include magnifying lens or bright light. The capability to view high quality x-rays on the Internet has the potential to facilitate information sharing, education, and encourage collaborative studies.
The use of radiographic images as a means of assessing the progression of rheumatoid arthritis (RA) in individual patients and clinical studies has been standard practice for several decades. Underlying this practice is the belief that radiographic changes are the consequence of inflammatory changes intrinsic to RA . Recent discoveries in molecular biology indicate that two cytokines (TNF-alpha, IL-1) are responsible for enhancing cartilage and bone breakdown through their effects on chondrocytes and osteoclasts [2, 3]. Patient treatment decisions and clinical trials will focus extensively on the radiographic progression of the disease as a primary outcome measure [4, 5].
The permanent nature of radiographs facilitates simultaneous comparison of images taken over an elapsed time period. Also, multiple readers can interpret the same set of images, allowing for a greater degree of reliability . The objectivity of scoring can be heightened through masking names and dates on films, and by randomizing the sequence in which films are viewed . The advancement in x-ray technology, coupled with a greater emphasis on the use of standardized patient positioning and radiographic techniques, has further enhanced the reproducibility of scoring .
The Internet has revolutionized the computer and communications world in an unprecedented manner. At once it possesses global broadcasting capabilities, acts as a mechanism for information dissemination and is a medium for collaboration and interaction between individuals without regard for geographic location . New approaches wedding technological advancements in digitizing methods and Internet communication will likely permit clinicians to participate in outcome studies involving radiological progression, particularly if digital radiographs posted on the Internet can be shown to be reliable representations of plain film x-rays.
The aim of this article is to report the reliability of Internet hand x-rays for scoring erosions and joint space narrowing in patients with Rheumatoid Arthritis.
Basic characteristics of the patient sample.
Number of Patients (%)
Rheumatoid Factor, positive
Time from onset of symptoms, years
Time from diagnosis, years
Education Level, grade
Scanning and posting of radiographs on the internet web site
The 36 plain film x-rays were scanned using a Scanmaster DX with a digitizing area from ½" × ½" to 14" × 17" and a maximum film size of 15" × 18". The resolution was 1K, 2K, 4K, and 8K on standard film sizes, and up to 9 lp/mm for custom resolution. The grayscale resolution was 12-bits (4096 gray levels), and the detector for the scanner was a high definition CCD. The interface was SCSI-2.
Scoring of erosions and joint space narrowing
After summing the ERO and JSN scores for both hands, a composite score was created by adding the adjusted ERO and JSN scores together. The minimum value that could be obtained was 0, while the maximum value was 200 (i.e. 100+100).
The statistical analyses were completed using descriptive statistics (SPSS 10.0) and Lin concordance functions  (STATA Rel. 7.0). The Lin co-efficient unites measures of accuracy (i.e. nearness of the data's reduced major axis to the line of perfect concordance) and precision (i.e. tightness of the data about its reduced major axis) to determine whether observed data significantly diverge from the line of perfect concordance, which occurs at 45 degrees. The value of Lin's co-efficient increases in relation to the accuracy and precision of the observed data. The Bland-Altman limits of agreement procedure uses data-scale assessment in analyzing both the accuracy (i.e. bias) and the amount of variation or precision between any two measured values when the range of data is sufficiently limited. This graphical representation approach is complimentary to the relationship-scale approach of Lin.
General X-ray and clinical results
Standard reading of the plain x-rays showed that these films represented a wide range of radiological progression. Erosions ranged from 0 to 93 with average 45.5(SD 29.7). Joint space narrowing ranged from 30 to 92, with average 65.0 (SD 17.0). Global scores ranged from 30 to 183.5, with average 110.5(SD 43.6). This was consistent with the clinical characteristics of the patient population who had average duration from onset of symptoms of 13 years (range 1.5 to 58 years), and who were seropositive in 78% of cases. Eleven patients reported little or no disability, 13 required aids to daily living, and 12 were severely limited in their functional capacity, but were still ambulatory.
Repeated assessment of plain X-rays
To assess intra-rater reliability, the reader, in blinded and randomized fashion, re-scored a randomly selected set of films. Fifteen of 36 plain film x-rays were reassessed using a standard light box, without the aid of a magnifying lens or bright light. The Bland-Altman graphic demonstrated a slight divergence from perfect concordance (accuracy) and high precision (i.e. tightness of the data about the major axis represented in the Lin concordance model as "r" = 0.916). The divergence from perfect agreement can be summarized numerically by the average difference pre-test to post-test. This was -13.6 (95% CI -4.4, -22.8), indicating that the second reading had higher scores on average for the global measure, with the preponderance of differences occurring on films with higher global scores.
Intra-method reliability; plain and internet X-rays
This study examines the reliability of deriving ERO and JSN scores from rheumatoid hand x-rays posted on a secure Internet web site. Although several previous studies have investigated the efficacy of digital x-rays to interpret the musculoskeletal system [11–13] and others compare digital and plain film x-rays for assessing rheumatoid arthritis [6, 14], there has been limited investigation into the reliability of assessing Internet x-ray images of rheumatoid patients.
The anatomic locations selected for scoring the plain and Internet x-rays using the Sharp with Genant modification method were identical to those sites used by Genant et al . The selection of these sites was based on the relative ease of reading and the general frequency of involvement in ERO and JSN in the hand and wrist . The PIP, MCP, and carpal sites selected by Sharp represent the most active areas of hand involvement related to synovial inflammation of rheumatoid arthritis .
The literature suggests that the correlation co-efficient alone cannot be used to determine an agreement between two quantitative variables [16–18]. Pearson's correlation co-efficient has the capacity to measure the strength of association between two variables, but it does not provide information about the concordance of two variables [17, 19–22]. Having acknowledged the inherent limitations of Pearson's co-efficient, it must be stressed that an expectation for two independent readers, or the same reader on multiple trials, to replicate the same ERO and JSN scores on a joint by joint basis is too rigorous . Indeed, the literature supports the notion that it is not necessary for the same absolute radiologic scores to be recorded when application of the scoring-system is carried out by different readers , or when the scoring-system is applied multiple times by the same reader . Rather, it is the association of readings on a global (composite) level that is important for determining agreement with respect to overall disease severity. Concordance  as described by Lin, et al is a more appropriate method of examining both the precision and accuracy of intra-rater and inter-method reliability.
When the total ERO scores were compared as an indication of agreement of disease severity, the concordance was highly significant (0.887). However, interpretation of the Bland Altman graphic (see Figure 7) suggests a trend for patients with low to moderate levels of disease involvement to have Internet x-rays scored higher than corresponding plain x-rays. It has been noted (personal communication with Dr. J. Sharp) that there is a tendency to give higher scores for individuals with low levels of disease involvement, perhaps related to "over-reading" of x-rays with early disease progression. However, this phenomenon would occur in both plain and Internet readings. The Internet version may provide the reader with a basis for giving slightly higher scores at the low end of the scoring scale. For example, a joint that received a Sharp with Genant Modification JSN score of 0 (i.e., normal) when viewed on a plain film x-ray, without the use of a magnifying lens or bright light, could quite easily receive a score of 1 (i.e., mild change) when viewed on the Internet x-ray, which may provide clearer contrast on a monitor with super VGA capacity. This phenomenon may also be accentuated by a "ceiling effect" [24, 25]. Given the greater deterioration that characterizes higher ERO and JSN scores, the fine details detectable on Internet images are not likely to push the score higher than it would have otherwise been. Nevertheless, statistical anomalies must be considered as well; the sample size may have been too small to accurately reflect the true reliability, such that chance variation could also be an appropriate interpretation.
The potential for alteration of the image during image conversion and compression is a concern associated with posting radiograph images to a web site. In particular, the JPEG lossy compression algorithm consists of an image simplification stage that removes image complexity with some loss of fidelity, followed by a compression step, but in the case of images it is generally not critical to restore all of the data upon decompression . Recent technological advances in the area of image compression have led to the development of wavelet compression. With this technology an algorithm is used that converts the image to a mathematical expression and subsequently allows analysis of the image as a whole . Thus, an optimum compression ratio can be reached without comprising image quality. However, this technique was not used in the study. Nor were the JPEG formats compared to TIFF formats. These two points may be worthwhile areas of future research for the technical advancement of radiographic interpretation on the Internet.
Patient security and confidentiality represent another set of concerns connected with the posting and transmission of x-rays via the Internet [28, 29]. Depending on the risks associated with the system and the resources required for minimizing the potential risks an appropriate balance needs to be sought between privacy and connectivity [30, 31]. Internet technology can help to ensure security and confidentiality when transmitting patient identifiers through the use of strong encryption methods, such as is now used for credit card purchases on the Internet [31, 32]. Security also could be guaranteed through public-key algorithms known only to appropriate individuals .
The present study suggests that the Internet represents a valid medium for displaying and scoring hand x-rays of patients with RA. This finding was based on the use of standard JPEG radiograph images. The development of wavelet compression technology, where none of the image information is discarded during the compression process , will further enhance the quality of x-ray images available for posting on the Internet.
As an increasing number of clinical trials emphasize the radiographic progression of the disease as a primary outcome measure [5, 33–35], the capacity to employ a reliable method of scoring radiographs is placed at a premium. X-rays posted on the Internet have the potential to enhance reliability of scoring protocols used in clinical trials through heightened standardization and through the facilitation of information sharing and education. The ease of transmission of images and information over the Internet essentially renders geographic obstacles irrelevant, thereby fostering collaboration. Web sites can be set up with an atlas of x-ray images permitting clinicians around the globe to hone their scoring abilities in a tutorial like fashion. Similarly, the capacity to clearly display and disseminate standard scoring protocol via the Internet has the potential to enhance the consistency of x-ray scoring on a broad scale. Such advantages and opportunities are not available with the use of traditional plain film x-rays. We conclude that coupling Internet technology with standard approaches to radiographic analysis is a necessary step toward the realization of a new spectrum of potential benefits.
Author 1 (HA) performed the scoring of x-rays, and assisted in data management and data analysis. Author 2 (GM) drafted the manuscript, participated in data management and statistical analysis. Author 3 (LC) digitized x-rays and set up Internet web site from which x-rays were assessed. Author 4 (MW) participated in data collection and data management. Author 5 (LM) provided periodic review and design input for the study. Author 6 (SE) conceived of the study, participated in its design, and coordinated the study throughout.
The authors would like to thank Dr. J.B. Houpt, of Toronto, Ontario for his observation regarding the reduced quality of plain x-ray reading environments, in the absence of "bright lighting" and use of a magnifying lens.
- Sharp JT: Radiologic assessment as an outcome measure in rheumatoid arthritis. [Review] [59 refs]. Arthritis & Rheumatism. 1989, 32: 221-9.View ArticleGoogle Scholar
- Gravallese EM, Goldring SR: Cellular mechanisms and the role of cytokines in bone erosions in rheumatoid arthritis. [Review] [92 refs]. Arthritis & Rheumatism. 2000, 43: 2143-51. 10.1002/1529-0131(200010)43:10<2143::AID-ANR1>3.0.CO;2-S.View ArticleGoogle Scholar
- Feldmann M, Maini RN: The role of cytokines in the pathogenesis of rheumatoid arthritis. [Review] [40 refs]. Rheumatology (Oxford). 1999, 38 Suppl 2: 3-7.Google Scholar
- Sharp JT: An overview of radiographic analysis of joint damage in rheumatoid arthritis and its use in metaanalysis. [Review] [41 refs]. J Rheumatol. 2000, 27: 254-60.PubMedGoogle Scholar
- Kremer JM: Rational use of new and existing disease-modifying agents in rheumatoid arthritis. [Review] [87 refs]. Ann Intern Med. 2001, 134: 695-706.View ArticlePubMedGoogle Scholar
- Genant HK, Jiang Y, Peterfy C, Lu Y, Redei J, Countryman PJ: Assessment of rheumatoid arthritis using a modified scoring method on digitized and original radiographs [see comments]. Arthritis & Rheumatism. 1998, 41: 1583-90.View ArticleGoogle Scholar
- Pallen M: Introducing the Internet. [Review] [25 refs]. B M J. 1995, 311: 1422-4.View ArticleGoogle Scholar
- Sharp JT, Lidsky MD, Collins LC, Moreland J: Methods of scoring the progression of radiologic changes in rheumatoid arthritis. Correlation of radiologic, clinical and laboratory abnormalities. Arthritis & Rheumatism. 1971, 14: 706-20.View ArticleGoogle Scholar
- Sharp JT, Bluhm GB, Brook A, Brower AC, Corbett M, Decker JL, Genant HK, Gofton JP, Goodman N, Larsen A: Reproducibility of multiple-observer scoring of radiologic abnormalities in the hands and wrists of patients with rheumatoid arthritis. Arthritis & Rheumatism. 1985, 28: 16-24.View ArticleGoogle Scholar
- Lin LI: A concordance correlation coefficient to evaluate reproducibility. Biometrics. 1989, 45: 255-268.View ArticlePubMedGoogle Scholar
- Richmond BJ, Powers C, Piraino DW, Freed H, Meziane MA, Hale JC, Schluchter MD, Schils J, Gragg LA: Diagnostic efficacy of digitized images vs plain films: a study of the joints of the fingers. AJR American Journal of Roentgenology. 1992, 158: 437-41.View ArticlePubMedGoogle Scholar
- Buckwalter KA, Braunstein EM: Digital skeletal radiography. [Review] [62 refs]. AJR American Journal of Roentgenology. 1992, 158: 1071-80.View ArticlePubMedGoogle Scholar
- Bramble JM, Murphey MD: Comparison of digital and conventional musculoskeletal radiography: observer performance study. [letter; comment]. Radiology. 1990, 177: 587-9.View ArticlePubMedGoogle Scholar
- Jonsson A, Borg A, Hannesson P, Herrlin K, Jonsson K, Sloth M, Petterson H: Film-screen vs. digital radiography in rheumatoid arthritis of the hand. An ROC analysis. Acta Radiologica. 1994, 35: 311-8.PubMedGoogle Scholar
- Sharp JT, Young DY, Bluhm GB, Brook A, Brower AC, Corbett M, Decker JL, Genant HK, Gofton JP, Goodman N: How many joints in the hands and wrists should be included in a score of radiologic abnormalities used to assess rheumatoid arthritis?. Arthritis & Rheumatism. 1985, 28: 1326-35.View ArticleGoogle Scholar
- Ruckmann A, Ehle B, Trampisch HJ: How to evaluate measuring methods in the case of non-defined external validity. J Rheumatol. 1995, 22: 1998-2000.PubMedGoogle Scholar
- Bland JM, Altman DG: Statistical methods for assessing agreement between two methods of clinical measurement. Lancet. 1986, 1: 307-10.View ArticlePubMedGoogle Scholar
- O'Sullivan MM, Lewis PA, Newcombe RG, Broderick NJ, Robinson DA, Coles EC, Jessop JD: Precision of Larsen grading of radiographs in assessing progression of rheumatoid arthritis in individual patients. Ann Rheum Dis. 1990, 49: 286-9.View ArticlePubMedPubMed CentralGoogle Scholar
- Giraudeau B, Ravaud P: Methodologic issues for the assessment of reproducibility: comment on the article by Genant et al [letter; comment]. Arthritis & Rheumatism. 1999, 42: 1556-7. 10.1002/1529-0131(199907)42:7<1556::AID-ANR35>3.0.CO;2-A.View ArticleGoogle Scholar
- Lee J, Koh D, Ong CN: Statistical evaluation of agreement between two methods for measuring a quantitative variable. Computers in Biology & Medicine. 1989, 19: 61-70.View ArticleGoogle Scholar
- Kramer MS, Feinstein AR: Clinical biostatistics. LIV. The biostatistics of concordance. [erratum appears in Clin Pharmacol Ther 1989 Sep;46(3):309]. Clinical Pharmacology & Therapeutics. 1981, 29: 111-23.View ArticleGoogle Scholar
- Muller R, Buttner P: A critical discussion of intraclass correlation coefficients. [see comments]. Statistics in Medicine. 1994, 13: 2465-76.View ArticlePubMedGoogle Scholar
- Mewa AA, Pui M, Cockshott WP, Buchanan WW: Observer differences in detecting erosions in radiographs of rheumatoid arthritis. A comparison of posteroanterior, Nørgaard and Brewerton views. J Rheumatol. 1983, 10: 216-21.PubMedGoogle Scholar
- Wassenberg S, Rau R: Problems in evaluating radiographic findings in rheumatoid arthritis using different methods of radiographic scoring: examples of difficult cases and a study design to develop an improved scoring method. J Rheumatol. 1995, 22: 1990-7.PubMedGoogle Scholar
- Larsen A, Thoen J: Hand radiography of 200 patients with rheumatoid arthritis repeated after an interval of one year. Scand J Rheumatol. 1987, 16: 395-401.View ArticlePubMedGoogle Scholar
- Copeland L, Kay R: Data and image compression. Computerworld. 2000, 34:Google Scholar
- Hill J: JPEG for the millenium. Presentations. 2000, 14: 17-8.Google Scholar
- Menduno M: Prognosis: wired. Why Internet technology is the next medical breakthrough. Hospitals & Health Networks. 1932, 72: 28-30.Google Scholar
- Winker MA, Flanagin A, Chi-Lum B, White J, Andrews K, Kennett RL, DeAngelis CD, Musacchio RA: Guidelines for medical and health information sites on the internet: principles governing AMA web sites. American Medical Association. JAMA. 2000, 283: 1600-6. 10.1001/jama.283.12.1600.View ArticlePubMedGoogle Scholar
- Jadad AR: Promoting partnerships: challenges for the internet age. [see comments]. [Review] [42 refs]. B M J. 1999, 319: 761-4.View ArticleGoogle Scholar
- Edworthy SM: World wide web: opportunities, challenges, and threats. Lupus. 1999, 8: 596-605. 10.1191/096120399680411434.View ArticlePubMedGoogle Scholar
- Parente ST: Beyond the hype: a taxonomy of e-health business models. Health Affairs. 2000, 19: 89-102. 10.1377/hlthaff.19.6.89.View ArticlePubMedGoogle Scholar
- Lipsky PE, van der Heijde DM, St Clair EW, Furst DE, Breedveld FC, Kalden JR, Smolen JS, Weisman M, Emery P, Feldmann M, Harriman GR, Maini RN, Anti-Tumor Necrosis Factor Trial in Rheumatoid Aw, Concomitant Therapy SG: Infliximab and methotrexate in the treatment of rheumatoid arthritis. Anti-Tumor Necrosis Factor Trial in Rheumatoid Arthritis with Concomitant Therapy Study Group. [see comments]. N Engl J Med. 2000, 343: 1594-602. 10.1056/NEJM200011303432202.View ArticlePubMedGoogle Scholar
- Sharp JT, Strand V, Leung H, Hurley F, Loew-Friedrich I: Treatment with leflunomide slows radiographic progression of rheumatoid arthritis: results from three randomized controlled trials of leflunomide in patients with active rheumatoid arthritis. Leflunomide Rheumatoid Arthritis Investigators Group. [erratum appears in Arthritis Rheum 2000 Jun;43(6):1345]. Arthritis & Rheumatism. 2000, 43: 495-505. 10.1002/1529-0131(200003)43:3<495::AID-ANR4>3.0.CO;2-U.View ArticleGoogle Scholar
- Strand V, Cohen S, Schiff M, Weaver A, Fleischmann R, Cannon G, Fox R, Moreland L, Olsen N, Furst D, Caldwell J, Kaine J, Sharp J, Hurley F, Loew-Friedrich I: Treatment of active rheumatoid arthritis with leflunomide compared with placebo and methotrexate. Leflunomide Rheumatoid Arthritis Investigators Group. Arch Intern Med. 1999, 159: 2542-50. 10.1001/archinte.159.21.2542.View ArticlePubMedGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2474/3/13/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article: verbatim copying and redistribution of this article are permitted in all media for any purpose, provided this notice is preserved along with the article's original URL.