This article has Open Peer Review reports available.
Feasibility of a standardized ultrasound examination in patients with rheumatoid arthritis: a quality improvement among rheumatologists cohort
- Karen Ellegaard1Email author,
- Søren Torp-Pedersen1,
- Robin Christensen1, 2,
- Michael Stoltenberg3,
- Annette Hansen4,
- Tove Lorenzen5,
- Dorthe V Jensen6,
- Hanne Lindegaard7,
- Lars Juul8,
- Henrik Røgind9,
- Per Bülow10,
- Stavros Chrysidis11,
- Marcin Kowalski12,
- Bente Danneskiold-Samsoe1, 13, 14 and
- Henning Bliddal1, 13, 14
© Ellegaard et al; licensee BioMed Central Ltd. 2012
Received: 16 August 2011
Accepted: 12 March 2012
Published: 12 March 2012
Quality improvement is important to facilitate valid patient outcomes. Standardized examination procedures may improve the validity of US.
The aim of this study was to investigate the learning progress for rheumatologists during training of US examination of the hand in patients with rheumatoid arthritis (RA).
Rheumatologists with varying degrees of experience in US were instructed by skilled tutors. The program consisted of two days with hands-on training followed by personal US examinations performed in their individual clinics. Examinations were sent to the tutors for quality control. The US examinations were evaluated according to a scoring sheet containing 144 items. An acceptable examination was defined as > 80% correct scores.
Thirteen rheumatologists participated in the study. They included a total of 104 patients with RA. Only few of the initial examinations were scored below 80%, and as experience increased, the scores improved (p = 0.0004). A few participants displayed decreasing scores.
The mean time spent performing the standardized examination procedure decreased from 34 min to less than 10 minutes (p = 0.0001).
With systematic hands-on training, a rheumatologist can achieve a high level of proficiency in the conduction of US examinations of the joints of the hand in patients with RA. With experience, examination time decreases, while the level of correctness is maintained. The results indicate that US may be applied as a valid measurement tool suitable for clinical practice and in both single- and multi-centre trials.
Ultrasound (US) is increasingly popular among rheumatologists for both diagnosis and monitoring of treatment of patients with rheumatoid arthritis (RA) [1–4]. US examination is assumed to be a relatively operator dependent technique. This may affect the validity of the technique [5, 6] and without previous coordination of standardized examination technique, experts on performing musculoskeletal US only obtain moderate correlations of US examinations in different anatomic regions in patients with various diagnoses [6–8] and even poorer when a combination of image acquisition and interpretation is evaluated . Consequently, standardized examination and interpretation procedures are required in order to improve the validity of rheumatological US examinations .
Educational programs in musculoskeletal US have been developed  and tested mostly within grey scale US [5, 11–13] and focused attempts to change clinician behaviour may lead to improved outcomes . Quality improvement, by educating rheumatologists to perform standardized US examination, is anticipated, while only sparse information is available as to the effect of a training program on the quality and precision of the US examination. It remains to be shown whether a learning curve might be established to describe standards for reaching and maintaining a sufficient level of US examination competence.
The ability to conduct a standardized measurement is a requirement of the OMERACT filter, which is the framework used to assess truth, discrimination, and feasibility of outcome measures in rheumatology trials and practices .
The one aim of the present prospective study was to evaluate the ability of a US training program to obtain standardized images of high quality in an acceptable time frame making the standardized procedure suitable for both clinical praxis and clinical trials. Thus, diagnostic utility of US and evaluation of the use of US to assess level of disease activity and monitor treatment response were not investigated in this study. A standardized examination procedure was used, which in a specialist setting has been shown to have an excellent reproducibility in patients with RA . Two separate outcome measures were used as indicators of the learning process; (1) The number of examinations performed by an investigator to achieve satisfactory skills and (2) the duration of each examination.
In order to be enrolled in the training program it was required that the participating rheumatologist had passed two standard courses of US diagnostics arranged by the Danish Society of Diagnostic Ultrasound. All rheumatological centres in Denmark employing a rheumatologist, who has passed the required US course, were invited to participate in the project. The participating centres borrowed identical ultrasound machines with a fixed preset. Ten machines were available for the study. Ten centres agreed to participate in the study.
Patient and physician characteristics at baseline
DAS28, score: 0-10
C-Reactive Protein, mg/l
Swollen joint count, 0-28
Tender joint count, o-28
VAS global (0-100 mm)
Disease duration (years)
US examination (minutes)
Scanning experience (months)
Scanning experience (number)
Degree of Supervision (1 to 3)*
The training program consisted of two parts. In the first part all the participating rheumatologists had an individual, two days, hands-on, training course. This part of the program took place before inclusion of patients. The rheumatologists were trained in a standardized US examination of the wrist and MCP II-V joints which has demonstrated an excellent reliability . The rheumatologist performed US examinations of the wrist and MCP joints in 10 patients with RA under supervision by one of two tutors (KE and STP). The tutors had extensive parallel working experience in performing the standardized US examination (KE five years and STP more than 10 years). In the second part the tutors evaluated all the performed US examinations, which were submitted from each centre for immediate feedback by e-mail. In case of failure to reach 80% correct scores, the centre was asked to recall the patient for a repeated examination.
The patients were enrolled consecutively in the rheumatological centres. Inclusion criteria were: RA diagnosed according to the American College of Rheumatology's criteria , Doppler activity in the wrist and/or MCP joints, ongoing therapy with methotrexate and scheduled for treatment with etanercept (only first time treatments). Exclusion criteria were treatments with other biologics within two month or lack of Doppler activity. The patients were scanned five times during the first year of etanercept therapy, or as long as they remained on this therapy. The US examinations were performed at baseline, after 2, 12, 24 and 52 weeks. The study was approved by the local ethics committee (KF 01 31 8007) and informed consent was obtained from each patient before study entry. At baseline, the patients accepted a possible delay of treatment up to one week in the case of an unacceptable baseline ultrasound examination.
At all centres, the US examinations were performed with a GE Logiq® 9 (Milwaukee, Wisconsin, USA) using a 14 MHz centre frequency linear array matrix transducer. A preset for the ultrasound examinations was installed on all machines and remained unchanged throughout the study period. The participants were allowed to make any changes to the grey scale settings in the scanning situation. The participants were not allowed to adjust the Doppler parameters except for box size and position and Doppler focus.
Landmarks at standardized ultrasound examination of wrist and MCP joints
Wrist dorsal central
The third extensor digitorum tendon, tip of radius, carpometacarpal joint, os lunatum and os capitatum.
Wrist dorsal radial
The extensor carpi radialis longus tendon, tip of radius, carpometacarpal joint, os scaphoideum and os trapezoideum
Wrist dorsal ulnar
The middle of caput ulnae placed minimum 1 centimetre from the right side and minimum 2 from the left side of the image margins
Wrist volar central
N. medianus, tip of radius, radiocarpal and intercarpal joints
MCP* dorsal radial
Joint space, diaphysis of the metarcarpal and proximal phalanx bones. The joint space has to be placed minimum 2 centimetre from the right side and minimum 1 centimetre from the left side of the image margins
MCP* dorsal ulnar
Joint space, diaphysis of the metarcarpal and proximal phalanx bones. The joint space has to be placed minimum 2 centimetre from the right side and minimum 1 centimetre from the left side of the image margins
MCP* joint volar
Joint space and flexor tendon. The joint space has to be placed minimum 2 centimetres from the right side and minimum 1 centimetre from the left side of the image margins.
All US examinations were sent to the tutors for evaluation. The examinations were evaluated according to a standardized scoring system assessing the quality of all stored images. The participants received an evaluation of each US examination by e-mail. The first 50 examinations were evaluated by both tutors in order to obtain consensus. A total of 16 positions were investigated at each US examination. The total number of outcome items scored in each of the 16 positions was nine, thus a total of 144 scores was given. The scoring system was dichotomous as each outcome was assessed as accepted/not accepted. The US evaluation contained the following items: storing of all relevant images, correct annotation and image orientation (left-right), well-defined landmarks, correct focus in both grey-scale and Doppler images, presence of air or Doppler artefacts and finally correct position of the Doppler box.
In order to get the US examination approved, 80% of the items should be accepted. The cut-off point of 80% was chosen in accordance with other studies investigating learning experience in musculoskeletal US [12, 13]. At each US evaluation, the duration of the examination was noted in order to evaluate the changes in the time spent.
The data structure of the study design included clustered data (i.e, patients within clinics with repeated measures), thus the hierarchical model with continuous data was applied. Statistical analyses were based on a linear mixed model: Patients and Rheumatologists were applied as random factors when assessing associations over time. This was modelled using the Restricted Maximum Likelihood (REML) default option in SAS PROC MIXED: based on a random coefficient model in order to assess the different possible linear associations across rheumatology clinics simultaneously . Random coefficient models emerge as natural mixed model extensions of simple linear regression models in a hierarchical (nested) data setup. To combine the individual study results we performed the hierarchical modelling using SAS software (version 9.2).
Patients and rheumatologists
Nine of the 13 participating rheumatologists (69%) had either a consistent or slightly improved learning curve throughout the project, whereas 4 (31%) demonstrated a decreasing learning curve (Figure 3). Of these, two dropped out of the project within the first three months.
A standardized examination procedure in patients with RA is useful for both assessment of disease activity, monitoring of treatment and in clinical trials. The results from this training program suggest that skills in standardized US examination of the hand in patients with RA can be achieved by most rheumatologists, after a short training program. Nearly, all the participating rheumatologists demonstrated scores above the preset cut-off level from the very beginning, without obvious differences due to variation in previous experience. This was also demonstrated by the lack of association between time spent on the standardized US examination and the rheumatologists' US experience. However, it must be noted that nearly all participants had a moderate to high degree of US training including several courses and two full days of hand-on experience with the present technique.
In accordance with recent results from other groups , this study indicates that the pitfall of US operator dependency may be avoided with a short focused training program provided that the anatomical region under investigation is sufficiently small (wrist and mcp-joints in this study). Also, bear in mind that we have only focused on ability to obtain images of a certain quality and not focused on ability to diagnose pathology.
Very low scores were only seen in a few cases and were associated with early exit from the study by some of the participants. Consequently, the study gives no answer to the obvious question whether even such rheumatologists might have achieved US examination at a higher level with further training.
The final average examination time of less than 10 minutes (Figure 4) for the standardized procedure of 16 positions, was below the examination time used in another longitudinal study in which US 17 positions were examined .
Despite the continuous feed-back, many participants made one persistent mistake throughout the entire study: imprecision of the standardized landmarks in the images of the MCP joints. Besides this mistake, only small flaws in the examinations were noted and with the exception of landmarks in MCP images from the analyses, the scoring level would have increased to nearly 100%. The importance of obtaining precise landmarks is recognised in US training programs [12, 13] and precision of landmarks is mandatory in both longitudinal and multicentre studies to achieve comparable images.
We chose the hand as target for this training program, because the joints of the hand are frequently involved in RA ,  Furthermore, it has been indicated that it is difficult to acquire satisfactory skills in US examination of the hand . Thus, by choosing the hand, we avoided the bias that good learning curves were obtained by examining a simple joint.
The good results may also be attributed to our use of the same preset on all machines. This preset ensured comparable images of a relatively high quality in all patients instead of e.g. using the factory preset MSK where each exam would require some adjustments. We wished to investigate the ability to obtain reliable colour Doppler images which in our opinion demands a fixed preset. Therefore, we scored the participants' ability to correctly adjust Doppler focus and not ability to adjust Doppler gain, PRF, wall filter etc. The use of a fixed preset is a prerequisite for monitoring disease activity with Doppler and at the same time it minimises the risk of poor image quality caused by incorrect machine settings .
Perhaps the most important result of our study was the reduction in time spent on the examination to a feasible level for clinical practice. This result was achieved at the cost of a small reduction in score, i.e. quality, which may partly be explained by the very high scores among most of the participants from the very beginning of the study period, causing a ceiling effect, or a type of negative learning progression in a few cases. The scores did not deteriorate in a way that could indicate development of some sort of carelessness with the routine. However, the participants in trials may be more keen and accurate with supervised examinations than in the daily clinic and the results require confirmation in a clinical setting.
Standardized examination procedures improve the validity of US and make it more suitable for both clinical practice and follow-up studies. As the training program had the result that most rheumatologists achieved satisfactory skills in performing a standardized US examination it might be assumed that training will improve the quality of the US procedure and thereby the patient outcome . In order to answer this question satisfactorily the effect of a standardized examination procedure on the monitoring of treatment of patients with RA must be clarified.
The present results indicate that a learning program may ensure the acquisition of standardized high quality images, which is a prerequisite for using US for making reliable diagnoses and follow-up examination e.g. according to the OMERACT filter . Standardization may enable comparison of examinations performed at different institutions and make performance of multicentre trials possible.
In our study the diagnostic skills of the participating rheumatologists were not assessed and it could be assumed that skills in US scanning for a diagnostic purpose will demand more training to achieve a satisfactory level.
In conclusion, this study shows that skills in standardized US examinations of the hand are relatively easy to obtain for most rheumatologists even with limited US experience.
The EURA study was sponsored by the Oak Foundation and a restricted grant from Wyeth A/S Denmark. KE and STP have received fees for educational activities sponsored by Wyeth. None of the authors are employed by Wyeth.
The EURA study was sponsored by the Oak Foundation and a restricted grant from Wyeth A/S Denmark.
- Taylor PC, Steuer A, Gruber J, Cosgrove DO, Blomley MJ, Marsters PA: Comparison of ultrasonographic assessment of synovitis and joint vascularity with radiographic evaluation in a randomized, placebo-controlled study of infliximab therapy in early rheumatoid arthritis. Arthritis Rheum. 2004, 50: 1107-1116. 10.1002/art.20123.View ArticlePubMedGoogle Scholar
- Brown AK, Conaghan PG, Karim Z, Quinn MA, Ikeda K, Peterfy CG: An explanation for the apparent dissociation between clinical remission and continued structural deterioration in rheumatoid arthritis. Arthritis Rheum. 2008, 58: 2958-2967. 10.1002/art.23945.View ArticlePubMedGoogle Scholar
- Filippucci E, Farina A, Carotti M, Salaffi F, Grassi W: Grey scale and power Doppler sonographic changes induced by intra-articular steroid injection treatment. Ann Rheum Dis. 2004, 63: 740-743. 10.1136/ard.2003.007971.View ArticlePubMedPubMed CentralGoogle Scholar
- Terslev L, Torp-Pedersen S, Qvistgaard E, Danneskiold-Samsoe B, Bliddal H: Estimation of inflammation by Doppler ultrasound: quantitative changes after intra-articular treatment in rheumatoid arthritis. Ann Rheum Dis. 2003, 62: 1049-1053. 10.1136/ard.62.11.1049.View ArticlePubMedPubMed CentralGoogle Scholar
- Filippucci E, Unlu Z, Farina A, Grassi W: Sonographic training in rheumatology: a self teaching approach. Ann Rheum Dis. 2003, 62: 565-567. 10.1136/ard.62.6.565.View ArticlePubMedPubMed CentralGoogle Scholar
- Scheel AK, Schmidt WA, Hermann KG, Bruyn GA, D'Agostino MA, Grassi W: Interobserver reliability of rheumatologists performing musculoskeletal ultrasonography: results from a EULAR "Train the trainers" course. Ann Rheum Dis. 2005, 64: 1043-1049. 10.1136/ard.2004.030387.View ArticlePubMedPubMed CentralGoogle Scholar
- Naredo E, Moller I, Moragues C, De Agustin JJ, Scheel AK, Grassi W: Interobserver reliability in musculoskeletal ultrasonography: results from a "Teach the Teachers" rheumatologist course. Ann Rheum Dis. 2006, 65: 14-19. 10.1136/ard.2005.037382.View ArticlePubMedGoogle Scholar
- Koski JM, Saarakkala S, Helle M, Hakulinen U, Heikkinen JO, Hermunen H: Assessing the intra- and inter-reader reliability of dynamic ultrasound images in power Doppler ultrasonography. Ann Rheum Dis. 2006, 65: 1658-1660. 10.1136/ard.2005.051250.View ArticlePubMedPubMed CentralGoogle Scholar
- Wakefield RJ, D'Agostino MA, Iagnocco A, Filippucci E, Backhaus M, Scheel AK: The OMERACT Ultrasound Group: status of current activities and research directions. J Rheumatol. 2007, 34: 848-851.PubMedGoogle Scholar
- Naredo E, Bijlsma JW, Conaghan PG, Acebes C, Balint P, Hammer HB: Recommendations for the content and conduct of EULAR Musculoskeletal Ultrasound Courses. Ann Rheum Dis. 2008, 67: 1017-1022.View ArticlePubMedGoogle Scholar
- Brown AK, O'connor PJ, Wakefield RJ, Roberts TE, Karim Z, Emery P: Practice, training, and assessment among experts performing musculoskeletal ultrasonography: toward the development of an international consensus of educational standards for ultrasonography for rheumatologists. Arthritis Rheum. 2004, 51: 1018-1022. 10.1002/art.20844.View ArticlePubMedGoogle Scholar
- Filippucci E, Meenagh G, Ciapetti A, Iagnocco A, Taggart A, Grassi W: E-learning in ultrasonography: a web based approach. Ann Rheum Dis. 2007, 66: 962-965. 10.1136/ard.2006.064568.View ArticlePubMedPubMed CentralGoogle Scholar
- Taggart A, Filippucci E, Wright G, Bell A, Cairns A, Meenagh G: Musculoskeletal ultrasound training in rheumatology: the Belfast experience. Rheumatology (Oxford). 2006, 45: 102-105. 10.1093/rheumatology/kei162.View ArticleGoogle Scholar
- Fan E, Laupacis A, Pronovost PJ, Guyatt GH, Needham DM: How to use an article about quality improvement 2. JAMA. 2010, 304: 2279-2287. 10.1001/jama.2010.1692.View ArticlePubMedGoogle Scholar
- Boers M, Brooks P, Strand CV, Tugwell P: The OMERACT filter for Outcome Measures in Rheumatology. J Rheumatol. 1998, 25: 198-199.PubMedGoogle Scholar
- Ellegaard K, Torp-Pedersen S, Lund H, Henriksen M, Terslev L, Jensen PS: Quantification of colour Doppler activity in the wrist in patients with rheumatoid arthritis - the reliability of different methods for image selection and evaluation. Ultraschall Med. 2008, 29: 393-398. 10.1055/s-2008-1027196.View ArticlePubMedGoogle Scholar
- Arnett FC, Edworthy SM, Bloch DA, McShane DJ, Fries JF, Cooper NS: The American Rheumatism Association 1987 revised criteria for the classification of rheumatoid arthritis. Arthritis Rheum. 1988, 31: 315-324. 10.1002/art.1780310302.View ArticlePubMedGoogle Scholar
- Torp-Pedersen ST, Terslev L: Settings and artefacts relevant in colour/power Doppler ultrasound in rheumatology. Ann Rheum Dis. 2008, 67: 143-149. 10.1136/ard.2007.078451.View ArticlePubMedGoogle Scholar
- Marklund M, Christensen R, Torp-Pedersen S, Thomsen C, Nolsoe CP: Signal intensity of normal breast tissue at MR mammography on midfield: applying a random coefficient model evaluating the effect of doubling the contrast dose 1. Eur J Radiol. 2009, 69: 93-101. 10.1016/j.ejrad.2007.09.006.View ArticlePubMedGoogle Scholar
- Kissin EY, Nishio J, Yang M, Backhaus M, Balint PV, Bruyn GA: Self-directed learning of basic musculoskeletal ultrasound among rheumatologists in the United States 1. Arthritis Care Res (Hoboken). 2010, 62: 155-160.Google Scholar
- Backhaus M, Ohrndorf S, Kellner H, Strunk J, Backhaus TM, Hartung W: Evaluation of a novel 7-joint ultrasound score in daily rheumatologic practice: a pilot project. Arthritis Rheum. 2009, 61: 1194-1201. 10.1002/art.24646.View ArticlePubMedGoogle Scholar
- Fleming A, Crown JM, Corbett M: Incidence of joint involvement in early rheumatoid arthritis. Rheumatol Rehabil. 1976, 15: 92-96. 10.1093/rheumatology/15.2.92.View ArticlePubMedGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2474/13/35/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.