Interrater reliability of clinical tests to evaluate scapulothoracic motion
© Baertschi et al.; licensee BioMed Central Ltd. 2013
Received: 22 March 2013
Accepted: 31 October 2013
Published: 5 November 2013
Decreased scapulothoracic motion has been associated with various pathologies of the shoulder. Reliable and simple assessment methods of scapular mobility are, however lacking. The aim of this study was to evaluate the interrater reliability of four clinical tests to assess scapulothoracic motion in patients with a slightly restricted shoulder flexion.
A total of nineteen patients with a symptomatic slight restriction of shoulder flexion and twenty asymptomatic subjects were evaluated. The investigation consisted of four palpatory tests to assess scapulothoracic motion. A two-level rating scale (positive, negative) was utilised. Interrater reliability was evaluated using kappa coefficients.
We found substantial to almost perfect (Kappa = 0.63-0.4) interrater reliability for the four tests.
Our study demonstrates that the four mobility tests of the shoulder are a reliable and simple instrument to assess patients with a slightly restricted shoulder flexion. Future studies should be conducted to evaluate the validity of these tests and to establish their clinical usefulness.
Scapular movement plays a key role in shoulder and arm function. The scapula acts as a stable base for optimal muscle activation and as a transfer link for developed forces in the kinetic chain . Shoulder pain and several pathologies, such as shoulder impingement, rotator cuff tendinopathy, rotator cuff tears, glenohumeral instability, adhesive capsulitis and stiff shoulders, are associated with alteration in scapular kinematics [2–4]. Therefore, the evaluation of scapular kinematics should form a part of clinical shoulder joint examination.
Previous assessment methods of scapular motion focussed on visual observation [5–8]. In these studies winging, or dysrhythmia of scapular motion during shoulder flexion or abduction was rated. Decreased scapular motion was not an assessment criterion. However, a correlation between decreased scapular upward rotation and glenohumeral instability has been demonstrated [3, 9, 10]. Furthermore, it is generally believed, that reductions in scapular upward rotation and posterior tilt during arm elevation could contribute to subacromial impingement by reducing the available subacromial space [3, 5, 11–14]. Contrary to this, another investigation showed an increase of the subacromial space with reduction of scapular upward rotation . Conflicting results are also found in the direction of scapular motion alterations in shoulder impingement . The numerous methods of recording scapular motion, the variation in movement patterns in subjects and the investigation of scapular motion in different static positions or variations of shoulder elevation, might have contributed to this variability of findings.
By using x-ray cinematographic analyses, Stenvers and radiologists from the Martini hospital in Groningen associated decreased scapular upward rotation, posterior tilt and external rotation with a slight restriction of shoulder flexion (± 150°) and altered motion of the clavicle and the cervicothoracic junction . Stenvers further noted that in these patients glenohumeral range of motion in flexion and abduction is not, or only insignificantly restricted. He described this clinical pattern as, “the slightly restricted shoulder”. He observed the following disorders in patients with a slightly restricted shoulder: subacromial impingement, coracoclavicular compression, excessive torsion in the acromioclavicular joint and a tendency towards glenohumeral instability .
To identify patients with decreased scapular motion and to allow a more targeted treatment of these patients, physiotherapists require a reliable and easily performed clinical assessment method.
One measurement instrument with good to excellent intrarater reliability in the assessment of scapular upward rotation in different positions of shoulder abduction, is the inclinometer .
This method would be well suited to documenting progress during therapy. However, in order to select patients for targeted treatment, data both from healthy subjects and for interrater reliability, are not available. Furthermore, only shoulder abduction and not flexion has been investigated. Asymmetry of scapular motion, however, is more evident in flexion than in abduction .
Stenvers and Overbeek  described four palpatory mobility tests of the shoulder to identify patients with a slightly restricted shoulder. These tests are easy to apply in clinical practice and include evaluation of scapular, clavicular and cervicothoracic motion. To our knowledge, this is the first study to investigate the reliability of palpatory tests on scapulothoracic motion.
The aim of this study was to evaluate the interrater reliability of four tests in the assessment of scapular motion during shoulder flexion.
A total of thirty nine participants were recruited from the Physiotherapy Department of the Balgrist University Hospital in Zurich, Switzerland; nineteen patients with symptomatic slight restriction of shoulder flexion and twenty controls with no shoulder symptoms. Sample size calculation was based on identifying a moderate strength of agreement (Kappa > 0.4) at a significance level of 0.01 and a power of 80% .
Patients were included if they presented with shoulder complaints and passive shoulder flexion of at least 130 degrees in a standing position. Thus, patients with a predominantly glenohumeral restriction were excluded. Controls had no shoulder complaints or any other complaints which might have had an impact on shoulder function. Participants were excluded if they had had shoulder or spinal surgery less than six weeks previously, or suffered from neurological disorders or scoliosis.
Approval was obtained from the Cantonal Ethics Committee of Zurich (KEK-ZH: 2011–0387). Written informed consent of each study participant was obtained.
Test 1: Scapular axillary hair test at the end of flexion.
Test 2: Clavicular movement during the first 60 degrees of flexion.
Test 3: Scapular posterior tilting during the last phase of flexion.
Test 4: Movement of the cervicothoracic junction during the last phase of flexion.
Decreased scapular upward rotation, posterior tilt and external rotation, combined with altered motion of the clavicle and the cervicothoracic junction, was defined as a restriction of scapular motion. Flexion was defined as lifting the arm in the sagittal plane. Tests were performed in a standardised order. Participants stood upright, legs one foot-width apart and facing straight ahead. The symptomatic shoulder or alternately the right and left shoulder (controls) was evaluated. Each test was rated either positive (restriction of scapular motion is present) or negative (scapular motion is normal).
Test 1: Scapular axillary hair test at the end of flexion
The investigator passively moves the patient’s arm to the end of shoulder flexion. In this position the distance from the most lateral scapular point (crista margo lateralis inferior) and the vertical extension of the dorsal axillary hair borderline is determined.
Negative: If the most lateral scapular point is in the vertical extension of the dorsal axillary hair borderline, the test was rated negative.
Positive: If the determined distance was one finger width or more, the test was rated positive.
Test 2: Clavicular movement during the first 60 degrees of flexion
During normal shoulder flexion the clavicle moves anteriorly during the first 60 degrees. As a result, space in supraclavicular fossa increases. Subsequently, the clavicle continues to move cranially and posteriorly and, finally, caudally (ellipsoid path) . Other studies have described posterior rotation, retraction and minimal elevation of the clavicle during normal elevation of the arm [23, 24].
The investigator passively moves the patient’s arm to approximately 60 degrees of shoulder flexion. By means of palpation the clavicular motion is simultaneously assessed.
A decreased rotation can be observed in addition to the following pattern: at the beginning of the movement the clavicle moves posteriorly, followed later by movement in a dorsocranial direction (Figure 1F) . Thus, the palpating finger is pushed out of the supraclavicular fossa.
A different movement pattern of the clavicle was described in another study that analysed three-dimensional motion of the clavicle in symptomatic shoulder patients . The authors found a greater clavicular elevation at 90 and 120 degrees of shoulder flexion in subacromial impingement compared to healthy subjects. The differences between groups below 90 degrees of flexion were statistically not significant.
Negative: If the clavicle “stood still”, or a small fosse was formed for the palpating finger, the test was rated negative.
Positive: The test was rated positive if the clavicle pushed the palpating finger cranially out of the supraclavicular fossa.
Test 3: Scapular posterior tilting during the last phase of flexion
The investigator passively moves the patient’s arm to the end of shoulder flexion. The scapula is palpated simultaneously.
Negative: If the inferior angle of the scapula moved caudal and anterior at the end of shoulder flexion, the test was rated negative.
Positive: If this movement could not be felt, the test was rated positive.
Test 4: Movement of the cervicothoracic junction during the last phase of flexion
During shoulder movement not only the scapulothoracic, the acromioclavicular, sternoclavicular and glenohumeral joints are involved, but also the cervical and thoracic spine. At the normal end of range flexion of the shoulder the cervicothoracic junction moves in extension, contralateral lateral flexion and ipsilateral rotation. There is a significant correlation between restriction of scapular motion and restricted movement of cervicothoracic junction [20, 26].
The investigator passively moves the patient’s arm to approximately 30 degrees before the end of shoulder flexion. During passive end of range flexion (the last 30°) the spinal segments C7-T4 are palpated.
Negative: If an ipsilateral rotation of the spinous processes from C7-T4 could be palpated, the test was rated negative.
Positive: If this movement could not be palpated, the test was rated positive.
Mean age, yrs (SD, range)
46 (15, 21–66)
32 (9, 20–55)
Dominant hand: right/left
Shoulder tested: right/left
Mean symptom duration, mths (SD, range)
26 (29, 1–120)
SPADI a); mean (SD, range)
17 (12, 4–48)
24 (17, 6–64)
12 (10, 0–39)
Rotator cuff repair
Osteosynthesis (humerus, clavicle)
Rotator cuff tear
Nonspecific shoulder pain
Other complaints (number)
Sports on a regular basis (number)
Two experienced physiotherapists specializing in musculoskeletal therapy performed the mobility tests of the shoulder. They received further training from J.D. Stenvers at the training centre for neck, shoulder and arm disorders (NSA). Prior to the experiment they underwent an additional training in order to be familiarised with the test performance and rating criteria. Raters were blinded to the patients’ diagnoses. The order of the four tests for each participant was the same for both testing sessions. Each session did not last longer than five minutes. The order of the two raters was changed after each participant. There was a break of at least thirty minutes to maximum half a day between the two testing sessions to minimise testing bias between the two investigations. Participants did not receive any therapeutic treatment for the shoulder during the break. To verify whether pain intensity was comparable at the start of both testing sessions, raters recorded the participant’s current pain intensity on a visual analogue scale (VAS: 0–10) at the beginning of each testing session.
A paired samples t-test was conducted to compare pain intensity at the beginning of the two testing sessions.
Interrater reliability was evaluated using kappa coefficients  and percentage agreement. However, where the prevalence is not around 50% to a particular outcome between raters or in the presence of bias, kappa can be affected . Therefore, prevalence and bias indices, as well as a prevalence-adjusted-bias-adjusted kappa (PABAK) coefficient were calculated . The classification system proposed by Landis and Koch  was used to determine the level of reliability for Kappa and PABAK: <0: poor, 0.00-0.20: slight, 0.21-0.40: fair, 0.41-0.60: moderate, 0.61-0.80: substantial, 0.81-1.00: almost perfect. A kappa coefficient of over 0.40 for clinical tests was considered to be acceptable in comparable studies [8, 22, 35, 36]. Accordingly in this study, kappa coefficients of over 0.40 were judged as being satisfactory.
Statistical analysis was performed using IBM-SPSS 17 (SPSS, Inc., Chicago, USA).
The participants’ characteristics are presented in Table 1. Patients presented with eight different clinical diagnoses. Median symptom duration was two years. Pain and disability in daily living were rather low (Shoulder Pain and Disability Index: 17/ 100). Head, spinal, elbow or hand disorders had been experienced by nine patients (47%) and six controls (30%) during the previous twelve months. Approximately 90% of participants practiced sports on a regular basis.
Glenohumeral and cervical spine range of motion
Mean glenohumeral range of motion, degree (SD, range)
85 (8, 70–95)
91 (2, 90–95)
88 (5, 80–95)
93 (4, 85–100)
External rotation in 0° abduction
40 (15, 15–70)
60 (12, 30–80)
External rotation in 90° abduction
79 (17, 45–100)
94 (5, 80–100)
Internal rotation in 90° abduction
36 (17, 10–75)
53 (14, 20–80)
Mean cervical range of motion, degree: (SD, range)
58 (7, 45–70)
61 (11, 45–80)
56 (11, 35–80)
74 (10, 55–90)
Rotation to the right
72 (7, 60–85)
81 (9, 60–90)
Rotation to the left
71 (10, 50–85)
81 (8, 65–90)
Mean pain intensity at the start of the session was 1.3/ 10 for rater A and 1.4/ 10 for rater B, indicating no significant difference in pain level at the start of each testing session (paired t-test, p = 0.21).
The frequency of positive rating was 63-89% for patients and 5-30% for controls.
Reliability of the moblity tests of the shoulder
Kappa (κ) 95% CI
PABAK 95% CI
Test 1 Patients
0.60 (0.11, 1.00)
0.79 (0.51, 1.00)
0.74 (0.40, 1.00)
0.80 (0.54, 1.00)
0.79 (0.60, 0.98)
0.79 (0.60, 0.99)
Test 2 Patients
0.52 (0.12, 0.92)
0.57 (0.21, 0.95)
0.35 (0.00, 0.86)
0.70 (0.39, 1.00)
0.63 (0.39, 0.87)
0.64 (0.40, 0.88)
Test 3 Patients
0.60 (0.11, 1.00)
0.79 (0.51, 1.00)
0.57 (0.14, 1.00)
0.70 (0.39, 1.00)
0.74 (0.53, 0.95)
0.74 (0.53, 0.95)
Test 4 Patients
0.64 (0.00, 1.00)
0.90 (0.70, 1.00)
0.74 (0.40, 1.00)
0.80 (0.54, 1.00)
0.84 (0.67, 1.00)
0.85 (0.68, 1.00)
To our knowledge, this is the first study to investigate the reliability of palpatory tests of scapulothoracic motion. We found substantial to almost perfect reliability for the four mobility tests of the shoulder in patients with a slightly restricted shoulder and asymptomatic participants when performed by two experienced physiotherapists.
Looking at the kappa values of patients and controls separately, the results were fair to substantial. These differences can be explained by the smaller sample sizes and high prevalence indices. A prevalence effect exists when the proportion of agreements on the positive classification differs from that of the negative classification . The number of positive tests was very high in patients and very low in controls. Both the high prevalence indices and the smaller sample size affected the confidence interval . To examine the effects of prevalence on kappa values and confidence interval, PABAK values were calculated. These values were higher than the unadjusted kappa values. However, for interrater agreement, it is the result from the total sample size that is of particular interest. The bias indices were low over all, indicating that systematic bias of an examiner did not influence the results.
Other studies investigating scapular motion have obtained lower interrater reliability than this study [6, 7]. Uhl et al.  and McClure et al.  assessed scapular motion through visual observation and not by use of palpation. The use of palpatory tests might have contributed to the higher reliability. McClure et al.  used a 3-level scale (normal, subtle, obvious change of motion) to rate scapular motion. The use of only two levels (positive, negative) may also account for the higher kappa values observed in this study.
Stenvers et al.  argue that the four tests are of little value on their own. They propose a diagnosis of a restriction of scapular movement only when at least three of the four tests are positive. This clinical application of the four analysed tests has no negative influence on interrater agreement. If the scapulothoracic motion of the thirty-nine participants is assessed as Stenvers has suggested, the interrater reliability is substantial (Kappa = 0.74). Whether this “overall assessment “of the four tests achieves a greater specificity and sensitivity than the individual tests cannot be evaluated from this study. A reference test, or gold standard, is not available.
Several factors should be considered when interpreting the level of reliability found in this study. Since the four tests were always conducted consecutively, the rater’s decision may have been influenced by the outcome of preceding tests. However, for decision making in clinical practice, it is standard to consider the results of all four tests rather than taking into account just one single test.
Another limitation of this study is that, since controls had no limitation of shoulder movement, raters could not be blinded to the participants’ group. Thus, their judgement may have been influenced by their expectations. Blinding was also difficult due to the evaluation of pain at the beginning of every testing session.
Before generalising the results of this study, it should be considered that the two raters were experienced physiotherapists with a specialisation in musculoskeletal therapy and had also received further training in the treatment of neck, shoulder and arm disorders. Future research is required to investigate whether satisfactory levels of reliability can also be achieved using less experienced clinicians without specific training.
Our study demonstrates that the four mobility tests of the shoulder are a reliable and simple instrument in the assessment of patients with a slightly restricted shoulder flexion. Nevertheless, validity has to be investigated before the clinical utilisation of these tests can be approved, e.g. comparing the four mobility tests of the shoulder with electromagnetic kinematic testing.
Moreover, further studies are needed to show correlations between restricted scapular motion and shoulder pathologies. Additionally, the effect of specific treatment of patients with restricted scapular motion would be of great interest.
The authors wish to thank all participants and Linda Dyer and Marcel Enzler (Uniklinik Balgrist, Zurich) for performing the assessments. Moreover, a cordial thanks is owed to Karen Linwood-Williams for the correction of the English text.
This study originated during the Master of Advanced Studies course in Musculoskeletal Physiotherapy at the Zurich University of Applied Sciences.
- Kibler WB, Ludewig PM, McClure P, Uhl TL, Sciascia A: Scapular Summit 2009: introduction. July 16, 2009, Lexington, Kentucky. J Orthop Sports Phys Ther. 2009, 39 (11): A1-A13. 10.2519/jospt.2009.0303.View ArticlePubMedGoogle Scholar
- Kibler WB, McMullen J: Scapular dyskinesis and its relation to shoulder pain. J Am Acad Orthop Surg. 2003, 11 (2): 142-151.View ArticlePubMedGoogle Scholar
- Ludewig PM, Reynolds JF: The association of scapular kinematics and glenohumeral joint pathologies. J Orthop Sports Phys Ther. 2009, 39 (2): 90-104. 10.2519/jospt.2009.2808.View ArticlePubMedPubMed CentralGoogle Scholar
- Stenvers JD: The primary frozen shoulder. A retrospective study on the treatment by physiotherapy. Dissertation. 1994, The Netherlands: University GroningenGoogle Scholar
- Tate AR, McClure P, Kareha S, Irwin D, Barbe MF: A clinical method for identifying scapular dyskinesis, part 2: validity. J Athl Train. 2009, 44 (2): 165-173. 10.4085/1062-6050-44.2.165.View ArticlePubMedPubMed CentralGoogle Scholar
- Uhl TL, Kibler WB, Gecewich B, Tripp BL: Evaluation of clinical assessment methods for scapular dyskinesis. Arthroscopy. 2009, 25 (11): 1240-1248. 10.1016/j.arthro.2009.06.007.View ArticlePubMedGoogle Scholar
- McClure P, Tate AR, Kareha S, Irwin D, Zlupko E: A clinical method for identifying scapular dyskinesis, part 1: reliability. J Athl Train. 2009, 44 (2): 160-164. 10.4085/1062-6050-44.2.160.View ArticlePubMedPubMed CentralGoogle Scholar
- Schöttker-Königer T, Morrison F, Taa M, Paa A, Saa A: Intertester-Reliabilität des Skapula-Dyskinesis-Tests - Wie gut sind Physiotherapeuten in der Bewegungsbeobachtung. Manuelle Therapie. 2011, 15 (4): 145-152. 10.1055/s-0031-1281683.View ArticleGoogle Scholar
- Illyes A, Kiss RM: Kinematic and muscle activity characteristics of multidirectional shoulder joint instability during elevation. Knee Surg Sports Traumatol Arthrosc. 2006, 14 (7): 673-685. 10.1007/s00167-005-0012-7.View ArticlePubMedGoogle Scholar
- Ogston JB, Ludewig PM: Differences in 3-dimensional shoulder kinematics between persons with multidirectional instability and asymptomatic controls. Am J Sports Med. 2007, 35 (8): 1361-1370. 10.1177/0363546507300820.View ArticlePubMedGoogle Scholar
- Struyf F, Nijs J, Baeyens JP, Mottram S, Meeusen R: Scapular positioning and movement in unimpaired shoulders, shoulder impingement syndrome, and glenohumeral instability. Scand J Med Sci Sports. 2011, 21 (3): 352-358. 10.1111/j.1600-0838.2010.01274.x.View ArticlePubMedGoogle Scholar
- Endo K, Ikata T, Katoh S, Takeda Y: Radiographic assessment of scapular rotational tilt in chronic shoulder impingement syndrome. J Orthop Sci. 2001, 6 (1): 3-10. 10.1007/s007760170017.View ArticlePubMedGoogle Scholar
- Ludewig PM, Cook TM: Alterations in shoulder kinematics and associated muscle activity in people with symptoms of shoulder impingement. Phys Ther. 2000, 80 (3): 276-291.PubMedGoogle Scholar
- Seitz AL, McClure PW, Lynch SS, Ketchum JM, Michener LA: Effects of scapular dyskinesis and scapular assistance test on subacromial space during static arm elevation. J Shoulder Elbow Surg. 2012, 21 (5): 631-640. 10.1016/j.jse.2011.01.008.View ArticlePubMedGoogle Scholar
- Karduna AR, Kerner PJ, Lazarus MD: Contact forces in the subacromial space: effects of scapular orientation. J Shoulder Elbow Surg. 2005, 14 (4): 393-399. 10.1016/j.jse.2004.09.001.View ArticlePubMedGoogle Scholar
- Stenvers JD, Overbeek WJ: Mobiliteitstesten van de schouder. 1977, Niederlande: Neurochirurgische Universiteitskliniek Groningen, Retrieved from http://www.nsastenvers.nl/pagina83a.html Google Scholar
- Stenvers JD: Schulterbeschwerden, Weiterbildungszentrum für Nacken-, Schulter- und Armbeschwerden NSA. 2003, Niederlande: KursunterlagenGoogle Scholar
- Watson L, Balster SM, Finch C, Dalziel R: Measurement of scapula upward rotation: a reliable clinical procedure. Br J Sports Med. 2005, 39 (9): 599-603. 10.1136/bjsm.2004.013243.View ArticlePubMedPubMed CentralGoogle Scholar
- Sim J, Wright CC: The kappa statistic in reliability studies: use, interpretation, and sample size requirements. Phys Ther. 2005, 85 (3): 257-268.PubMedGoogle Scholar
- Stenvers JD, Woerden HH, Kingma LM: Anatomie, Beeldvorming & Mobiliteitstesten van de licht beperkte schouder. DVD, Ned Tijdschr Manuele Ther. 2009, 5 (3): 34-Google Scholar
- de Wijer A: Temporomandibular and Cervical Spine Disorders. Dissertation. 1995, The Netherlands: University UtrechtGoogle Scholar
- Lopes Cardozo M: Dissertation. The treatment of the costoclavicular compression syndrome by resection of the first rib. A retrospective study done by 200 operated patients. 1976, The Netherlands: University GroningenGoogle Scholar
- Ludewig PM, Phadke V, Braman JP, Hassett DR, Cieminski CJ, LaPrade RF: Motion of the shoulder complex during multiplanar humeral elevation. J Bone Joint Surg Am. 2009, 91 (2): 378-389. 10.2106/JBJS.G.01483.View ArticlePubMedPubMed CentralGoogle Scholar
- McClure PW, Michener LA, Sennett BJ, Karduna AR: Direct 3-dimensional measurement of scapular kinematics during dynamic movements in vivo. J Shoulder Elbow Surg. 2001, 10 (3): 269-277. 10.1067/mse.2001.112954.View ArticlePubMedGoogle Scholar
- McClure PW, Michener LA, Karduna AR: Shoulder function and 3-dimensional scapular kinematics in people with and without shoulder impingement syndrome. Phys Ther. 2006, 86 (8): 1075-1090.PubMedGoogle Scholar
- Crosbie J, Kilbreath SL, Hollmann L, York S: Scapulohumeral rhythm and associated spinal motion. Clin Biomech (Bristol, Avon). 2008, 23 (2): 184-192. 10.1016/j.clinbiomech.2007.09.012.View ArticleGoogle Scholar
- van de Pol RJ, van Trijffel E, Lucas C: Inter-rater reliability for measurement of passive physiological range of motion of upper extremity joints is better if instruments are used: a systematic review. J Physiother. 2010, 56 (1): 7-17. 10.1016/S1836-9553(10)70049-7.View ArticlePubMedGoogle Scholar
- Bruzek R: Assessment: Inklinometer. Bewegungsausmass messen. Physiopraxis. 2008, 1: 34-35.Google Scholar
- Whitcroft KL, Massouh L, Amirfeyz R, Bannister G: Comparison of methods of measuring active cervical range of motion. Spine (Phila Pa 1976). 1976, 35 (19): E976-E980.View ArticleGoogle Scholar
- Roach KE, Budiman-Mak E, Songsiridej N, Lertratanakul Y: Development of a shoulder pain and disability index. Arthritis Care Res. 1991, 4 (4): 143-149. 10.1002/art.1790040403.View ArticlePubMedGoogle Scholar
- Angst F, Goldhahn J, Pap G, Mannion AF, Roach KE, Siebertz D, Drerup S, Schwyzer HK, Simmen BR: Cross-cultural adaptation, reliability and validity of the German Shoulder Pain and Disability Index (SPADI). Rheumatology (Oxford). 2007, 46 (1): 87-92. 10.1093/rheumatology/kel040.View ArticleGoogle Scholar
- Kilem LG: Intrarater Reliability. Encyclopedia of Clinical Trials. 2008, New York: Wiley IntersienceGoogle Scholar
- Cicchetti DV, Feinstein AR: High agreement but low kappa: II. Resolving the paradoxes. J Clin Epidemiol. 1990, 43 (6): 551-558. 10.1016/0895-4356(90)90159-M.View ArticlePubMedGoogle Scholar
- Landis JR, Koch GG: The measurement of observer agreement for categorical data. Biometrics. 1977, 33 (1): 159-174. 10.2307/2529310.View ArticlePubMedGoogle Scholar
- Luomajoki H, Kool J, de Bruin ED, Airaksinen O: Reliability of movement control tests in the lumbar spine. BMC Musculoskelet Disord. 2007, 8: 90-10.1186/1471-2474-8-90.View ArticlePubMedPubMed CentralGoogle Scholar
- Seffinger MA, Najm WI, Mishra SI, Adams A, Dickerson VM, Murphy LS, Reinsch S: Reliability of spinal palpation for diagnosis of back and neck pain: a systematic review of the literature. Spine (Phila Pa 1976). 2004, 29 (19): E413-E425. 10.1097/01.brs.0000141178.98157.8e.View ArticleGoogle Scholar
- Grouven U, Bender R, Ziegler A, Lange S: The kappa coefficient. Dtsch Med Wochenschr. 2007, 132 (Suppl 1): e65-e68.View ArticlePubMedGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2474/14/315/prepub
This article is published under license to BioMed Central Ltd. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.