- Research article
- Open Access
- Open Peer Review
The factor validity of the Western Ontario Rotator Cuff Index
BMC Musculoskeletal Disordersvolume 6, Article number: 22 (2005)
The Western Ontario Rotator Cuff Index (WORC) is a self-report questionnaire developed specifically to evaluate disability in persons with pathology of the rotator cuff of the shoulder. The authors created items in 5 categories based on a model of quality of life, but never validated this structure. The purpose of this study was to examine the validity of the original 5-domain model of the WORC by performing factor analysis.
Three hundred twenty nine subjects (age, mean: 52, SD: 12) were tested prior to undergoing surgery for rotator cuff pathologies. They completed the WORC, a self-report questionnaire, which has 21 items on the effect of the rotator cuff problem on symptoms, activities and emotions. Statistical calculations included correlations between items, Cronbach's alpha of the total scale and subscales, and principal component factor analysis with oblique rotation.
Correlations ranged from .09 to .70 between all the items, from .29 to .70 between items within a subscale, and from .53 to .72 between subscale scores. Cronbach's alpha was .93 for the total scale, and .72 to .82 for the subscales. The factor analysis produced 3 factors that explained 57% of the variance. The first factor included symptoms and emotional items, the second included strength items and the third included daily activities.
The results of this study did not support the 5-domain model of the WORC.
The Western Ontario Rotator Cuff Index is a recent self-report questionnaire that was designed to measure "health-related quality of life" in persons with injuries and conditions of the rotator cuff of the shoulder. Kirkley et al  felt the measure should represent the impact of the condition on health as defined by the World Health Organization – "a state of complete physical, mental and social well-being". They, therefore, included items in 5 domains in the questionnaire: 1) pain and physical symptoms, 2) sports and recreation, 3) work, 4) lifestyle, and 5) emotions. The authors followed a systematic, clinimetric method of generating and reducing the items. This resulted in 21 items that respondents answered on visual analogue scales (VAS) with anchors such as no pain/difficulty and extreme pain/difficulty.
Items for the WORC were derived from published health status scales, functional measures of the shoulder, discussions with healthcare professionals, and interviews with 30 patients from a registry of 150 with rotator cuff pathology. Both professionals and patients were asked to identify ways in which the shoulder condition affected quality of life in general, and the 5 domains in particular. The 30 patients interviewed included males and females, aged 30–76, with different degrees of rotator cuff pathology from tendinitis to massive tears.
An original list of 321 items was reduced to 76 by the investigators eliminating duplicated, incomprehensible or ambiguous items. A random selection of 100 patients from the same registry were then asked to indicate whether they experienced each of the items, and to rate the importance of the symptom/disability to their overall shoulder functioning. A frequency importance product was calculated for each item and the 50 items with the highest values were correlated with each other. For every pair of items with coefficients greater than 0.6, one of the items was eliminated, resulting in the final 21 questions. It is not clear whether this criterion applied to items across domains because the only example provided included 3 items from the same domain.
In that same paper , the authors reported an ICC for reliability of .96 when they tested subjects over a 2-week period and omitted those who reported any change on a global rating scale. The ICCs for the subscales ranged from .54 (4-item work) to .91 (6-item physical symptoms).
Construct validity has been tested by the original authors  and others [2, 3] by examining the correlation of the WORC with other shoulder measures (Constant, American Shoulder and Elbow Surgeons Standardized Shoulder Assessment Form [ASES], Disabilities of the Arm, Shoulder and Hand [DASH], University of California Los Angeles [UCLA], Simple Shoulder Test [SST]). The correlations of the WORC total score with the other instruments have ranged from .48 to .91, with generally higher correlations with instruments that have disability items similar to those in the WORC (see Table 1). The correlations of the change scores were in a similar range (.44 to .85).
Two studies [3, 4] have examined the responsiveness of the WORC and other shoulder measures by calculating the standardized response mean (SRM) in patients who have been measured before and after surgery (see Table 2). It should be noted that the SRM of the WORC was not noticeably different from the comparative measures (Constant, SST and DASH) in the same study. Holtby and Razmjou  had lower overall SRMs than MacDermid et al  who included only the responders in their calculations. MacDermid et al  also reported the SRM for the subscales of the WORC. The values ranged from 1.2 for the work subscale to 1.8 for the lifestyle subscale.
When Kirkley et al  developed the WORC, they argued for the use of "disease-specific" measures to evaluate orthopedic treatment because they are more responsive than global health measures. However, they set out to develop, not only measures specific to the shoulder, but instruments specific to conditions of the shoulder. Now there exist Western Ontario tools for the measurement of disability in shoulder instability (WOSI)  osteoarthritis  and rotator cuff conditions .
The results provided above, however, suggest that generic measures of the shoulder may perform as well as condition-specific measures. The WORC was highly correlated with both the DASH and the SST , and had a standardized response mean similar to these two instruments . Therefore, it may not be necessary to have a tool that is specific to a particular condition in the shoulder. Moreover, the WORC is more time consuming to complete and to score, and may not be as attractive as the other scales for use in a clinical setting.
One advantage of the WORC may be its comprehensiveness. It was designed to tap 5 domains of health and may provide information that is unavailable in the other measures. However, the subscales have not been studied in detail, nor has there ever been a confirmation that the WORC items fall into the 5 domains. The purpose of the present study was to examine the validity of the original 5-domain model of the WORC by performing factor analysis.
The data were drawn from a database that included all patients who were to undergo arthroscopic acromioplasty for surgical management of impingement or rotator cuff pathology of the shoulder (Table 3) in a tertiary level hospital in Toronto, Canada between October 2000 and July 2004. Complete data were available on 329 (196 males, 133 females) out of a total of 334 patients. All patients subsequently underwent arthroscopic acromioplasty with or without rotator cuff repair. A number of patients had superior labral anterior and posterior (SLAP) lesions that required surgical repair.
The data of this study were prospectively collected. All patients were sent a number of questionnaires that included the WORC, 3–4 weeks before surgery via mail. Just prior to surgery, the patients were then seen by a physical therapist who performed some physical tests (not reported in this study), and checked that the questionnaires were completed. The data extracted for this study included demographics (Table 3) and the scores on each of the individual items of the WORC questionnaire.
As indicated previously, the WORC is a 21-item questionnaire examining the impact of rotator cuff pathology on "quality of life". Subjects answer each question on a 100 mm visual analogue scale and the higher numbers indicate worse pain or difficulty. The questions in each of the theoretical domains are presented together. The WORC total is obtained by adding the scores on all the items. The subscale scores are totals of the item scores in that domain.
The WORC questionnaire has been published in full . However the 1998 copyright version obtained from the authors and used in this study varies slightly from the published version. The minor differences are noted in Table 4.
Descriptive statistics were calculated for the 21 items, for the subscale scores and for the total WORC score. Correlations between the items and between the subscales were examined with Pearson Product Moment Correlations. A Cronbach's alpha was calculated to determine the internal consistency of the total score and the subscale scores.
Principal component analysis was the extraction method used for the factor analysis. Only factors with eigenvalues greater than 1 were considered. The Kaiser-Meyer-Olkin Measure and Bartlett's Test of Sphericity were performed to determine whether the data were suitable for factor analysis.  Because all the subscales were correlated, an oblique rotation method (SPSS direct oblimin option, SPSS version 11.0.1, SPSS Inc) was used. An item was considered to be loaded on a factor if its pattern matrix coefficient was .5 or greater. We also noted those items that loaded between .4 and .5 but had no higher loading on another factor.
The descriptive statistics, alpha coefficients and inter-item correlations are outlined in Table 5. The correlations between items ranged from .09 to .70, with the lowest correlations being between the emotion items and two of the sports/recreation items. The correlations between items within a subscale varied between .29 and .70. The correlation between subscale scores varied between .53 and .72. Internal consistency of the subscales was .72 to .82 (Cronbach's alpha). The Cronbach's alpha for the total scale was .93.
The data met the criteria for factor analysis. As can be seen from Table 6, the factor analysis revealed 3 factors that explained 57% of the variance. The factors converged in 19 iterations. Factor 1 included all the emotional items and some symptoms not related to specific tasks (shoulder clicking, neck discomfort, and affect on fitness). Three additional items loaded between .4 and .5. They were all questions about pain. Two of the sports items (ability to throw hard/far, and difficulty with push-ups) loaded on factor 2, with the weakness item loading between .4 and .5. The third factor included several items that asked about difficulty performing specific activities. The Cronbach's alpha values for the three factors were: .87 (9 items), .67 (3 items) and .89 (8 items) respectively.
To see the factor loadings with items listed by the domains of the original scale, see additional file 1: Pattern Matrix by domains.doc.
The main purpose of this study was to determine whether the WORC items fell into 5 domains as proposed by the creators of the scale. Although some of the items grouped together as hypothesized, the factor analysis did not support the 5-domain construct of the WORC.
The factor analysis revealed 3 factors, not 5. The 3 factors appear to be: 1) symptoms and emotions, 2) strenuous shoulder tasks, and 3) difficulty with daily tasks. Based on the groupings, it appears that symptoms of pain are associated with emotions, and lack of range of motion or stiffness with difficulty with daily activities. The symptom of "weakness' was associated with two very specific shoulder tasks – throwing hard and push-ups. Based on the mean values for these two items (S8, S9), they were likely the most difficult tasks as well. Thus it is not surprising that "weakness" was associated with difficulty with these activities.
Although factor analysis has not been previously performed on the WORC, other authors have reported a mix of symptoms, disability and social/emotional items within factors derived from other shoulder measures. Veehof et al  noted that all 30 items of the DASH loaded positively on the first factor following principal component factor analysis. Only 3 loaded less than .50. The DASH has questions on physical function, symptoms and social/role function. Similarly, Roddey et al  reported that both the pain and disability items of the SPADI loaded on one factor (.613 to .905). On the other hand, two factors were derived from the Simple Shoulder Test (SST) , which was designed to measure one construct, functional ability in activities of daily living. All of these results suggest that patients with shoulder problems may not differentiate disability and symptoms, and that such theoretical groupings of items are not appropriate.
This lack of separation of pain and disability has been seen in measures of the lower limb as well. Kennedy et al  found that the items of the Western Ontario and McMaster Osteoarthritis Index (WOMAC) factored out on type of activity rather than pain or difficulty. The authors  felt their results might be due to the similarity of the questions in the two domains. For example, 'pain with sitting or lying' is in the pain subscale, and 'difficulty with lying in bed' and 'difficulty with sitting' are both in the physical function subscale. There is no such duplication in the WORC items, and yet, in the present study, there was at least one symptom question, and one "difficulty" question in each factor. Thus, it may be that individuals do not inherently separate symptoms and functional ability in musculoskeletal conditions, no matter how the questions are worded.
In their systematic review of shoulder measures, Bot et al  considered a measure to have good internal consistency when its structure was explored by factor analysis, and Cronbach's alpha for each separate factor was .70 to .90. Two of the three subscales derived from the factor analysis met this criterion. The middle factor/subscale, with only 3 items, did not meet the .70 criterion. However, the Cronbach's alpha increased to .70 when the weakness item, which loaded only .41, was removed. The other two factor/subscales had alpha coefficients higher than the original subscales.
The WORC was originally developed and tested on a heterogeneous group of patients that likely had a wider range of disease severity than the pre-surgical patients used in the present study. Because Kirkley et al  did not present any descriptive statistics for the total or subscale scores of the WORC, the actual range of disability of the subjects in the two studies can not be directly compared. Even so, it is possible, that the results might have been closer to the 5-domain model proposed by Kirkley et al  if the subjects were similar in the two studies. However, one would expect a robust measure to have similar properties when used on all types of patients for which it was intended. Additional research should be conducted to confirm the subscales found in the present study, to examine their properties and determine the value of their use in the clinical or research setting.
The results of this study indicate that the WORC has 3 factors, which explain 57% of the variance. All factors include both 'function' and 'symptom' questions. The three items from the original emotional scale were the only ones that grouped together, but that factor also included items from 3 other subscales. The results of this factorial analysis do not support the 5-domain structure proposed by the creators of the WORC. Based on the results of the present study and on previous work conducted on the WORC, there does not appear to be a significant advantage to using this condition-specific questionnaire over some other well-established measures for the shoulder.
Kirkley A, Alvarez C, Griffin S: The development and evaluation of a disease-specific quality-of-life questionnaire for disorders of the rotator cuff: The Western Ontario Rotator Cuff Index. Clin J Sport Med. 2003, 13: 84-92. 10.1097/00042752-200303000-00004.
Getahun TY, MacDermid JC, Patterson SD: Concurrent validity of patient rating scales in assessment of outcome after rotator cuff repair. J Musculoskelet Res. 2000, 4: 119-127. 10.1142/S021895770000015X.
Holtby R, Razmjou H: Measurement properties of a rotator cuff outcome measure in patients undergoing shoulder surgery: A preliminary report. J Shoulder Elbow Surg.
MacDermid JC, Drosdowech D, Faber K: Responsiveness of self-report scales in patients recovering from rotator cuff surgery. J Shoulder Elbow Surg.
Kirkley A, Griffin S, McLintock H, Ng L: The development and evaluation of a disease-specific quality of life measurement tool for shoulder instability. The Western Ontario Shoulder Instability Index (WOSI). Am J Sports Med. 1998, 26: 764-772.
Lo IK, Griffin S, Kirkley A: The development of a disease-specific quality of life measurement tool for osteoarthritis of the shoulder: The Western Ontario Osteoarthritis of the Shoulder (WOOS) Index. Osteoarthritis Cartilage. 2001, 9: 771-778. 10.1053/joca.2001.0474.
Norusis MJ: Advanced statistics SPSS/PC+. 1986, Chicago: SPSS Inc
Veehof MM, Sleegers EJ, van Veldhoven NH, Schuurman AH, Meeteren NL: Psychometric qualities of the Dutch language version of the Disabilities of the Arm, Shoulder, and Hand Questionnaire (DASH-DLV). J Hand Ther. 2004, 15: 347-354.
Roddey TS, Olson SL, Cook KF, Gartsman GM, Hanten W: Comparison of the University of California-Los Angeles Shoulder Scale and the Simple Shoulder Test with the shoulder pain and disability index: single-administration reliability and validity. Phys Ther. 2000, 80: 759-768.
Kennedy D, Stratford PW, Pagura SM, Wessel J, Gollish JD, Woodhouse LJ: Exploring the factorial validity and clinical interpretability of the Western Ontario and McMaster Universities Osteoarthritis Index (WOMAC). Physiother Can. 2003, 55: 160-168.
Bot SD, Terwee CB, van der Windt DA, Bouter LM, Dekker J, de Vet HC: Clinimetric evaluation of shoulder disability questionnaires: a systematic review of the literature. Ann Rheum Dis. 2004, 63: 335-341. 10.1136/ard.2003.007724.
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2474/6/22/prepub
The authors wish to thank Terry Leeke, data management consultant at the Research Facilitation Office of the Centre for Research In Women's Health for development of the shoulder database, from which data for this study were extracted.
The author(s) declare that they have no competing interests.
JW proposed the study, developed the research protocol, and wrote the first draft of the paper. HR was responsible for subject selection, data collection and management of the database. YM was involved in the review of the literature, data input and initial analysis of data. RH was involved in patient recruitment and providing clinical and surgical diagnosis. All authors were involved in the preparation of the manuscript, and read and approved the final version.