The factor validity of the Western Ontario Rotator Cuff Index

Background The Western Ontario Rotator Cuff Index (WORC) is a self-report questionnaire developed specifically to evaluate disability in persons with pathology of the rotator cuff of the shoulder. The authors created items in 5 categories based on a model of quality of life, but never validated this structure. The purpose of this study was to examine the validity of the original 5-domain model of the WORC by performing factor analysis. Methods Three hundred twenty nine subjects (age, mean: 52, SD: 12) were tested prior to undergoing surgery for rotator cuff pathologies. They completed the WORC, a self-report questionnaire, which has 21 items on the effect of the rotator cuff problem on symptoms, activities and emotions. Statistical calculations included correlations between items, Cronbach's alpha of the total scale and subscales, and principal component factor analysis with oblique rotation. Results Correlations ranged from .09 to .70 between all the items, from .29 to .70 between items within a subscale, and from .53 to .72 between subscale scores. Cronbach's alpha was .93 for the total scale, and .72 to .82 for the subscales. The factor analysis produced 3 factors that explained 57% of the variance. The first factor included symptoms and emotional items, the second included strength items and the third included daily activities. Conclusion The results of this study did not support the 5-domain model of the WORC.


Background
The Western Ontario Rotator Cuff Index is a recent selfreport questionnaire that was designed to measure "health-related quality of life" in persons with injuries and conditions of the rotator cuff of the shoulder. Kirkley et al [1] felt the measure should represent the impact of the condition on health as defined by the World Health Organization -"a state of complete physical, mental and social well-being". They, therefore, included items in 5 domains in the questionnaire: 1) pain and physical symptoms, 2) sports and recreation, 3) work, 4) lifestyle, and 5) emotions. The authors followed a systematic, clinimetric method of generating and reducing the items. This resulted in 21 items that respondents answered on visual analogue scales (VAS) with anchors such as no pain/difficulty and extreme pain/difficulty. Items for the WORC were derived from published health status scales, functional measures of the shoulder, discussions with healthcare professionals, and interviews with 30 patients from a registry of 150 with rotator cuff pathology. Both professionals and patients were asked to identify ways in which the shoulder condition affected quality of life in general, and the 5 domains in particular. The 30 patients interviewed included males and females, aged 30-76, with different degrees of rotator cuff pathology from tendinitis to massive tears.
An original list of 321 items was reduced to 76 by the investigators eliminating duplicated, incomprehensible or ambiguous items. A random selection of 100 patients from the same registry were then asked to indicate whether they experienced each of the items, and to rate the importance of the symptom/disability to their overall shoulder functioning. A frequency importance product was calculated for each item and the 50 items with the highest values were correlated with each other. For every pair of items with coefficients greater than 0.6, one of the items was eliminated, resulting in the final 21 questions. It is not clear whether this criterion applied to items across domains because the only example provided included 3 items from the same domain.
In that same paper [1], the authors reported an ICC for reliability of .96 when they tested subjects over a 2-week period and omitted those who reported any change on a global rating scale. The ICCs for the subscales ranged from .54 (4-item work) to .91 (6-item physical symptoms).
Construct validity has been tested by the original authors [1] and others [2,3] Table 1). The correlations of the change scores were in a similar range (.44 to .85).
Two studies [3,4] have examined the responsiveness of the WORC and other shoulder measures by calculating the standardized response mean (SRM) in patients who have been measured before and after surgery (see Table 2). It should be noted that the SRM of the WORC was not noticeably different from the comparative measures (Constant, SST and DASH) in the same study. Holtby and Razmjou [3] had lower overall SRMs than MacDermid et al [4] who included only the responders in their calculations. MacDermid et al [4] also reported the SRM for the subscales of the WORC. The values ranged from 1.2 for the work subscale to 1.8 for the lifestyle subscale.
When Kirkley et al [1] developed the WORC, they argued for the use of "disease-specific" measures to evaluate orthopedic treatment because they are more responsive than global health measures. However, they set out to develop, not only measures specific to the shoulder, but instruments specific to conditions of the shoulder. Now there exist Western Ontario tools for the measurement of disability in shoulder instability (WOSI) [5] osteoarthritis [6] and rotator cuff conditions [1].
The results provided above, however, suggest that generic measures of the shoulder may perform as well as condi-  [2] Cross-sectional .88 .91 tion-specific measures. The WORC was highly correlated with both the DASH and the SST [2], and had a standardized response mean similar to these two instruments [4]. Therefore, it may not be necessary to have a tool that is specific to a particular condition in the shoulder. Moreover, the WORC is more time consuming to complete and to score, and may not be as attractive as the other scales for use in a clinical setting.
One advantage of the WORC may be its comprehensiveness. It was designed to tap 5 domains of health and may provide information that is unavailable in the other measures. However, the subscales have not been studied in detail, nor has there ever been a confirmation that the WORC items fall into the 5 domains. The purpose of the present study was to examine the validity of the original 5domain model of the WORC by performing factor analysis.

Subjects
The data were drawn from a database that included all patients who were to undergo arthroscopic acromioplasty for surgical management of impingement or rotator cuff pathology of the shoulder (

Design
The data of this study were prospectively collected. All patients were sent a number of questionnaires that included the WORC, 3-4 weeks before surgery via mail. Just prior to surgery, the patients were then seen by a physical therapist who performed some physical tests (not reported in this study), and checked that the questionnaires were completed. The data extracted for this study included demographics (Table 3) and the scores on each of the individual items of the WORC questionnaire.

WORC measure
As indicated previously, the WORC is a 21-item questionnaire examining the impact of rotator cuff pathology on "quality of life". Subjects answer each question on a 100 mm visual analogue scale and the higher numbers indicate worse pain or difficulty. The questions in each of the theoretical domains are presented together. The WORC total is obtained by adding the scores on all the items. The subscale scores are totals of the item scores in that domain.
The WORC questionnaire has been published in full [1]. However the 1998 copyright version obtained from the authors and used in this study varies slightly from the published version. The minor differences are noted in Table 4.

Data analyses
Descriptive statistics were calculated for the 21 items, for the subscale scores and for the total WORC score. Correlations between the items and between the subscales were examined with Pearson Product Moment Correlations. A Cronbach's alpha was calculated to determine the internal consistency of the total score and the subscale scores.
Principal component analysis was the extraction method used for the factor analysis. Only factors with eigenvalues greater than 1 were considered. The Kaiser-Meyer-Olkin Measure and Bartlett's Test of Sphericity were performed to determine whether the data were suitable for factor analysis. [7] Because all the subscales were correlated, an oblique rotation method (SPSS direct oblimin option, SPSS version 11.0.1, SPSS Inc) was used. An item was considered to be loaded on a factor if its pattern matrix coef- ficient was .5 or greater. We also noted those items that loaded between .4 and .5 but had no higher loading on another factor.

Results
The descriptive statistics, alpha coefficients and inter-item correlations are outlined in Table 5. The correlations between items ranged from .09 to .70, with the lowest cor-  The data met the criteria for factor analysis. As can be seen from Table 6, the factor analysis revealed 3 factors that explained 57% of the variance. The factors converged in 19 iterations. Factor 1 included all the emotional items and some symptoms not related to specific tasks (shoulder clicking, neck discomfort, and affect on fitness). Three additional items loaded between .4 and .5. They were all questions about pain. Two of the sports items (ability to throw hard/far, and difficulty with push-ups) loaded on factor 2, with the weakness item loading between .4 and .5. The third factor included several items that asked about difficulty performing specific activities. The Cronbach's alpha values for the three factors were: .87 (9 items), .67 (3 items) and .89 (8 items) respectively.
To see the factor loadings with items listed by the domains of the original scale, see additional file 1: Pattern Matrix by domains.doc.

Discussion
The main purpose of this study was to determine whether the WORC items fell into 5 domains as proposed by the creators of the scale. Although some of the items grouped together as hypothesized, the factor analysis did not support the 5-domain construct of the WORC. The factor analysis revealed 3 factors, not 5. The 3 factors appear to be: 1) symptoms and emotions, 2) strenuous shoulder tasks, and 3) difficulty with daily tasks. Based on the groupings, it appears that symptoms of pain are associated with emotions, and lack of range of motion or stiffness with difficulty with daily activities. The symptom of "weakness' was associated with two very specific shoulder tasks -throwing hard and push-ups. Based on the mean values for these two items (S8, S9), they were likely the most difficult tasks as well. Thus it is not surprising that "weakness" was associated with difficulty with these activities.
Although factor analysis has not been previously performed on the WORC, other authors have reported a mix of symptoms, disability and social/emotional items within factors derived from other shoulder measures. Veehof et al [8] noted that all 30 items of the DASH loaded positively on the first factor following principal component factor analysis. Only 3 loaded less than .50. The DASH has questions on physical function, symptoms and social/role function. Similarly, Roddey et al [9] reported that both the pain and disability items of the SPADI loaded on one factor (.613 to .905). On the other hand, two factors were derived from the Simple Shoulder Test (SST) [9], which was designed to measure one construct, functional ability in activities of daily living. All of these results suggest that patients with shoulder problems may not differentiate disability and symptoms, and that such theoretical groupings of items are not appropriate.
This lack of separation of pain and disability has been seen in measures of the lower limb as well. Kennedy et al [10] found that the items of the Western Ontario and McMaster Osteoarthritis Index (WOMAC) factored out on type of activity rather than pain or difficulty. The authors [10] felt their results might be due to the similarity of the questions in the two domains. For example, 'pain with sitting or lying' is in the pain subscale, and 'difficulty with lying in bed' and 'difficulty with sitting' are both in the physical function subscale. There is no such duplication in the WORC items, and yet, in the present study, there was at least one symptom question, and one "difficulty" question in each factor. Thus, it may be that individuals do not inherently separate symptoms and functional ability in musculoskeletal conditions, no matter how the questions are worded.
In their systematic review of shoulder measures, Bot et al [11] considered a measure to have good internal consistency when its structure was explored by factor analysis, and Cronbach's alpha for each separate factor was .70 to .90. Two of the three subscales derived from the factor analysis met this criterion. The middle factor/subscale, with only 3 items, did not meet the .70 criterion. How-ever, the Cronbach's alpha increased to .70 when the weakness item, which loaded only .41, was removed. The other two factor/subscales had alpha coefficients higher than the original subscales.
The WORC was originally developed and tested on a heterogeneous group of patients that likely had a wider range of disease severity than the pre-surgical patients used in the present study. Because Kirkley et al [1] did not present any descriptive statistics for the total or subscale scores of the WORC, the actual range of disability of the subjects in the two studies can not be directly compared. Even so, it is possible, that the results might have been closer to the 5-domain model proposed by Kirkley et al [1] if the subjects were similar in the two studies. However, one would expect a robust measure to have similar properties when used on all types of patients for which it was intended. Additional research should be conducted to confirm the subscales found in the present study, to examine their properties and determine the value of their use in the clinical or research setting.

Conclusion
The results of this study indicate that the WORC has 3 factors, which explain 57% of the variance. All factors include both 'function' and 'symptom' questions. The three items from the original emotional scale were the only ones that grouped together, but that factor also included items from 3 other subscales. The results of this factorial analysis do not support the 5-domain structure proposed by the creators of the WORC. Based on the results of the present study and on previous work conducted on the WORC, there does not appear to be a significant advantage to using this condition-specific questionnaire over some other well-established measures for the shoulder.