Agreement between a self-administered questionnaire on musculoskeletal disorders of the neck-shoulder region and a physical examination
© Perreault et al; licensee BioMed Central Ltd. 2008
Received: 15 January 2007
Accepted: 17 March 2008
Published: 17 March 2008
In epidemiological studies on neck-shoulder disorders, physical examination by health professionals, although more expensive, is usually considered a better method of data collection than self-administered questionnaires on symptoms. However, little is known on the comparison of these two methods of data collection. The agreement between self-administered questionnaires and the physical examination on the presence of neck-shoulders disorders was assessed in the present study.
This study was conducted among clerical workers using video display units. Prevalent cases were workers for whom neck-shoulder symptoms were present for at least 3 days during the previous 7 days and for whom pain intensity was greater than 50 mm on a 100 mm visual analogue scale. All 85 workers meeting this definition and a random sample of 102 workers who did not meet this definition were selected. Physical examination included measures of active range of motion and musculoskeletal strength. Cohen's kappa and global percent agreement were calculated to compare the two methods of data collection. The effect on the agreement of different question and physical examination definitions and the importance of the time interval elapsed between the administrations of the tests were also evaluated.
Kappa coefficients ranged from 0.19 to 0.54 depending on the definitions used to ascertain disorders. The agreement was highest when the two instruments were administered 21 days apart or less (Kappa = 0.54, global agreement = 77%). It was not substantially improved by the addition of criteria related to functional limitations or when comparisons were made with alternative physical examination definitions. Pain intensity recorded during physical examination maneuvers was an important element of the agreement between questionnaire and physical examination findings.
These results suggest a fair to good agreement between the presence of musculoskeletal disorders ascertained by self-administered questionnaire and physical examination that may reflect differences in the constructs measured. Shorter time lags result in better agreement. Investigators should consider these results before choosing a method to measure the presence of musculoskeletal disorders in the neck-shoulder region.
Musculoskeletal disorders are among the principal causes of activity limitation and long term disability [1–3]. In 2004 musculoskeletal disorders accounted for 38% of work-related problems compensated by the Quebec Workers' Compensation Board (CSST) [4, 5]. For the same year, new cases of musculoskeletal disorders (including low back pain) resulted in nearly 130 million dollars in salary compensation alone [4, 5]. In the United States, musculoskeletal disorders accounted for 30% of the injuries and illnesses with days away from work in 2005 . According to the US Bureau of Labor Statistics, the median length of absence resulting from musculoskeletal disorders was 9 days; among those problems, shoulders disorders resulted in the longest absences from work with a median of 15 days .
In epidemiological studies, data on neck-shoulder disorders are often collected by physical examination [7, 8], by questionnaire [9–15] or with both instruments [16–24]. Physical examination by health professionals is usually recognized as more objective than questionnaires. However, questionnaires permit data collection on many participants for a fraction of the cost and time of a physical examination. Few epidemiological studies on neck and upper extremity musculoskeletal disorders have systematically compared the findings of questionnaires with those obtained by physical examination [16, 19, 22–25]. Only four studies published in English have reported the sensitivity and specificity of a questionnaire compared to clinical examination of the neck-shoulder region to identify individuals with neck-shoulder disorders [16, 19, 23, 24].
The present study was part of a larger investigation on the prevalence of musculoskeletal disorders among video display unit (VDU) users . The main objective of the present study was to assess the agreement between a self-administered questionnaire and the physical examination made by a health professional on the presence of musculoskeletal disorders of the neck-shoulder region. Secondary objectives were to assess the effects on the agreement of different questionnaire and physical examination definitions and the importance of the time interval elapsed between the administrations of the tests.
Study setting and selection of participants
Study participants were selected from a population of 627 women and men working in a large university and in other institutions involved in university services. To be eligible, the workers had to meet the following criteria: 1) be a clerical worker, technician, professional or executive, and 2) use a VDU for at least five hours per week. All participants had to provide an informed consent. This study was approved by the ethics committees of all the institutions involved.
Neck-shoulder disorders were defined by the presence of symptoms for at least three days during the last seven days, with peak pain intensity in the last week greater than 50 millimeters on a 100-millimeter visual analogue scale (VAS). For the agreement study, all workers meeting this definition of neck-shoulder disorders (n = 85) and a random 20% of those that did not meet this definition (n = 102) were selected. The final sample for this agreement study thus included 187 workers.
Arrangements had been made with the employers to allow data collection during working hours. Workers completed a self-administered questionnaire at their workstation. They were also asked to attend a physical examination at the workplace. All workers were seen between March 1994 and May 1996.
Definitions and corresponding prevalence of musculoskeletal problems in the neck-shoulder region among all VDU users (n = 627)
Primary definition : Presence of symptoms for at least 3 days during the last 7 days, with worst pain intensity greater than 50 mm on the 100 mm VAS scale.
Limitations in activities of daily living : Primary definition and a score equal or greater than 2 on the functional limitations scale.
Limitations in work activities : Primary definition and limited work activities.*
Limitations in household activities : Primary definition and limited household activities.*
Limitations in leisure activities : Primary definition and limited leisure activities.*
Primary definition: any one of the following three :
• Diminution ≥ 30% of the normal active range of motion (American Academy of Orthopaedic Surgeons, 1966);
• Diminution of normal muscular strength (score ≤ 4 on the Lovett scale) (Daniels, 1995);
• Pain of moderate intensity or worse (≥ 3 on the 11-point Numerical Rating Scale) produced at the relevant site during any maneuver.
Definition based solely on decreased range of motion or muscle strength : One of the following two :
• Diminution ≥ 30% of the normal active range of motion (American Academy of Orthopaedic Surgeons, 1966);
• Diminution of normal muscular strength (score ≤ 4 on the Lovett scale) (Daniels, 1995).
Definition based solely on pain manifested during maneuvers :
• Pain of moderate intensity or worse (≥ 3 on the 11-point Numerical Rating Scale) produced at the relevant site during any maneuver.
The physical examination was performed according to a standard protocol used in previous studies [29, 31]. At the beginning of the study, a sample of ten participants was successively examined by two occupational therapists blind to each other's scoring. Discordant results were discussed to ensure standardization of the procedures. After standardization of the procedure, the same trained occupational therapist, blind to the participants' questionnaire answers, performed all the physical examinations. The physical examination was composed of 78 items, of which 18 were related to the neck-shoulder region. The examination included measures of active range of motion (ROM) and muscular strength. Active range of motion was measured with a universal goniometer (360 degrees) and a small half-circle goniometer (180 degrees). Measured joint movements were: neck flexion, extension and lateral rotations, shoulder flexion, abduction and external rotation, wrist flexion and extension and movements of the fingers including thumb flexion and extension. Muscular strength was assessed by manual muscle testing. All these maneuvers aimed at assessing the integrity and the performance of the structures and the soft tissues surrounding joints. Decrease in ROM was considered significant if it was 30% or less of the normal expected active range of motion, based on the norms of the American Academy of Orthopaedic Surgeons . Decrease in muscular strength was considered significant if it was scored 4 or less on the Lovett scale . After performing each maneuver, the subject was asked to record his/her pain level on an 11-point numerical rating scale (NRS). NRS are more appropriate to use in face-to-face and telephone interviews than VAS, and their psychometric qualities are comparable to those of VAS [35, 36]. Pain was considered significant if it was reported at the relevant site during maneuvers and was of moderate intensity or worse (a score of 3/10 or more).
The primary physical examination case definition identified those showing limited range of motion or decreased muscular strength or the presence of site-specific pain during maneuvers. Two other definitions were also used: one based solely on decreased range of motion or muscular strength and another based solely on pain manifested during maneuvers (see Table 1 for complete definitions).
Data analyses were performed with the SAS software. All workers were classified as cases or non-cases according to each of the questionnaire and physical examination definitions. The prevalence of musculoskeletal disorders in the neck-shoulder region measured according to the five questionnaire and the three physical examination definitions was estimated for the entire VDU study sample. This was done on the basis of workers who were examined (n = 187), using a weighted sum of the proportion of physical examination cases among workers negative and positive to the questionnaire.
Cohen's Kappa and global percent agreement were used as measures of agreement between results obtained from the self-administered questionnaire and the physical examination . Cohen's Kappa is a measure of agreement corrected for the agreement that could be expected by chance alone [38–40]. For all Kappa values, 95% confidence intervals were calculated . Landis and Koch (1977)  and Fleiss (1981)  have presented different ranges of values for Kappa according to the degree of agreement they suggest. According to them, Kappa values lower than 0.40 represent a poor agreement beyond chance, values between 0.40 and 0.75 are considered as fair to good agreement beyond chance and Kappa values higher than 0.75 represent excellent agreement beyond chance. The global percent agreement is the raw proportion of workers with the same classification on both measures . Percent agreement among cases and non-cases, which corresponds to positive and negative predictive values, was also calculated, as well as sensitivity and specificity . Finally, a stratified analysis was performed to determine the effect on the agreement of the time elapsed between the administrations of the questionnaire and the physical examination. The chi-square test was used to compare percentages.
The participation rate was 84% (89.2% for the cases and 77.7% for the non-cases according to the primary questionnaire definition). The VDU users in the agreement study were similar on demographic and occupational characteristics to all VDU users. Study participants were primarily female (83%). The mean age was 44 years. More than 80% of the participants were clerical workers, 11% were professional and executives and 7% were technicians. The average use of VDU was 20 hours per week.
According to the questionnaire definitions, the prevalence of musculoskeletal disorders varied from 2.9% to 17.1% (Table 1). More positive neck-shoulder findings were reported from the physical examination than from the self-administered questionnaire.
Distribution of study participants according to the primary questionnaire and physical examination case definitions (n = 187)
N (expected value)
N (expected value)
Agreement between the five questionnaire definitions and the primary physical examination definition
Agreement among cases(1)
Agreement among non-cases(2)
1. Primary (n = 187)(3)
2. Limitations in activities of daily living (n = 153)(3)
3. Limitations in work activities (n = 128)(3)
4. Limitations in household activities (n = 135)(3)
5. Limitations in leisure activities (n = 138)(3)
Agreement between the three physical examination definitions and the primary questionnaire definition (n = 187)
Physical Examination definitions
Agreement among cases(1)
Agreement among non-cases(2)
1. Primary Definition
2. Definition based on decreased range of motion or muscular strength
3. Definition based solely on pain
Effect of time elapsed between the administrations of the questionnaire and the physical examination(1) (n = 187)
Agreement among cases(2)
Agreement among non-cases (3)
≤ 21 days (n = 92)
>21 days (n = 95)
In this study of VDU users, the agreement between a self-administered questionnaire on musculoskeletal disorders of the neck-shoulder region and a physical examination of the same region was examined in a sample of university clerical workers. Prevalence figures observed with questionnaire definitions were lower than those obtained from physical examination definitions. Results show an overall Kappa of 0.44 and a global agreement of 72% between the two instruments. The agreement was not substantially improved by the addition of questionnaire criteria related to functional limitations. The agreement diminished when the physical examination definition excluded the manifestation of pain. The percent agreement tended to be higher among cases than among non-cases. Higher agreement was observed with shorter time lapses between the administrations of the tests.
In order to be valid, a measure must first be reliable . The questionnaire used here was adapted from questionnaires used in previous studies [9, 28–31]. Some items were taken from the Standardized Nordic Questionnaire, which showed an acceptable degree of reliability for the neck-shoulder region [27, 28]. Furthermore, previous studies suggested that questions related to the presence, duration and intensity of symptoms provide reliable information on musculoskeletal symptoms [27, 28, 45]. Thus, it is reasonable to consider that the questionnaire used in the present study had an acceptable level of reliability.
Previous studies also provide evidence of construct validity of subjective symptoms reported in questionnaires . Also, VAS are considered among the best instruments to measure pain . To reduce the impact of potential error in recall in this study , only symptoms in the last seven days were considered. Furthermore, the fact that the questionnaire prevalence of musculoskeletal disorders in the neck-shoulder region was comparable (17%) to what was observed in previous studies on VDU workers [15, 21] provides further support for the validity of outcome measures obtained from the questionnaire.
The results of the current study suggest a fair to good agreement between the presence of neck-shoulder disorders ascertained by self-administered questionnaire and physical examination. This finding is in accordance with those obtained in previous studies comparing data from questionnaire with clinical examination to identify cases of neck-shoulder disorders [16, 19, 23, 24]. These earlier studies have concluded that self-reported neck-shoulder symptoms by questionnaire gave fairly-good to good picture of the neck-shoulders disorders prevalence.
According to previous studies, tests used in physical examination, especially measurement of range of motion and manual muscle testing, have poor to good reliability [47–53]. However, the use of a rigorous standardized protocol, pretested by the examiner at the beginning of the current study, and the fact that only one person examined all the workers favored reliability. In their literature review, Gajdosik and Bohannon (1987) concluded that there was acceptable content validity for the measurement of range of motion . Nevertheless, the comparisons in the present study might have been compromised at least in part by measurement error which could explain some lack of association with symptoms.
The Kappa statistic provides a measure of agreement that corrects for the agreement that would be expected by chance alone . Global percent agreement was presented as well. According to suggested classifications [37, 41], all Kappa values reported in this study are relatively low. However, the Kappa statistic is strongly influenced by the prevalence of the phenomenon under study, which is determined by the observed proportion of individuals who fall in each category of the classification table. For a given observed proportion of individuals, Kappa gets its highest value when the expected proportion of positive individuals is small . In this study, the expected proportions were high. This may have led to an underestimation of the true agreement beyond chance [55, 56].
The different questionnaire definitions permitted the assessment of the influence of functional limitations on the agreement. The definition that included limitations in ADL gave similar agreement values when compared to the primary definition. On the other hand, definitions that included limitations in work, household and leisure activities resulted in poorer agreement. The lack of improvement in the agreement observed with the addition of functional limitations criterion may be explained by the fact that the questionnaire definition was already somewhat restrictive (pain reported in the neck-shoulder region for at least three days during the last seven days, with the worst pain intensity greater than 50 millimeters on the 100-millimeter VAS). Under these circumstances, the addition of the ADL limitations may not have contributed more information than the primary definition. Alternatively, the physical examination findings may not correspond closely enough to the domains that limit ADL. Furthermore, limitations measured in a dichotomous format (yes/no items) may not have been sufficiently sensitive in comparison to the more refined ADL limitations question. Finally, low prevalence figures (with more restrictive definitions) lead to lower Kappa values.
The inclusion of criteria related to functional limitations enhanced agreement among cases and reduced agreement among non-cases. Limitations in work and household questionnaire definitions resulted in as much as 92% and 95% agreement among cases respectively. These results suggest that the combined use of physical examination and questionnaire items that include functional limitations is useful when one wants to identify specifically cases that would be confirmed with physical examination. Results showed more workers with limitations in activities of daily living than workers with limitations in work activities. This might suggest that, in order to maintain themselves at work, workers with musculoskeletal disorders reduce their usual daily activities or they may learn to compensate in order to maintain ADL until much later in the disease process. It might also suggest that workers with musculoskeletal problems that manifested at work have already left work, due to the healthy worker effect . Individuals most likely to show limitations in range of motion or in muscular strength on physical examination and to report limitations in work activities on questionnaire were thus not included in this study.
According to our results, the measure of pain intensity provoked by specific maneuvers during the physical examination offered the best agreement when compared with the self-administered questionnaire. A low agreement was obtained with the physical examination definition based solely on decrease in range of motion or muscular strength. These results are consistent with the hypothesis that musculoskeletal disorders are progressive and that patients may have symptoms before objective physical findings appear . Also, cases defined by physical examination of range of motion and muscular strength may have been overlooked by the questionnaire; this would be consistent with previous studies that showed a low correlation between pain intensity and extent of tissue damaged [59, 60].
The definition based on questionnaire may not measure the same concept than the physical examination. While the physical examination measures the integrity and the absolute performance of the structures and tissues, self-reported symptoms are based on actual performance and sensation, much affected by pain perception. This distinction is supported by the large impact that pain has on the agreement. The results of this study suggest that pain intensity is an important feature in the agreement between a questionnaire on musculoskeletal disorders and a physical examination and support the construct validity of a case definition based on symptoms.
The higher prevalence of findings in the physical examination than in questionnaire might be due to the selection criteria used to define non-cases according to the questionnaire. Given that the questionnaire definition was somewhat restrictive, some non-cases were not totally free of symptoms. Indeed, 26 of those 102 workers classified as non-cases according to the primary questionnaire definition had symptoms in the week prior to the questionnaire. This could have lead to a classification bias and could have attenuated the true associations with physical examination.
The time interval elapsed between the administrations of the two tests ranged from two days to six months. Better agreement (k = 0.54) was observed with a smaller time interval (21 days or less). The temporal variability present in musculoskeletal disorder symptoms and the fact that severity of pain in musculoskeletal disorders can vary from day to day depending upon the types of activities the person has engaged in  are inherent difficulties for the measure of agreement between two tests [46, 61]. The longer interval between the tests might have allowed time for real changes in symptoms and consequently may have contributed to the relatively limited agreement found in this study. These results are consistent with those of Björkstén et al. (1999) who observed that shorter reference period for reporting musculoskeletal problems yielded better agreement between a questionnaire and a physical examination .
Finally, the current study's population consisted mainly of employed clerical women, thus the generalizability of the results is limited to similar populations.
In conclusion, the results of this study show that the agreement between a questionnaire on musculoskeletal disorders for the neck-shoulder region and a physical examination is fair to good. Inclusion of items related to functional limitations in questionnaires appears to be of limited value to improve the agreement. It is the physical examination definition that included pain manifestations that offered the best agreement with the questionnaire. A shorter time interval between the administrations of the two tests also yields a better agreement. Investigators should consider these results before choosing a method to measure the presence of musculoskeletal disorders of the neck-shoulder region.
This study was supported by the Quebec Rehabilitation Institute (IRDPQ), the Eastern Quebec Rehabilitation Research Consortium (CORREQ) and the Quebec Research Institute on Occupational Safety and Health (IRSST). Chantal Brisson held a research scholarship from the Medical Research Council of Canada (MRC) at the time of data collection. Clermont E. Dionne is a Quebec Health Research Fund (FRSQ) scholar. The authors thank Isabelle Leroux, MSc, for her contribution to the revision of the manuscript.
- Badley EM, Webster GK, Rasooly I: The impact of musculoskeletal disorders in the population: are they just aches and pains? Findings from the 1990 Ontario Health Survey. J Rheumatol. 1995, 22 (4): 733-739.PubMedGoogle Scholar
- Daveluy C, Pica L, Audet N, Courtemanche R, Lapointe F: Enquête sociale et de santé 1998, 2e édition. 2000, Québec , Institut de la statistique du QuébecGoogle Scholar
- Woolf AD, Pfleger B: Burden of major musculoskeletal conditions. Bull World Health Organ. 2003, 81 (9): 646-656.PubMedPubMed CentralGoogle Scholar
- CSST: Statistiques sur les affections vertébrales 2001-2004. 2005, Québec , Service de la statistique, Direction de la comptabilité et de la gestion de l’informationGoogle Scholar
- CSST: Statistiques sur les lésions en "ITE" du système musculo-squelettique 2001-2004. 2005, Québec , Service de la statistique, Direction de la comptabilité et de la gestion de l’informationGoogle Scholar
- U.S. Department of Labor: Nonfatal occupational injuries and illnesses requiring days away from work in 2005. 2006, Washington DC , Bureau of Labor StatisticsGoogle Scholar
- Ranney D, Wells R, Moore A: Upper limb musculoskeletal disorders in highly repetitive industries: precise anatomical physical findings. Ergonomics. 1995, 38 (7): 1408-1423. 10.1080/00140139508925198.View ArticlePubMedGoogle Scholar
- Toomingas A, Nemeth G, Alfredsson L: Self-administered examination versus conventional medical examination of the musculoskeletal system in the neck, shoulders, and upper limbs. The Stockholm MUSIC I Study Group. J Clin Epidemiol. 1995, 48 (12): 1473-1483. 10.1016/0895-4356(95)00061-5.View ArticlePubMedGoogle Scholar
- Bernard B, Sauter S, Fine L, Petersen M, Hales T: Job task and psychosocial risk factors for work-related musculoskeletal disorders among newspaper employees. Scand J Work Environ Health. 1994, 20: 417-426.View ArticlePubMedGoogle Scholar
- Ariens GA, Bongers PM, Hoogendoorn WE, Houtman IL, van der Wal G, van Mechelen W: High quantitative job demands and low coworker support as risk factors for neck pain: results of a prospective cohort study. Spine. 2001, 26 (17): 1896-1901. 10.1097/00007632-200109010-00016.View ArticlePubMedGoogle Scholar
- Feveile H, Jensen C, Burr H: Risk factors for neck-shoulder and wrist-hand symptoms in a 5-year follow-up study of 3,990 employees in Denmark. Int Arch Occup Environ Health. 2002, 75 (4): 243-251. 10.1007/s00420-001-0298-9.View ArticlePubMedGoogle Scholar
- Hannan LM, Monteilh CP, Gerr F, Kleinbaum DG, Marcus M: Job strain and risk of musculoskeletal symptoms among a prospective cohort of occupational computer users. Scand J Work Environ Health. 2005, 31 (5): 375-386.View ArticlePubMedGoogle Scholar
- Leroyer A, Edme JL, Vaxevanoglou X, Buisset C, Laurent P, Desobry P, Frimat P: Neck, shoulder, and hand and wrist pain among administrative employees: relation to work-time organization and psychosocial factors at work. J Occup Environ Med. 2006, 48 (3): 326-333. 10.1097/01.jom.0000196058.94482.99.View ArticlePubMedGoogle Scholar
- Ostergren PO, Hanson BS, Balogh I, Ektor-Andersen J, Isacsson A, Orbaek P, Winkel J, Isacsson SO: Incidence of shoulder and neck pain in a working population: effect modification between mechanical and psychosocial exposures at work? Results from a one year follow up of the Malmo shoulder and neck study cohort. J Epidemiol Community Health. 2005, 59 (9): 721-728. 10.1136/jech.2005.034801.View ArticlePubMedPubMed CentralGoogle Scholar
- Rossignol AM, Morse EP, Summers VM, Pagnotto LD: Video display terminal use and reported health symptoms among Massachusetts clerical workers. Journal of Occupational Medicine. 1987, 29 (2): 112-118.PubMedGoogle Scholar
- Akesson I, Johnsson B, Rylander L, Moritz U, Skerfving S: Musculoskeletal disorders among female dental personnel--clinical examination and a 5-year follow-up study of symptoms. Int Arch Occup Environ Health. 1999, 72 (6): 395-403. 10.1007/s004200050391.View ArticlePubMedGoogle Scholar
- Andersen JHK: Risk factors in the onset of neck/shoulder pain in a prospective study of workers in industrial and service companies. Occup Environ Med. 2003, 60: 649-654. 10.1136/oem.60.9.649.View ArticlePubMedPubMed CentralGoogle Scholar
- Bergqvist U, Wolgast E, Nilsson B, Voss M: Musculoskeletal disorders among visual display terminal workers: individual, ergonomic, and work organizational factors. Ergonomics. 1995, 38 (4): 763-776. 10.1080/00140139508925148.View ArticlePubMedGoogle Scholar
- Bjorksten MG, Boquist B, Talback M, Edling C: The validity of reported musculoskeletal problems. A study of questionnaire answers in relation to diagnosed disorders and perception of pain. Appl Ergon. 1999, 30 (4): 325-330. 10.1016/S0003-6870(98)00033-7.View ArticlePubMedGoogle Scholar
- Gerr F, Marcus M, Ensor C, Kleinbaum D, Cohen S, Edwards A, Gentry E, Ortiz DJ, Monteilh C: A prospective study of computer users: I. Study design and incidence of musculoskeletal symptoms and disorders. Am J Ind Med. 2002, 41 (4): 221-235. 10.1002/ajim.10066.View ArticlePubMedGoogle Scholar
- Hales TR, Sauter SL, Peterson MR, Fine LJ, Putz-Anderson V, Schleifer LR, Ochs TT, Bernard BP: Musculoskeletal disorders among visual display terminal users in a telecommunications company. Ergonomics. 1994, 37 (10): 1603-1621. 10.1080/00140139408964940.View ArticlePubMedGoogle Scholar
- Juul-Kristensen B, Kadefors R, Hansen K, Bystrom P, Sandsjo L, Sjogaard G: Clinical signs and physical function in neck and upper extremities among elderly female computer users: the NEW study. Eur J Appl Physiol. 2006, 96 (2): 136-145. 10.1007/s00421-004-1220-4.View ArticlePubMedGoogle Scholar
- Kaergaard A, Andersen JH, Rasmussen K, Mikkelsen S: Identification of neck-shoulder disorders in a 1 year follow-up study. Validation Of a questionnaire-based method. Pain. 2000, 86 (3): 305-310. 10.1016/S0304-3959(00)00261-X.View ArticlePubMedGoogle Scholar
- Ohlsson K, Attewell RG, Johnsson B, Ahlm A, Skerfving S: An assessment of neck and upper extremity disorders by questionnaire and clinical examination. Ergonomics. 1994, 37 (5): 891-897. 10.1080/00140139408963698.View ArticlePubMedGoogle Scholar
- Laubli T, Thomas C, Hinnen U, Hunting W, Zeier H, Mion H: Evaluation of Musculoskeletal Disorders by Questionnaire. Sozial-Und Praventivmedizin. 1991, 36 (1): 25-33. 10.1007/BF01322297.View ArticlePubMedGoogle Scholar
- Brisson C, Montreuil S, Punnett L: Effects of an ergonomic training program among video display unit workers. Scand J Work Environ Health. 1999, 25 (3): 255-263.View ArticlePubMedGoogle Scholar
- Dickinson CE, Campion K, Foster AF, Newman SJ, O'Rourke AMT, Thomas PG: Questionnaire development: an examination of the Nordic musculoskeletal questionnaire. Applied Ergonomics. 1992, 23 (3): 197-301. 10.1016/0003-6870(92)90225-K.View ArticlePubMedGoogle Scholar
- Kuorinka I, Jonsson B, Kilbom A, Vinterberg H, Biering-Sorensen F, Andersson G, Jorgensen K: Standardised Nordic questionnaire for the analysis of musculoskeletal symptoms. Applied Ergonomics. 1987, 18 (3): 233-237. 10.1016/0003-6870(87)90010-X.View ArticlePubMedGoogle Scholar
- Punnett L, Fine LJ, Keyserling WM, Herrin GD, Chaffin DB: Back disorders and nonneutral trunk postures of automobile assembly workers. Scand J Work Environ Health. 1991, 17 (5): 337-346.View ArticlePubMedGoogle Scholar
- Silverstein BA, Fine LJ, Armstrong JTJ: Occupational factors and carpal tunnel syndrome. Am J Industr Med. 1987, 11: 343-358. 10.1002/ajim.4700110310.View ArticleGoogle Scholar
- Silverstein BA, Fine LJ, Armstrong TJ: Hand wrist cumulative trauma disorders in industry. Br J Ind Med. 1986, 43: 779-784.PubMedPubMed CentralGoogle Scholar
- Scott J, Huskisson EC: Graphic Representation of Pain. Pain. 1976, 2 (2): 175-184. 10.1016/0304-3959(76)90113-5.View ArticlePubMedGoogle Scholar
- American Academy of Orthopaedic Surgeons, British Orthopaedic Association: Joint motion : method of measuring and recording. 1966, Edinburgh , Churchill Livingstone, 87 p.-Google Scholar
- Daniels L, Worthingham C: Le bilan musculaire: Technique de l'examen clinique. 1995, Paris , Éditions MalouineGoogle Scholar
- Jensen MP, Turner JA, Romano JM: What is the maximum number of levels needed in pain intensity measurement?. Pain. 1994, 58 (3): 387-392. 10.1016/0304-3959(94)90133-3.View ArticlePubMedGoogle Scholar
- Strong J, Ashton R, Chant D: Pain intensity measurement in chronic low back pain. Clin J Pain. 1991, 7 (3): 209-218.View ArticlePubMedGoogle Scholar
- Fleiss JL: Statistical methods for rates and proportion. 1981, New York , Wiley & Sons, 2nd EditionGoogle Scholar
- Armstrong BK, White E, Saracci R: Principles of exposure measurement in epidemiology. 1992, Oxford , Oxford University PressGoogle Scholar
- Bernard PM, Lapointe C: Mesures statistiques en épidémiologie. 1995, Québec , Presses de l'Université du Québec, 314-Google Scholar
- Maclure M, Willett WC: Misinterpretation and misuse of the kappa statistic. J Epidemiol. 1987, 126 (2): 161-169.View ArticleGoogle Scholar
- Landis JR, Koch GG: The measurement of observer agreement for categorical data. Biometrics. 1977, 33: 159-174. 10.2307/2529310.View ArticlePubMedGoogle Scholar
- Altman DG: Practical statistics for medical research. 1991, London , Chapman and HallGoogle Scholar
- Rothman KJ, Greenland S: Modern epidemiology. 1998, Philadelphia , Lippincott-Raven, xiii, 738 p.-2ndGoogle Scholar
- Abramson JH: Survey methods in community medicine : an introduction to epidemiological and evaluative studies. 1984, Edinburgh ; New York , Churchill Livingstone, x, 285 p.-3rdGoogle Scholar
- Baron S, Hales T, Hurrell J: Evaluation of symptom surveys for occupational musculoskeletal disorders. Am J Ind Med. 1996, 29 (6): 609-617. 10.1002/(SICI)1097-0274(199606)29:6<609::AID-AJIM5>3.0.CO;2-E.View ArticlePubMedGoogle Scholar
- Schierhout GH, Myers JE: Is self-reported pain an appropriate outcome measure in ergonomic-epidemiologic studies of work-related musculoskeletal disorders?. Am J Indust Med. 1996, 30 (1): 93-98. 10.1002/(SICI)1097-0274(199607)30:1<93::AID-AJIM16>3.0.CO;2-3.View ArticleGoogle Scholar
- Gajdosik RL, Bohannon RW: Clinical Measurement of Range of Motion - Review of Goniometry Emphasizing Reliability and Validity. Physical Therapy. 1987, 67 (12): 1867-1872.PubMedGoogle Scholar
- Hoppenbrouwers M, Eckhardt MM, Verkerk K, Verhagen A: Reproducibility of the measurement of active and passive cervical range of motion. J Manipulative Physiol Ther. 2006, 29 (5): 363-367. 10.1016/j.jmpt.2006.04.007.View ArticlePubMedGoogle Scholar
- Jepsen JR, Laursen LH, Larsen AI, Hagert CG: Manual strength testing in 14 upper limb muscles - A study of inter-rater reliability. Acta Orthopaedica Scandinavica. 2004, 75 (4): 442-448.View ArticlePubMedGoogle Scholar
- Marx RG, Bombardier C, Wright JG: What do we know about the reliability and validity of physical examination tests used to examine the upper extremity?. Journal of Hand Surgery-American Volume. 1999, 24 (1): 185-193. 10.1053/jhsu.1999.jhsu24a0185.View ArticleGoogle Scholar
- Salerno DF, Franzblau A, Werner RA, Chung KC, Schultz JS, Becker MP, Armstrong TJ: Reliability of physical examination of the upper extremity among keyboard operators. American Journal of Industrial Medicine. 2000, 37 (4): 423-430. 10.1002/(SICI)1097-0274(200004)37:4<423::AID-AJIM12>3.0.CO;2-W.View ArticlePubMedGoogle Scholar
- Valentine RE, Lewis JS: Intraobserver reliability of 4 physiologic movements of the shoulder in subjects with and without symptoms. Archives of Physical Medicine and Rehabilitation. 2006, 87 (9): 1242-1249. 10.1016/j.apmr.2006.05.008.View ArticlePubMedGoogle Scholar
- Viikari-Juntura E: Interexaminer reliability of observations in physical examinations of the neck. Phys Ther. 1987, 67 (10): 1526-1532.PubMedGoogle Scholar
- Cohen JA: A coefficient of agreement for nominal scales. Educational and Psychological Measurment. 1960, 20: 37-46. 10.1177/001316446002000104.View ArticleGoogle Scholar
- Feinstein AR, Cicchetti DV: High agreement but low kappa: I. the problems of two paradoxes. J Clin Epidemiol. 1990, 43 (6): 543-549. 10.1016/0895-4356(90)90158-L.View ArticlePubMedGoogle Scholar
- Byrt T, Bishop J, Carlin JB: Bias, Prevalence and Kappa. Journal of Clinical Epidemiology. 1993, 46 (5): 423-429. 10.1016/0895-4356(93)90018-V.View ArticlePubMedGoogle Scholar
- Punnett L: Adjusting for the healthy worker selection effect in cross-sectional studies. Int J Epidemiol. 1996, 25 (5): 1068-1076. 10.1093/ije/25.5.1068.View ArticlePubMedGoogle Scholar
- Stock SR: Workplace ergonomic factors and the development of musculoskeletal disorders of the neck and upper limbs: a meta-analysis. American Journal of Industrial Medicine. 1991, 19: 87-107. 10.1002/ajim.4700190111.View ArticlePubMedGoogle Scholar
- Gerr F, Letz R, Landrigan PJ: Upper-extremity musculoskeletal disorders of occupational origin. Annu Rev Public Health. 1991, 12: 543-566. 10.1146/annurev.pu.12.050191.002551.View ArticlePubMedGoogle Scholar
- McDowell I, Newell C: Measuring health: A guide to rating scales and questionnaires. 1987, New York , Oxford University Press, 342:Google Scholar
- Croft P, Pope D, Zonca M, Oneill T, Silman A: Measurement of Shoulder Related Disability - Results of a Validation-Study. Annals of the Rheumatic Diseases. 1994, 53 (8): 525-528.View ArticlePubMedPubMed CentralGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2474/9/34/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.