- Research article
- Open Access
- Open Peer Review
Validity of self-assessment of hallux valgus using the Manchester scale
© Menz et al; licensee BioMed Central Ltd. 2010
- Received: 23 May 2010
- Accepted: 20 September 2010
- Published: 20 September 2010
Hallux valgus (HV) is a common condition involving the progressive subluxation of the first metatarsophalangeal joint due to lateral deviation of the hallux and medial deviation of the first metatarsal. The objective of this study was to evaluate the re-test reliability and validity of self-assessment of HV using a simple clinical screening tool involving four standardised photographs (the Manchester scale), in order to determine whether this tool could be used for postal surveys of the condition.
HV was assessed with the Manchester scale in 138 people aged 65 to 93 years of age (102 women and 36 men) as part of a larger randomised controlled trial. At the six month follow-up assessment, HV was reassessed to determine re-test reliability, and participants were asked to self-assess their degree of HV independent of the examiners. Associations between (i) baseline and follow-up assessments of the examiners and (ii) participant and examiner assessments were performed using weighted kappa statistics. Analyses were then repeated after HV was dichotomised as present or absent using unweighted kappa, and sensitivity and specificity of self-assessment of HV was determined.
Re-test reliability of the examiners was substantial to almost perfect (weighted kappa = 0.78 to 0.90), and there was a substantial level of agreement between observations of the participants and the examiners (weighted kappa = 0.71 to 0.80). Overall, there was a slight tendency for participants to rate their HV as less severe than the examiners. When the Manchester scale scores were dichotomised, agreement was substantial to almost perfect for both re-test comparisons (kappa = 0.80 to 0.89) and substantial for comparisons between participants and examiners (kappa = 0.64 to 0.76). The sensitivity and specificity of self-assessment of HV using the dichotomous scale were 85 and 88%, respectively.
The Manchester scale demonstrates high re-test reliability, and self-assessment scores obtained by participants are strongly associated with scores obtained by examiners. These findings indicate that the tool can be used with confidence in postal surveys to document the presence and severity of HV.
- Line Drawing
- Postal Survey
- Percentage Agreement
- Weighted Kappa
- Hallux Valgus
Hallux valgus (HV) is a common condition affecting the forefoot in which the first metatarsophalangeal joint is progressively subluxed due to the lateral deviation of the hallux and medial deviation of the first metatarsal . The resultant deformity often leads to the development of a soft tissue and osseous prominence on the medial aspect of the first metatarsal head, commonly referred to as a "bunion" . Prevalence estimates of HV range from 21 to 65% [3–9], with the largest study so far undertaken (involving 4,249 people aged over 30 years) reporting a prevalence of 28% . HV has been shown to have a detrimental impact on health-related quality of life [11–14], and is associated with impaired gait  and balance  and an increased risk of falls [17, 18] in older people. Surgical correction of HV is one of the most commonly-performed orthopaedic foot and ankle procedures [19, 20].
HV is generally considered to be present when the angle formed by the bisections of the first metatarsal and the proximal phalanx obtained from foot radiographs is greater than 15 degrees [21, 22]. However, because it is not always feasible or necessary to obtain radiographs to assess HV, several other approaches have been suggested, including goniometric assessment, measurement of forefoot girth, and the use of standardised photographs or line drawings [23–26]. The most developed of these tools are the Manchester scale  and a line drawing tool described by Roddy et al . The Manchester scale consists of standardised photographs of feet with four grades of HV (none, mild, moderate and severe). Both re-test and inter-tester reliability of grading HV using the Manchester scale have been found to be excellent (kappa values of 0.77 and 0.86, respectively [25, 27]). More recently, Roddy et al  developed an instrument consisting of five line drawings, each drawing illustrating a sequential increase in the HV angle of approximately 15 degrees. This tool has also been shown to have excellent re-test reliability (kappa = 0.82).
Although either of these tools can be used to provide accurate information regarding the presence and severity of HV, each tool has advantages and disadvantages. The key advantages of the Manchester scale are that the photographs represent real cases of HV selected by a consensus panel of podiatrists to represent the full spectrum of the deformity, and that scores documented using this tool have been shown to be highly correlated with angular measurements obtained from foot radiographs . By comparison, the Roddy et al  tool uses stylised line drawings with hypothetical degrees of deformity, and has not yet been validated against radiographs. The key disadvantage of the Manchester scale is that it has not yet been validated as a self-assessment tool, thereby limiting its application to settings where trained observers are used to document the presence and severity of HV. Therefore, the primary objective of this study was to address this shortcoming by evaluating the level of agreement between trained clinical assessment and self-assessment of HV using the Manchester scale. A secondary objective was to evaluate re-test reliability of clinical observations of HV over a longer period than has been previously undertaken for this tool (i.e. six months compared to two weeks). In doing so, our aim was to determine whether the Manchester scale would be a suitable tool for self-assessment of HV in the context of a postal survey of foot disorders.
Participants were drawn from a larger randomised controlled trial investigating the efficacy of a podiatry intervention to prevent falls (Trial Registration Number: ACTRN12608000065392), the details of which are described elsewhere . Briefly, community dwelling men and women aged 65 years and over were recruited by a mail-out letter from a database of people who were accessing podiatry services at the La Trobe University Health Sciences Clinic, Bundoora, Victoria, Australia as well as from advertisements placed in seniors newspapers and websites. Inclusion criteria included an elevated risk of falling and current foot pain. Exclusion criteria included Parkinson's disease (or other neurodegenerative disorders), lower limb amputation and cognitive impairment. The Human Ethics Committee of La Trobe University approved the study (ID: 07-118) and all participants provided written informed consent.
Manchester scale assessment
At the baseline assessment, all participants were assessed for HV using the Manchester scale by one of two examiners - a physiotherapist with 22 years of general physiotherapy clinical experience (MF) and a physiotherapist with 10 years of general physiotherapy clinical experience (EW). Both examiners had been trained in the use of the tool by an experienced podiatrist (MJS) prior to commencement of the study, using a sample of 36 older people recruited to pilot the clinical assessments used in the randomised controlled trial . This process involved independent assessments by the podiatrist and the two examiners, which was followed by a discussion in which any discrepancies in interpretation of the scale were resolved.
where w represents the weighting, i is the number of the row, j is the number of the column, and k is the total number of categories (in this case, four). The following benchmarks for interpretation of κw scores were used: ≤ 0 = poor, 0.01 to 0.20 = slight, 0.21 to 0.40 = fair, 0.41 to 0.60 = moderate, 0.61 to 0.80 = substantial, and 0.81 to 1.00 = almost perfect . To explore the level of disagreement between examiner and participant assessments, the frequency of disagreement types was determined, i.e. the number of occasions in which scores varied by a single category, 2 categories, and 3 categories. These analyses were performed for left feet, right feet, and with both feet combined.
Secondly, HV was dichotomised using the Manchester scale by merging the first two categories (i.e. scores of 0 or 1) to indicate that HV was absent, and merging the second two categories (i.e. scores of 2 or 3) to indicate that HV was present. This cut-off was based on our previous study where we found that the mean hallux abductus angle obtained from radiographs for participants with a Manchester scale score of 2 was approximately 15 degrees , which is the commonly accepted minimum value for the diagnosis of HV [21, 22]. Re-test reliability and agreement between dichotomous scores obtained by the examiners and the participants was then determined using percentage agreement in addition to the standard (unweighted) kappa statistic (κ), with the same benchmarks for interpretation . These analyses were also performed for left feet, right feet, and with both feet combined.
Thirdly, the sensitivity and specificity were calculated for the dichotomous self-assessment scores, using the examiners' dichotomous scores as the diagnostic "gold standard". This analysis was undertaken for both feet combined.
Age (years) - mean (SD)
Height (cm) - mean (SD)
Weight (kg) - mean (SD)
Body mass index (kg/m2) - mean (SD)
Major medical conditions - n (%)
High blood pressure
Re-test reliability of HV assessment
Associations between baseline and 6 month follow-up assessments of hallux valgus using the Manchester scale (i.e. re-test reliability).
0.88 (0.81 to 0.89)
0.90 (0.89 to 0.91)
0.78 (0.77 to 0.81)
Agreement between examiner and participant assessment of HV
Associations between examiner and participant assessments of hallux valgus using the Manchester scale (i.e. validity).
0.71 (0.62 to 0.73)
0.80 (0.72 to 0.84)
0.76 (0.75 to 0.79)
Frequencies - n (%) of disagreement types between examiner and participant assessments of hallux valgus using the Manchester scale.
Difference = 1
Difference = 2
Difference = 3
Dichotomous assessment of HV
Re-test and examiner vs participant agreement of dichotomous grading of hallux valgus using the Manchester scale.
Baseline vs follow-up
Examiner vs participant
κ (95% CI)
κ (95% CI)
0.89 (0.81 to 0.98)
0.64 (0.49 to 0.78)
0.87 (0.79 to 0.96)
0.76 (0.64 to 0.87)
0.80 (0.72 to 0.87)
0.70 (0.61 to 0.79)
The objectives of this study were to evaluate the re-test reliability and validity of self-assessment of HV using a simple clinical screening tool involving four standardised photographs (the Manchester scale), in order to determine whether this tool could be used for postal surveys of foot disorders. The six month re-test reliability was very high, with κw values between 0.78 and 0.90, and percentage agreement between 95.8 and 98.1%. Slightly lower re-test reliability (κw = 0.77, percentage agreement = 84%) was reported by Menz et al  in three examiners assessing HV severity in 31 older people tested on two occasions, two weeks apart. This difference is likely to be due to the level of experience of the examiners. In the Menz et al  study, none of the three examiners had any experience in assessing foot disorders, whereas in the current study, the two examiners had recently been involved in undertaking foot assessments in a large number of participants involved in a clinical trial. The level of re-test reliability reported here for the Manchester scale is also similar to that reported for the line drawing scale described by Roddy et al , who evaluated the reliability of a single examiner assessing 25 participants on two occasions, three to six months apart. κw values were 0.79 for the left foot, 0.84 for the right foot, and 0.82 when both feet were combined.
There was a high level of agreement between Manchester scale scores documented by the two examiners and those documented independently by the participants. Although there was a slight tendency for participants to rate their HV as less severe than the examiners, overall agreement was substantial (κw values between 0.71 and 0.80 and percentage agreement between 95.1 and 96.1%), and when both feet were combined, 66% of the scores obtained were identical. Where disagreements were identified, the majority related to a difference of one category only. These findings compare favourably to results obtained with the five-level line drawing scale described by Roddy et al , who reported a lower overall κw value of 0.45.
Although the Manchester scale is designed to categorise HV into four severity categories, in some situations it may be useful to have a dichotomous case definition. In this study, we developed a dichotomised case definition of HV by combining the first two categories to indicate that HV is absent, and combining the second two categories to indicate that HV is present. As it cannot be assumed that the reliability and validity of the four level scale is the same as the dichotomised scale, we also analysed the Manchester scale scores after they had been dichotomised. This made little difference to the results, with similarly high re-test reliability (κ values between 0.80 and 0.89) and agreement between the examiners and participants (κ values between 0.64 and 0.76). If it is assumed that the examiners' scores represent the "gold standard", self-assessments performed by the participants demonstrated excellent diagnostic accuracy, with a sensitivity of 85% and a specificity of 88%. In the Roddy et al  study, the dichotomous definition of HV using the line drawings exhibited similar re-test reliability (κ = 0.83), but lower participant-examiner agreement (κ = 0.55) and lower diagnostic accuracy (sensitivity of 75% and specificity of 82%).
The findings reported here suggest that the Manchester scale  may be a slightly more reliable and valid indicator of HV than the Roddy et al  line drawing tool, however a direct comparison of the two tools would be required to adequately ascertain this. Nevertheless, several differences between the tools are worthy of consideration in this context. Firstly, although the inclusion of five rather than four levels of severity in the Roddy et al  tool potentially allows for greater precision, this may also make the classification task slightly more difficult than the four options available in the Manchester scale, particularly for participants assessing their own feet. Secondly, there may be some additional visual assistance provided by the provision of photographs of real feet in the Manchester scale as opposed to line drawings. Thirdly, the two most severe depictions of HV in the Roddy et al  tool are accompanied by an under-riding second toe. Because the second toe may adopt a variety of postures in people with HV (including over-riding  and valgus  toe deformity), the depiction of the under-riding toe may create some confusion, despite the instructions requesting participants to focus only on their big toe. The potential distraction introduced by the inclusion of lesser toe deformity was identified by Garrow et al  when designing the Manchester scale, which resulted in the selection of the most severe HV photograph having no major deformity of the second toe.
The findings reported here need to be considered in the context of several study design limitations. Firstly, we were unable to assess the inter-examiner reliability of HV assessment in this study, as participants were drawn from a randomised controlled trial and all follow-up assessments needed to be conducted by the same examiner who conducted the baseline assessments. However, the inter-examiner reliability reported previously by Garrow et al  was very high (κw values of 0.84 to 0.88). Secondly, the inclusion criteria for the larger trial from which this sample was obtained required participants to have current foot pain, which may have biased the sample towards having a higher than average prevalence of HV. Thirdly, participants' self-assessments were conducted in a clinical setting, and although the examiners did not provide any assistance, it is possible that the self-assessment scores may have been different if participants completed the task in their home environment. Finally, although the Manchester scale provides a useful overall indicator of the degree of angular deformity associated with HV, it is acknowledged that other factors, such as the degree of joint degeneration or sesamoid displacement, may be of equal or greater clinical importance in relation to the functional impact of the condition.
Assessment of HV using the Manchester scale demonstrates high re-test reliability, and self-assessment scores obtained by participants are strongly associated with scores obtained by examiners, irrespective of whether the four-level classification or dichotomised scale are used. These findings indicate that the tool can be used with confidence in postal surveys to document the presence and severity of HV.
This study was funded by a National Health and Medical Research Council of Australia Primary Health Care Project Grant (ID: 433027). HBM is currently a National Health and Medical Research Council fellow (Clinical Career Development Award, ID: 433049).
- Mann R, Coughlin M: Hallux valgus - etiology, anatomy, treatment and surgical considerations. Clin Orthop Relat Res. 1981, 157: 31-41.PubMedGoogle Scholar
- Thomas S, Barrington R: Hallux valgus. Curr Orthop. 2003, 17: 299-307. 10.1016/S0268-0890(02)00184-6.View ArticleGoogle Scholar
- Black JR, Hale WE: Prevalence of foot complaints in the elderly. J Am Podiatr Med Assoc. 1987, 77: 308-311.View ArticlePubMedGoogle Scholar
- Brodie BS, Rees CL, Robins DJ, Wilson AFJ: Wessex Feet: a regional foot health survey, Volume I: The survey. Chiropodist. 1988, 43: 152-165.Google Scholar
- Greenberg L, Davis H: Foot problems in the US. The 1990 National Health Interview Survey. J Am Podiatr Med Assoc. 1993, 83: 475-483.View ArticlePubMedGoogle Scholar
- Crawford VLS, Ashford RL, McPeake B, Stout RW: Conservative podiatric medicine and disability in elderly people. J Am Podiatr Med Assoc. 1995, 85: 255-259.View ArticlePubMedGoogle Scholar
- Benvenuti F, Ferrucci L, Guralnik JM, Gangemi S, Baroni A: Foot pain and disability in older persons: an epidemiologic survey. J Am Geriatr Soc. 1995, 43: 479-484.View ArticlePubMedGoogle Scholar
- Dunn JE, Link CL, Felson DT, Crincoli MG, Keysor JJ, McKinlay JB: Prevalence of foot and ankle conditions in a multiethnic community sample of older adults. Am J Epidemiol. 2004, 159: 491-498. 10.1093/aje/kwh071.View ArticlePubMedGoogle Scholar
- Cho NH, Kim S, Kwon DJ, Kim HA: The prevalence of hallux valgus and its association with foot pain and function in a rural Korean community. J Bone Joint Surg Br. 2009, 91: 494-498.View ArticlePubMedGoogle Scholar
- Roddy E, Zhang W, Doherty M: Prevalence and associations of hallux valgus in a primary care population. Arthritis Rheum. 2008, 59: 857-862. 10.1002/art.23709.View ArticlePubMedGoogle Scholar
- Lazarides SP, Hildreth A, Prassanna V, Talkhani I: Association amongst angular deformities in hallux valgus and impact of the deformity in health-related quality of life. Foot Ankle Surg. 2005, 11: 193-196. 10.1016/j.fas.2005.06.005.View ArticleGoogle Scholar
- Thordarson DB, Ebramzadeh E, Rudicel SA, Baxter A: Age-adjusted baseline data for women with hallux valgus undergoing corrective surgery. J Bone Joint Surg Am. 2005, 87A: 66-75. 10.2106/JBJS.B.00288.View ArticleGoogle Scholar
- Saro C, Jensen I, Lindgren U, Fellander-Tsai L: Quality-of-life outcome after hallux valgus surgery. Qual Life Res. 2007, 16: 731-738. 10.1007/s11136-007-9192-6.View ArticlePubMedGoogle Scholar
- Abhishek A, Roddy E, Zhang W, Doherty M: Are hallux valgus and big toe pain associated with impaired quality of life? A cross-sectional study. Osteoarthritis Cartilage. 2010, 18: 923-926. 10.1016/j.joca.2010.03.011.View ArticlePubMedGoogle Scholar
- Menz HB, Lord SR: Gait instability in older people with hallux valgus. Foot Ankle Int. 2005, 26: 483-489.PubMedGoogle Scholar
- Menz HB, Morris ME, Lord SR: Foot and ankle characteristics associated with impaired balance and functional ability in older people. J Gerontol A Biol Sci Med Sci. 2005, 60A: 1546-1552.View ArticleGoogle Scholar
- Menz HB, Morris ME, Lord SR: Foot and ankle risk factors for falls in older people: a prospective study. J Gerontol A Biol Sci Med Sci. 2006, 61A: M866-870.View ArticleGoogle Scholar
- Mickle KJ, Munro BJ, Lord SR, Menz HB, Steele JR: ISB Clinical Biomechanics Award 2009. Toe weakness and deformity increase the risk of falls in older people. Clin Biomech. 2009, 24: 787-791. 10.1016/j.clinbiomech.2009.08.011.View ArticleGoogle Scholar
- Menz HB, Gilheany MF, Landorf KB: Foot and ankle surgery in Australia: a descriptive analysis of the Medicare Benefits Schedule database, 1997-2006. J Foot Ankle Res. 2008, 1: 10-10.1186/1757-1146-1-10.View ArticlePubMedPubMed CentralGoogle Scholar
- Saro C, Bengtsson A-S, Lindgren U, Adami J, Blomqvist P, Fellander-Tsai L: Surgical treatment of hallux valgus and forefoot deformities in Sweden: A population-based study. Foot Ankle Int. 2008, 29: 298-304. 10.3113/FAI.2008.0298.View ArticlePubMedGoogle Scholar
- Hardy RH, Clapham JCR: Observations on hallux valgus. Based on a controlled series. J Bone Joint Surg. 1951, 33B: 376-391.Google Scholar
- Piggott H: The natural history of hallux valgus in adolescence and early adult life. J Bone Joint Surg. 1960, 42B: 749-760.Google Scholar
- Kilmartin TE, Barrington RL, Wallace WA: Metatarsus primus varus: a statistical study. J Bone Joint Surg Br. 1991, 73B: 937-940.Google Scholar
- Panchbhavi VK, Trevino SG: Evaluation of hallux valgus surgery using computer-assisted radiographic measurements and two direct forefoot parameters. Foot Ankle Surg. 2004, 10: 59-63. 10.1016/j.fas.2004.02.001.View ArticleGoogle Scholar
- Garrow AP, Papageorgiou A, Silman AJ, Thomas E, Jayson MI, Macfarlane GJ: The grading of hallux valgus. The Manchester Scale. J Am Podiatr Med Assoc. 2001, 91: 74-78.View ArticlePubMedGoogle Scholar
- Roddy E, Zhang W, Doherty M: Validation of a self-report instrument for assessment of hallux valgus. Osteoarthritis Cartilage. 2007, 15: 1008-1012. 10.1016/j.joca.2007.02.016.View ArticlePubMedGoogle Scholar
- Menz HB, Tiedemann A, Kwan MMS, Latt MD, Lord SR: Reliability of clinical tests of foot and ankle characteristics in older people. J Am Podiatr Med Assoc. 2003, 93: 380-387.View ArticlePubMedGoogle Scholar
- Menz HB, Munteanu SE: Radiographic validation of the Manchester scale for the classification of hallux valgus deformity. Rheumatology. 2005, 44: 1061-1066. 10.1093/rheumatology/keh687.View ArticlePubMedGoogle Scholar
- Spink MJ, Menz HB, Lord SR: Efficacy of a multifaceted podiatry intervention to improve balance and prevent falls in older people: study protocol for a randomised trial. BMC Geriatr. 2008, 8: 30-10.1186/1471-2318-8-30.View ArticlePubMedPubMed CentralGoogle Scholar
- Spink MJ, Fotoohabadi MR, Menz HB: Foot and ankle strength assessment using hand-held dynamometry: reliability and age-related differences. Gerontology. 2009,Google Scholar
- Cohen J: A coefficient of agreement for nominal scales. Educat Psychol Meas. 1960, 20: 37-46. 10.1177/001316446002000104.View ArticleGoogle Scholar
- Fleiss JL: Measuring nominal scale agreement among many raters. Psychol Bull. 1971, 76: 378-382. 10.1037/h0031619.View ArticleGoogle Scholar
- Landis JR, Koch GG: The measurement of observer agreement for categorical data. Biometrics. 1977, 33: 159-174. 10.2307/2529310.View ArticlePubMedGoogle Scholar
- Kaz AJ, Coughlin MJ: Crossover second toe: demographics, etiology, and radiographic assessment. Foot Ankle Int. 2007, 28: 1223-1237. 10.3113/FAI.2007.1223.View ArticlePubMedGoogle Scholar
- Kilmartin TE, O'Kane C: Correction of valgus second toe by closing wedge osteotomy of the proximal phalanx. Foot Ankle Int. 2007, 28: 1260-1264. 10.3113/FAI.2007.1260.View ArticlePubMedGoogle Scholar
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2474/11/215/prepub