The design of a valid and reliable questionnaire to measure osteoporosis knowledge in women: the Osteoporosis Knowledge Assessment Tool (OKAT)
© Winzenberg et al; licensee BioMed Central Ltd. 2003
Received: 09 April 2003
Accepted: 24 July 2003
Published: 24 July 2003
Osteoporosis knowledge is an important contributor to improving exercise and calcium intake behaviour. However, there are few validated instruments for measuring osteoporosis knowledge levels. The aim of this study was to design a valid and reliable instrument to measure osteoporosis knowledge in Australian women.
A 20 item instrument with true, false and don't know responses was drafted, based on the Osteoporosis Australia Osteoporosis Prevention and Self-management course and the information leaflet "Understanding Osteoporosis". The scoring range was 1 to 20. This was administered to a 467 randomly-selected, healthy women aged 25–44 years. Questionnaire performance was assessed by Flesch reading ease, index of difficulty, Ferguson's sigma, inter-item and item-total correlations, Cronbach's alpha and principal component factor analysis.
Flesch reading ease was higher than desirable at 45, but this was due to the use of the word osteoporosis in many items. Of the individual items 17 had an index of difficulty less than 0.75. The questionnaire had a Ferguson's sigma of 0.96, a Cronbach's alpha of 0.70 and factor analysis consistent with only one factor (osteoporosis knowledge) being measured. Levels of osteoporosis knowledge were low with a mean score of 8.8 out of 20 which suggests the OKAT may be sensitive to change.
The OKAT for measuring osteoporosis knowledge has good psychometric properties in Australian 25–44 year old females. While it should be applicable to other Caucasian populations, this will require confirmation by further research.
Osteoporosis is a major and growing public health problem in both sexes but particularly in women [1, 2]. It is responsible for approximately 75,000 fracture events each year in Australia in those over 60 years of age at a cost to the community of $774 M . It is estimated that the proportion of women with osteoporosis increases from 15% in those aged 60 to 64 years up to 71% in those over 80 years of age. In the United States, 10 million people already have osteoporosis and 18 million more have low bone mass . Physical activity and adequate calcium intake are both important for the prevention of osteoporosis . There is evidence suggesting that osteoporosis knowledge is one contributor to osteoporosis preventive behaviour, though this is not a clear-cut relationship. Cross-sectional studies have varied in whether they have found an association between levels of osteoporosis knowledge and osteoporosis preventive behaviours [5–9]. Prospective studies similarly have been conflicting with some studies demonstrating increases in osteoporosis knowledge and concurrent improvements in osteoporosis preventive behaviours [10–12] but others demonstrating changes in knowledge but not behaviour [13, 14]. The limited research examining the ways in which osteoporosis knowledge might affect behaviour have demonstrated a relationship. In one study, causal analysis was used to demonstrate that osteoporosis knowledge was an important contributor to improving exercise and calcium intake behaviour . Another study utilizing the precaution adoption process model found that women who were in the never-engaged stage of change (ie unaware of the health problem and of the precaution recommended to reduce the risk of experiencing the problem) had the lowest levels of osteoporosis knowledge and that knowledge levels tended to be higher in women at higher stages of change .
Few studies have reported levels of osteoporosis knowledge in random, population-based samples [13, 17–19]. None of these have utilized validated instruments to measure osteoporosis knowledge and information available about the psychometric properties of the tools used was limited. None have specifically examined knowledge in women in the 25–44 year age range. Therefore, standardized osteoporosis knowledge levels in the general population and specifically in the 25–44 year age range remain unclear.
Measurement of osteoporosis knowledge is itself problematic and this may be part of the reason for the variation in the results of the studies described previously. There are three partially or fully validated instruments in the literature to measure osteoporosis knowledge [20–22]. We required an instrument in which all items were relevant to the Australian population. For example, two items in the Facts on Osteoporosis Quiz  referred to comparisons to African-American women, which were not relevant to populations outside the USA. The planned use of the instrument was to study women who had reached peak bone mass, but were premenopausal, so the instrument needed to have been fully validated in the 25–44 year age range. As the instrument was to be administered as part of a large study, it had to be suitable for self-administration, and easy to code the results. None of the existing instruments satisfied all these criteria.
The aim of this study was therefore to describe the development of and psychometric properties of an instrument to measure osteoporosis knowledge for use in a population-based random sample of 25–44 year old women.
The study was carried out in Southern Tasmania, Australia as part of an ongoing study examining the effects of lifestyle factors on bone mineral density in women aged 25–44. The population of the region in 2000 was 194,389 of which, 28,839 were women aged between 25 and 44 years of age . Subjects were randomly selected in this age range from the 2000 electoral roll. Subjects were excluded if they had previously had measurement of bone density, had thyroid disease, renal failure, malignancy or rheumatoid arthritis, had a history of hysterectomy, were on hormone replacement therapy, were pregnant or planning pregnancy within 2 years of study entry, or were lactating. Ethics approval was obtained from the Royal Hobart Hospital Ethics Committee. All subjects gave written informed consent.
Osteoporosis knowledge was measured in subjects at baseline from April to November 2000. The survey instrument was based on knowledge content common to the Osteoporosis Prevention and Self-management course (OPSMC) and an information leaflet produced by Osteoporosis Australia "Understanding Osteoporosis". The OPSMC is a chronic disease self-management course developed by the Arthritis Foundation of Victoria and utilized by Osteoporosis Australia. Osteoporosis Australia is the peak body involved in community-based osteoporosis education in Australia. The OPSMC is a small group education program that aims to increase knowledge, improve confidence and awareness and self-management of osteoporosis prevention with an emphasis on promoting appropriate lifestyle change. Educational methods include lectures, discussion, brainstorming, demonstration and small group work. The information leaflet covers information on osteoporosis including its definition, natural history of bone strength, risk factors and preventive behaviours, including physical activity and calcium intake.
The items were selected from the common material in the two interventions by a consultant rheumatologist and researcher who has specialist expertise in osteoporosis. The knowledge instrument aimed to measure a broad range of osteoporosis knowledge items that would be applicable to the Australian setting and to avoid items that would be difficult to adapt to other settings in which osteoporosis demographics might be different. It is a 20 item questionnaire, with each item having true, false and don't know options (see Additional file: 1). The analysis was performed by scoring 1 for a correct response and 0 for an incorrect or don't know response. The total score could range from 0 to 20. Face validity was also assessed using a panel of 20 people from our institution. These included research nurses, research assistants and administrative staff.
Other factors measured in the protocol were height by stadiometer (Leicester height measure, Invicta Plastics Ltd, Oadby, England) and weight by a single set of calibrated scales (Heine, Dover NH USA). Body mass index was calculated (weight/ht2). Questionnaire assessment was also made of smoking history (current/former/never), number of children, family history of osteoporosis and/or fracture, as well as fracture history in the subject, education level (4 point scale: less than grade 10, up to grade 10, completed grade12, tertiary), employment status of main financial provider in the household (employed or unemployed), hours of employment of the respondent (0, less than or equal to 20 or >20 hours per week) and marital status (6 categories).
The psychometric properties of the osteoporosis knowledge questionnaire were assessed by examining:
1. The Flesch reading ease. This assesses readability based on the average number of syllables per word and the average number of words per sentence. Scores range from 0 to 100. Standard writing averages approximately 60 to 70 . The higher the score, the greater the number of people who can readily understand the document.
2. The index of difficulty. This is defined as the proportion of patients answering the item correctly and is calculated by: number of correct responses/total number of responses . An item with an index of difficulty higher than 0.75 is deemed to be poor as it is too frequently answered correctly.
3. Item discrimination . This tests how well an item discriminates between people who have a low and high osteoporosis knowledge score. For each item, a D-value is calculated by subtracting for each item the proportion of respondents answering correctly in the lowest quartile from those answering correctly in the highest quartile, aiming for a mean D-value of 50%.
4. Discriminatory power was measured by Ferguson's sigma. This has a minimum of 0 if all subjects get the same score, and a maximum of 1 if the subjects are equally divided among all possible scores as is desirable.
5. Inter-item correlation matrix. This was performed to check for negative correlations and to screen for items with consistently weak correlations with other items (r < 0.09, based on a sample size of 467, and p > 0.10).
6. Item-total correlations. The correlation of an item with the remainder of the scale with that item omitted is the item-total correlation. A correlation of <0.20 is considered poor .
7. Cronbach's alpha. This is a measure of inter-item consistency, and ranges from 0.0 to 1.0. If omitting an item increases Cronbach's alpha significantly, then excluding the item increases the homogeneity of the scale, which is desirable.
8. Factor analysis. Principal components factor analysis ascertains whether the underlying factors identified statistically within data collected by a survey instrument are consistent with the theoretical factors one was aiming to measure with the instrument. It also assesses whether the loading of individual items on the identified factors are consistent with the premise on which the survey instrument was constructed, though in the case of dichotomous items, these loadings need to be interpreted with caution . In the case of questionnaires assessing knowledge, avoiding dichotomous items is problematic. Factor analysis has been used before in this context . In this study, we would expect only one main factor (namely osteoporosis knowledge) with items loading above 0.3  on this factor.
All analyses were performed in Stata version 7 (Stata Corporation, Texas, USA), except for the Flesch reading ease which was calculated in Microsoft Word 2000(Microsoft Corporation). A p-value < 0.05 (two tailed) was regarded as statistically significant.
Characteristics of participants (n = 467).
Mean (SD) or %
Education level, %
Grade ten or less
Completed grade 12
University or other tertiary institution
Main financial provider unemployed, %
Number of children, median, (range)
Family history osteoporosis, %
Family history of fracture, %
History of fracture, %
Currently smoking, %
Ever smoked, %
Married or de facto, %
Psychometric characteristics of the OKAT
Flesch Reading Ease
The index of difficulty for most items was satisfactory (between 0.12 and 0.66). Items 1 (Osteoporosis leads to an increased risk of bone fractures.), 4 (Osteoporosis is more common in men.) and 10 (Any type of physical activity is beneficial for osteoporosis.) scored above 0.75, indicating that most subjects answered these questions correctly.
The mean D-value for the questionnaire was 44%. Ferguson's sigma for the questionnaire is 0.96 which is very high.
Psychometric Properties of the OKAT by Item
Index of Difficulty
Item Discrimination (%)
Principal factor analysis generated only one factor with an eigenvalue above 1 (Factor one – eigenvalue 2.3). Items loaded from 0.026 to 0.46 on this factor. Items 1, 4, 7, 9, 10, 11 and 18 loaded less than 0.3. Elimination of these items iteratively from the factor analysis did not alter the distribution of eigenvalues. There was still only one factor with an eigenvalue greater than one and the distribution of the loadings of the items on this factor was unchanged. Iterative elimination of the items with low loadings on factor analysis from item-total and Cronbach's alpha analyses caused changes of less than 1.5%. When items 1,4,9, 11 and 18 were eliminated there were decreases in alpha, rather than the desired increases.
The OKAT performed satisfactorily on virtually all components of the analysis.
The questionnaire had a lower than preferred Flesch reading ease. This was due to the use of the word osteoporosis in 12 of the 20 items. If this word was removed or substituted for by "thin bones" the Flesch reading ease became highly acceptable. As the word osteoporosis was the most accurate description of the disease about which knowledge was being measured and is widely recognised in the general population , its use was retained and the resulting decrease in Flesch reading ease accepted.
The questionnaire had a satisfactory index of difficulty. Item discrimination was satisfactory and Ferguson's sigma was close to the ideal value of 1.0. The three items with a high index of difficulty were retained as it was considered they related to core information about osteoporosis namely, the definition of osteoporosis, female preponderance and the physical activity requirements needed for prevention. Though factor analysis must be interpreted cautiously when analysing dichotomous variables, the fact that the analysis generated only one factor with an eigenvalue above 1 is consistent with osteoporosis knowledge being the main factor being measured by the questionnaire, and this provides some support for the construct validity of the instrument. Certainly, if more than one underlying factor had been found, we would have had cause to question construct validity.
The elimination of the items performing poorly on item-total correlation and principal factor analysis changed Cronbach's alpha by less than 1.5% and had minimal effect on the factor analysis outcome, indicating that the elimination of these items had little effect on the psychometric qualities of the questionnaire. The 20 item Cronbach's alpha of 0.69 was satisfactory, particularly given that in order to achieve high discriminatory power, a scale must include very easy items as well as very difficult items, which tends to decrease the internal consistency of the scale . The elimination of individual items that performed less well did not alter the overall psychometric properties of the questionnaire and would have reduced the breadth of knowledge covered by the questionnaire ie affected content validity. These items evaluated knowledge of the meaning of osteoporosis, its prevalence, the adverse health outcomes of osteoporosis and of the physical activity requirements for prevention. Previous studies [20, 21] have described the problem of clinically pertinent items not performing ideally under psychometric testing, but still being important for the overall context of the instrument. We dealt with this problem in a similar manner to those studies. If the items contained what we considered to be core knowledge we retained them, but only after (1) examining the psychometric properties with and without the items to ensure that we did not adversely affect the reliability of the questionnaire and (2) describing the reasons for retention and the reasons for considering removing the item.
Baseline levels of osteoporosis knowledge measured using this instrument were low, with the average score of 8.8 being 44% of the possible maximum score. Most other studies of osteoporosis knowledge levels have shown low levels, but these have been in highly selected populations or have not used validated instruments to measure osteoporosis knowledge [6, 8, 10, 12, 14, 29–32]. One study showed knowledge levels of 78% of the maximum score in 16–59 year old Norwegian women but the instrument was not validated . As the average score using our instrument was low, there is scope for the instrument to be sensitive to change, which is a property that will be valuable if using the questionnaire to assess changes in knowledge with interventions. However, sensitivity to change remains to be tested.
This study has a number of potential limitations. While the sample was randomly selected, selection bias is possible due to the moderate response rate. Indeed proportion of current smokers in the sample is lower than the Tasmanian prevalence of daily smoking in females aged 25–44 in 1998 of 29%  and the proportion of women in married or in a de facto relationship is slightly higher than the Tasmanian proportion of 64%. However, the spread of education levels and the unemployment rate approximates the overall population figures for these variables. The range of demographics covered within the sample means that the validation has occurred over a reasonably heterogeneous group and so the questionnaire appears suitable for use in women aged 25–44 across a range of demographics. Although the osteoporosis knowledge instrument is based on the OPSMC content and the osteoporosis information leaflet, it covers a broad spectrum of osteoporosis knowledge and may be suitable for use to assess the impact of other osteoporosis educational interventions or to assess osteoporosis knowledge levels in young female Caucasian populations. The OKAT would require validation in other populations and some modifications might be necessary to reflect regional variations in osteoporosis demographics.
The OKAT for measuring osteoporosis knowledge has good psychometric properties in Australian 25–44 year old females. While it should be applicable to other Caucasian populations, this will require confirmation by further research.
List of Abbreviations
Osteoporosis Knowledge Assessment Tool
Osteoporosis Prevention and Self-management Course
- Cooley H, Jones G: A population-based study of fracture incidence in southern Tasmania: lifetime fracture risk and evidence for geographic variations within the same country. Osteoporos Int. 2001, 12: 124-130. 10.1007/s001980170144.View ArticlePubMedGoogle Scholar
- Jones G, Nguyen T, Sambrook PN, Kelly PJ, Gilbert C, Eisman JA: Symptomatic fracture incidence in elderly men and women: the Dubbo Osteoporosis Epidemiology Study (DOES). Osteoporos Int. 1994, 4: 277-282.View ArticlePubMedGoogle Scholar
- Randell A, Sambrook PN, Nguyen TV, Lapsley H, Jones G, Kelly PJ, Eisman JA: Direct clinical and welfare costs of osteoporotic fractures in elderly men and women. Osteoporos Int. 1995, 5: 427-432.View ArticlePubMedGoogle Scholar
- Osteoporosis Prevention, Diagnosis and Therapy. NIH Consensus Statement 2000. 2000, 17: 1-45.
- Taggart HM, Connor SE: The relation of exercise habits to health beliefs and knowledge about osteoporosis. J Am Coll Health. 1995, 44: 127-130.View ArticlePubMedGoogle Scholar
- Kasper MJ, Peterson MG, Allegrante JP, Galsworthy TD, Gutin B: Knowledge, beliefs, and behaviors among college women concerning the prevention of osteoporosis. Arch Fam Med. 1994, 3: 696-702. 10.1001/archfami.3.8.696.View ArticlePubMedGoogle Scholar
- Terrio K, Auld GW: Osteoporosis knowledge, calcium intake, and weight-bearing physical activity in three age groups of women. J Community Health. 2002, 27: 307-320. 10.1023/A:1019840709367.View ArticlePubMedGoogle Scholar
- Wallace LS: Osteoporosis prevention in college women: application of the expanded health belief model. Am J Health Behav. 2002, 26: 163-172.View ArticlePubMedGoogle Scholar
- Satterfield T, Johnson SM, Slovic P, Neil N, Schein JR: Perceived risks and reported behaviors associated with osteoporosis and its treatment. Women Health. 2000, 31: 21-40.View ArticlePubMedGoogle Scholar
- Cook B, Noteloviz M, Rector C, Krischer J: An Osteoporosis Patient Education and Screening Program:Results and Implications. Patient Education and Counseling. 1991, 17: 135-145.View ArticleGoogle Scholar
- Brecher LS, Pomerantz SC, Snyder BA, Janora DM, Klotzbach-Shimomura KM, Cavalieri TA: Osteoporosis prevention project: a model multidisciplinary educational intervention. J Am Osteopath Assoc. 2002, 102: 327-335.PubMedGoogle Scholar
- Curry LC, Hogstel MO, Davis GC, Frable PJ: Population-based osteoporosis education for older women. Public Health Nurs. 2002, 19: 460-469. 10.1046/j.1525-1446.2002.19607.x.View ArticlePubMedGoogle Scholar
- Blalock SJ, Currey SS, DeVellis RF, DeVellis BM, Giorgino KB, Anderson JJ, Dooley MA, Gold DT: Effects of educational materials concerning osteoporosis on women's knowledge, beliefs, and behavior. Am J Health Promot. 2000, 14: 161-169.View ArticlePubMedGoogle Scholar
- Sedlak CA, Doheny MO, Jones SL: Osteoporosis education programs: changing knowledge and behaviors. Public Health Nurs. 2000, 17: 398-402. 10.1046/j.1525-1446.2000.00398.x.View ArticlePubMedGoogle Scholar
- Piaseu N, Schepp K, Belza B: Causal analysis of exercise and calcium intake behaviors for osteoporosis prevention among young women in Thailand. Health Care Women Int. 2002, 23: 364-376. 10.1080/0739933029008937.View ArticlePubMedGoogle Scholar
- Blalock SJ, DeVellis RF, Giorgino KB, DeVellis BM, Gold DT, Dooley MA, Anderson JJ, Smith SL: Osteoporosis prevention in premenopausal women: using a stage model approach to examine the predictors of behavior. Health Psychol. 1996, 15: 84-93. 10.1037//0278-618.104.22.168.View ArticlePubMedGoogle Scholar
- Phillipov G, Phillips PJ, Leach G, Taylor AW: Public perceptions and self-reported prevalence of osteoporosis in South Australia. Osteoporos Int. 1998, 8: 552-556. 10.1007/s001980050098.View ArticlePubMedGoogle Scholar
- Magnus JH, Joakimsen RM, Berntsen GK, Tollan A, Soogaard AJ: What do Norwegian women and men know about osteoporosis?. Osteoporos Int. 1996, 6: 32-36.View ArticlePubMedGoogle Scholar
- Waller J, Eriksson O, Foldevi M, Kronhed AC, Larsson L, Lofman O, Toss G, Moller M: Knowledge of osteoporosis in a Swedish municipality--a prospective study. Prev Med. 2002, 34: 485-491. 10.1006/pmed.2002.1007.View ArticlePubMedGoogle Scholar
- Ailinger RL, Harper DC, Lasus HA: Bone up on osteoporosis. Development of the Facts on Osteoporosis Quiz. Orthop Nurs. 1998, 17: 66-73.View ArticlePubMedGoogle Scholar
- Pande KC, de Takats D, Kanis JA, Edwards V, Slade P, McCloskey EV: Development of a questionnaire (OPQ) to assess patient's knowledge about osteoporosis. Maturitas. 2000, 37: 75-81. 10.1016/S0378-5122(00)00165-1.View ArticlePubMedGoogle Scholar
- Redman BK: Measurement tools in patient education. 1998, New York, Springer Pub. Co.Google Scholar
- Population by Age and Sex, Tasmania. 2002, Australian Bureau of Statistics
- Flesch R: A new readability yardstick. J Appl Psychol. 1948, 221-233.Google Scholar
- Anastasi A: Psychological Testing, sixth edition. 1988, New York, MacMillanGoogle Scholar
- Streiner EL, Norma GR: Selecting the Items. Health Measurement Scales: A Practical Guide to their development and Use. 1989, Oxford, Oxford University Press, 39-52.Google Scholar
- Comrey Andrew L.: Common methodological problems in factor analytic studies. Journal of Consulting & Clinical Psychology. 1978, 46: 648-659. 10.1037//0022-006X.46.4.648.View ArticleGoogle Scholar
- Tabachnick Barbara G., Fidell Linda S.: Using multivariate statistics. 1989, New York, Harper & Row, xxii, 746 p.-2ndGoogle Scholar
- Kasper MJ, Peterson MG, Allegrante JP: The need for comprehensive educational osteoporosis prevention programs for young women: results from a second osteoporosis prevention survey. Arthritis Rheum. 2001, 45: 28-34. 10.1002/1529-0131(200102)45:1<28::AID-ANR80>3.3.CO;2-7.View ArticlePubMedGoogle Scholar
- Ailinger RL, Emerson J: Women's knowledge of osteoporosis. Appl Nurs Res. 1998, 11: 111-114.View ArticlePubMedGoogle Scholar
- Piaseu N, Belza B, Mitchell P: Testing the effectiveness of an osteoporosis educational program for nursing students in Thailand. Arthritis Rheum. 2001, 45: 246-251. 10.1002/1529-0131(200106)45:3<246::AID-ART256>3.3.CO;2-P.View ArticlePubMedGoogle Scholar
- Ribeiro V, Blakeley J, Laryea M: Women's knowledge and practices regarding the prevention and treatment of osteoporosis. Health Care Women Int. 2000, 21: 347-353. 10.1080/073993300245195.View ArticlePubMedGoogle Scholar
- Health and Wellbeing in Tasmania. 1999, Hobart, Department of Health and Human Services
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2474/4/17/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article: verbatim copying and redistribution of this article are permitted in all media for any purpose, provided this notice is preserved along with the article's original URL.