FRAX® tool, the WHO algorithm to predict osteoporotic fractures: the first analysis of its discriminative and predictive ability in the Spanish FRIDEX cohort

The major manifestation or clinical consequence of osteoporosis is the appearance of osteoporotic fracture or fragility fracture[1]. It is well known that osteoporotic fractures involve a higher incidence of new fractures and lead to disability[2]. Hip fractures and those of the vertebrae with clinical manifestations are especially important since they carry an increase in mortality[3, 4]. There is currently wide consensus regarding the need to develop strategies for the prevention of fractures and in the last years it has been recommended that the decision and the threshold of intervention be based on clinical assessment of risk of fragility fracture[5–8] and not only on the values of BMD and the relative risk as in the meta-analysis by Marshall D et al.[9].

Multiple epidemiological studies have described different clinical risk factors of osteoporotic fracture (CRFs) and which are been associated with an increased risk of developing osteoporosis and/or fragility fractures. Nonetheless, not all have determined a strong association, and the presence of these CRFs has not been uniform in the different studies and systematic reviews[10–14].

Most of the most powerful CRFs are concordant in different populations and, in general, similar for different fractures. Fractures related to falls have additional risk factors such as the number of falls, scarce physical activity and others such as the use of a walking stick, the need for help to get up from a chair, etc.). The CRFs associated with lifestyle such as smoking, alcohol intake or caffeine, low calcium consumption and scarce physical exercise have shown greater variability and lesser uniformity among the different studies[6, 7]. Finally, the influence of some risk factors on the risk of fragility fracture has been demonstrated in different meta-analyses and systematic reviews[15–20]. As previously commented, since more than 15 years ago there has been evidence that BMD below the standard values is one of the important risk factors for fragility fracture[21, 22]. More recently, however, other CRFs with as great or greater specific weight in the determination of risk of fragility fracture have been reported[11–13, 22].

It is well known that there is an important variation in the relative risk of hip fracture in both men and women at an international level. The WHO itself has performed numerous investigations on this difference. In one of the latest studies this difference was defined as a standardised rate at 10 years, being, in the most extreme cases, 15-fold greater between countries such as Norway and Chile[23].

The studies performed by the Bone and Mineral Research Program, in Garvan Institute of Medical Research show that the combination of BMD and non-invasive clinical risk factors in a nomogram could be useful for identifying high-risk individuals for intervention to reduce the risk of hip fracture[24]. With the objective to make a purpose of when were the better moment and the patient who better benefits of new drugs available for the prevention of osteoporotic, World Health Organization Collaborating Centre for Metabolic Bone Diseases, University of Sheffield, UK developed the FRAX tool. Both are useful tools to estimate absolute risk of fracture for clinical practice but both have limitations: They discriminative ability was only moderate in older women (mean 74 years old) which may limit their clinical utility[25].

Both Garvan and FRAX are widely available tools:http://www.garvan.org.au/bone-fracture-risk/ andhttp://www.shef.ac.uk/FRAX/ but both models still need to be validated in different populations before they can be generalized to other populations and further studies will be needed to validate their contribution in selecting patients who will achieve fracture risk reduction with anti-osteoporosis therapy. With the current available algorithms, a possible clinical application may be to use FRAX as the primary model and to consider using Garvan in patients with recurrent fractures and falls[25].

Since the technical reports of 1994[26] and their review in 2001 few changes have been made with respect to the WHO recommendations on the management of osteoporosis. In 2007, the WHO published a new tool for the evaluation of absolute risk of fragility fracture: the FRAX tool[27–30]. This tool was developed by WHO to evaluate fragility fracture risk for a 10 year period in patients for many countries[31–33].

The extension of a method for calculating the risk (probability) of fractures using the FRAX tool is foreseeable in Spain similar to what is occurring in other countries since its publication[34, 35]. But before its clinical use its necessary to validate the calculator in a local cohort[29, 30].

Among the person completing 10 years since their inclusion in the cohort, 1,308 could be contacted for this study (Figure 1). About 69 (5.3%) patients died (43.4%). Thirty nine have been detected by searching the telephone number and detect the death. In the other 30 cases were detected through contact with family and reported only 2 cases of fracture between baseline and the date of death. A total of 770 women fulfilled the inclusion criteria and provided informed consent to participate.

During the 10 years of study 65 women presented a total of 82 major osteoporotic fractures which could be contrasted: 17 women with 18 hip fractures, 10 with 18 proximal humeral fractures, 25 with 30 forearm fractures, and 14 with 16 vertebral fractures. All the fractures were caused by low intensity impact according to the classical definition of fragility fracture[26].

The values of the different AUC-ROC for major and hip fracture calculated in the cohort of Spanish women are shown in Table 4. That is, of BMD by DXA with the T-score of the femoral neck (FN) and with the T-score of spine L1-L4 and the FRAX tool in three ways: without BMD, with the FN T-score and with spine L1-L4 T-score. The best result was for FRAX tool for hip fracture without the T-score (0.888). In all cases the results presented significant differences with the reference (0.50) except for BMD with spine L1-L4 T-score (p=0.067). Figures 2 and3 are graphs of the AUC-ROC of the FRAX tool for major fracture and hip fracture. A determination of the AUC-ROC specifically for vertebral fracture was performed, being 0.752 (CI 95%; 0.643-0.861) for the FRAX tool without BMD, 0.815 (CI 95%; 0.725-0.905) with the FN T-score and 0.710 (CI 95%; 0.575-0.844) with L1-L4 T-score, without significant differences among them (p=0.157) (graph not shown). We compare AUC of the ROC curve of FRAX tool for major and hip fracture with a simple model including only age. The AUC in a model that includes only age was 0.668 for major fracture and 0.882 for hip fracture with no significant differences with the results of FRAX tool (p=0.565 and p=0.976 respectively).

		AUC ROC	CI 95%	p - value
AUC ROC 10-year prediction of MAJOR FRACTURE	BMD with FN T-score	0.661	(0.583-0.739)	p<0.001
	BMD with L1-L4 T-score	0.638	(0.565-0.711)	p<0.001
	FRAX® tool without BMD	0.693	(0.622-0.763)	p<0.001
	FRAX® tool with FN T-score	0.716	(0.646-0.786)	p<0.001
	FRAX® tool with spine L1-L4 T-score	0.712	(0.644-0.780)	p<0.001
AUC ROC 10-year prediction of HIP FRACTURE	BMD with FN T-score	0.779	(0.631-0.929)	p<0.001
	BMD with L1-L4 T-score	0.630	(0.487-0.773)	(ns)
	FRAX® tool without BMD	0.888	(0.824-0.952)	p<0.001
	FRAX® tool with FN T-score	0.849	(0.737-0.962)	p<0.001
	FRAX® tool with spine L1-L4 T-score	0.767	(0.658-0.876)	p<0.001

AUC ROC: Area Under Curve of Receiver Operating Characteristics; BMD: Bone Mineral Density; FN: Femoral Neck; L1-L4: Lumbar Spine; CI: Confidence Interval; ns: Not significant.

The ObsFx/ExpFx ratio was 2.4 (CI 95%; 1.9 - 3.1) for major fracture and 2.8 (CI 95%; 1.7 - 4.6) for hip fracture (Table 5) with the FRAX tool without BMD and 2.2 (CI 95%; 1.7 - 2.8) and 2.3 (CI 95%; 1.4 - 3.8), respectively with femoral neck T-score. Expressed in percentages, the FRAX tool without BMD predicts 41.1% of the cases of women with major fracture in 10 years and 46% on adding the algorithm of the T-score of the femoral neck, with these values being 35.5% and 42.8% for hip fractures, respectively.

With respect to the analysis of the sample of the FRIDEX cohort we performed a goodness-of-fit test which stratifies the results in quintiles of risk associated with quintiles of results of fracture.

Figure 4 shows the Hosmer-Lemeshow test for major fracture, with the cases of the sample distributed into quintiles and the line of regression for the results of the FRAX tool without BMD and with the FN T-score. The lower part of the figure represents the same results after calibration (simulation) by the number of times that the ObsFx is greater than the ExpFx (Table 5). Figure 5 shows the results for hip fracture in the same way.

According to the comparative analysis of the baseline characteristics between the participants and the non participants for any reason we found that the non participants did not differ from the participants except in that they were one year older and more patients were on glucocorticoids. Thus, the participants in the study did not present worst conditions of the cohort.

Self-reported generally even structured interview have a significant correlation with those in the medical record. In any case always been found documented as explained. In all cases of fracture the medical records of the patients were reviewed and, when necessary, we requested a medical report for its validation. All cases of fracture that could not be verified or those arising from a motor vehicle accident or major trauma were excluded from analysis, fractures in the history of the subjects under study. A potential limitation of self-reported fractures is in vertebral fractures. In our study the total self-reported fractures were 16% higher than they were registered and so were excluded from the final analysis. It can be an advantage for risk predictions proposed by FRAX.

The present study is centered on the discriminatory and predictive capacity of the FRAX. Analysis of the AUC-ROC was used to analyse the discriminatory capacity of this tool. As shown in (Table 4) the results of the FRAX without DXA values were greater than the AUC-ROC of BMD with values of the T-score of the femoral neck. Thus, these results demonstrate that the FRAX without the determination of BMD presents a discriminatory capacity not inferior to and even somewhat better than the DXA, according to the AUC-ROC. Analysis of the BMD with the DXA technique for the axial skeleton has traditionally been considered as the best predictive test known to determine fragility fractures[9, 26, 36] with the strategy of intervention for their prevention in medical practice having been based on this test in Spain[35] and in the remainder of the international scientific community until the appearance of the importance of other risk factors for fracture[27–33].

On analysing the role of the determination of BMD of L1-L4 in the different tests, it was found that the discriminatory capacity for major fracture using the AUC- ROC was lower than that of the determination of BMD with the T-score of the femoral neck, although statistical significance was maintained (Table 4). This inferiority was maintained for hip fracture but with no significant differences since the confidence interval integrates the value 0.50 which is the value of statistical significance for this test. Part of the debate on the possible weaknesses of the FRAX has been centered on the lack of the BMD values of the lumbar spine in its algorithm. This criticism is based on the traditional consideration that the BMD of each area presents the best predictive capacity for fractures in the same area, especially for the vertebrae and the hip[36] and, thus, it has been argued that the prediction of vertebral fractures could be improved. The discriminatory capacity measured with the AUC-ROC worsened with the incorporation of the L1-L4 T-score in the algorithm of the FRAX for major or hip fractures (Table 4). This result is congruent, but on introducing the L1-L4 T-score value in the FRAX (as a simulation) to analyse what would happen with vertebral fracture, the result of the AUC-ROC for vertebral fracture worsened slightly with respect to that obtained with the FN T-score, although without significant differences. Thus, on introducing the values of the L1-L4 T-score in the FRAX in this study the result did not provide an improvement in the discrimination of vertebral fractures measured with the AUC-ROC. Although it has described that a correction can adapt the lumbar spine BMD and improve the prediction for major and vertebral fractures of FRAX[31] in our study by incorporating the lumbar spine BMD did not improve the discriminative ability of FRAX measured by AUC with femoral neck BMD neither for major or vertebral fracture (data not showed).

The adjusted predictive capacity of the FRAX analysed using the ObsFx/ExpFx ratio was far from the 1 value which would be the desired result in the case of good adjustment of the predictive capacity of the FRAX in our country. In our cohort this ratio was of 2.4 for major fracture and 2.8 for hip fracture. These values improved minimally on the introduction of the T-score of the femoral neck in the algorithm (2.2 and 2.3 respectively). Indeed, the FRAX predicted the risk of major fracture in 41.1% of the women and 35.5% for hip fracture without BMD, with these values improving only slightly with 46% and 42.8%, respectively on performing the BMD with DXA.

These data seem to coincide with the analysis recently carried out in two cohorts of French women with a similar overall discriminatory value for fracture and low overall sensitivity (48-50% for FRAX predictions) and better than BMD alone[33, 37]. In Spain our group previously demonstrated that the FRAX has good capacity to detect densitometric osteoporosis but also with imbalance in the predictive capacity[38–40]. Nonetheless, a two recent studies in Spain had shown similar results to ours for major fractures with an ObsFx/ExpFx ratio of 3.1 (CI 95%: 2.8-3.5) and 0.8 (CI 95%: 0.7-1.1) for hip fracture[41]. Although the initial formation of the two cohorts followed very similar schemes, the method of follow up in our study was notably different. In the present study we only analysed fragility fractures reported by the women, which could be contrasted with electronic record or clinical reports. In the second study the results of ratio ObsFx/ExpFx were 0.66 and 1.10 for major and hip fracture respectively[34]. The most important methodological differences were that the study was carry out for a three years period, the authors do not included vertebral fractures[34].

The ROC curve has several problems. For analysis of sensitivity and specificity we have not a gold standard of FRAX for Spanish population. Moreover, ROC needs a gold standard of illness (fracture) and we do not have because of the electronic records are not completely reliable and we needed to make a double check (self-reported validate against records). On the other hand, the area under the ROC curve is important, since it measures the discrimination power of the model. Nevertheless, tests of discrimination alone are not sufficient for model evaluation, since they do not indicate whether calibration is also good[34, 35, 42].

In our study, on application of the Hosmer-Lemeshow test a good correlation was observed between the different quintiles of risk in all the simulation (Figures 4,5) but with a line which groups the results of the regression deviated from the reference toward the values observed. This circumstance led us to carry out a calibration multiplying each of the values resulting from the prediction made by the FRAX by a constant based on the ObsFx/ExpFx ratio for major fracture and for hip fracture. As shown in the lower part (calibration) of Figures 4 and5, on multiplication of the results of the FRAX by the ObsFx/ExpFx ratios, the results with their CI 95% adjust perfectly to the diagonal of reference in the Hosmer-Lemeshow test.

The FRAX tool can therefore be considered to present with a poor discriminatory capacity for women to have major osteoporotic fractures within 10 years, with this capacity being good for hip fractures without the need of determining the BMD, although this improves somewhat with its determination. The FRAX tool shows a scarce predictive capacity of the risk of fracture and predicts less than 50% of those which occur. The reason for this underdiagnosis may be because the Spanish cohort introduced as the reference in the FRAX tool is not representative of the current female population since these women present significantly more fractures than those actually predicted by the FRAX tool.

We have excluded from the analysis of the cohort of women receiving active treatment for the bone at baseline of the study because of the FRAX has so defined, but we have not been excluded women who received treatment during the 10-year period. This can be a potential confounding factor, however exclude women would mean removing the greatest potential for fracture, but keep going who have received treatment can be reduced the all risk of new fractures observed. Other potential confounding factor can be the Calcium/Vit D supplement intake because we have not excluded at baseline or during the study period. There is important discussion in the literature about the role of these supplements in reducing the risk of fracture, except in a subgroup of patients taking bone active drugs for the potential hypocalcaemia or in patients admitted to nursing homes. These patients are not included in this study. Moreover there is no significant difference between Calcium/Vit D supplement intake between participants and no participants.

New epidemiological studies are needed in our country to compare these results on major and other fragility fractures which, although not severe, also affect the quality of life[43]. However, together with other authors in our country[6, 10, 34, 38–41] we believe that there are sufficient data to promote the habit of investigating the risk factors of fragility fracture among Spanish physicians, especially in primary care, to determine the absolute risk and be able to propose changes in lifestyle in persons with a high risk as well as evaluate which patients should be referred for determination of the BMD by DXA[38]. In our opinion, the current state of the FRAX needs some adjustments such as those proposed in this study. Something similar to this need for adaptation and adjustments happened in Spain with the application of the first Framingham-type cardiovascular risk scales which required adaptations such as the REGICOR scale and others in our country[44–46].

We know that the promoters of the FRAX are committed to the adaptation of the tool to the different countries with the publication of new studies such as what has been done up to now. We also consider that with improvements this may be a very useful tool especially in the first level of care and this has been demonstrated by the important extension in its use worldwide[28, 35].

In summary, FRAX without BMD demonstrates a poor discriminative capacity for major fractures and a good discriminative capacity for hip fractures with the AUC ROC for Spanish women but its predictive capacity does not adjust well with the current algorithm leading to underdiagnosis for major fracture and hip fractures. On introducing the values of the L1-L4 T-score in the FRAX tool, the result did not provide an improvement in the discrimination of vertebral fractures measured with the AUC-ROC. Simple models based on age or BMD alone predicted 10-year risk of major and hip osteoporotic fractures, as well as more complex FRAX models.

We advise our Spanish colleagues to use the FRAX tool in clinical practice but weighing the resulting value of each individual case of the FRAX without BMD by a calibration value to obtain an absolute risk value of major o hip fracture at 10 years. New studies may allow a single value which is easier to remember in clinical practice. The result obtained will be more adjusted to the reality of the risk of fragility fracture in our country according to the results found in the present and other studies[34, 38, 41].