Open Access
Open Peer Review

This article has Open Peer Review reports available.

How does Open Peer Review work?

Prediction of pain outcomes in a randomized controlled trial of dose–response of spinal manipulation for the care of chronic low back pain

  • Darcy Vavrek1,
  • Mitchell Haas1Email author,
  • Moni Blazej Neradilek2 and
  • Nayak Polissar2
BMC Musculoskeletal Disorders201516:205

https://doi.org/10.1186/s12891-015-0632-0

Received: 28 April 2015

Accepted: 14 July 2015

Published: 19 August 2015

Abstract

Background

No previous studies have created and validated prediction models for outcomes in patients receiving spinal manipulation for care of chronic low back pain (cLBP). We therefore conducted a secondary analysis alongside a dose-response, randomized controlled trial of spinal manipulation.

Methods

We investigated dose, pain and disability, sociodemographics, general health, psychosocial measures, and objective exam findings as potential predictors of pain outcomes utilizing 400 participants from a randomized controlled trial. Participants received 18 sessions of treatment over 6-weeks and were followed for a year. Spinal manipulation was performed by a chiropractor at 0, 6, 12, or 18 visits (dose), with a light-massage control at all remaining visits. Pain intensity was evaluated with the modified von Korff pain scale (0–100). Predictor variables evaluated came from several domains: condition-specific pain and disability, sociodemographics, general health status, psychosocial, and objective physical measures. Three-quarters of cases (training-set) were used to develop 4 longitudinal models with forward selection to predict individual “responders” (≥50 % improvement from baseline) and future pain intensity using either pretreatment characteristics or post-treatment variables collected shortly after completion of care. The internal validity of the predictor models were then evaluated on the remaining 25 % of cases (test-set) using area under the receiver operating curve (AUC), R2, and root mean squared error (RMSE).

Results

The pretreatment responder model performed no better than chance in identifying participants who became responders (AUC = 0.479). Similarly, the pretreatment pain intensity model predicted future pain intensity poorly with low proportion of variance explained (R2 = .065). The post-treatment predictor models performed better with AUC = 0.665 for the responder model and R2 = 0.261 for the future pain model. Post-treatment pain alone actually predicted future pain better than the full post-treatment predictor model (R2 = 0.350). The prediction errors (RMSE) were large (19.4 and 17.5 for the pre- and post-treatment predictor models, respectively).

Conclusions

Internal validation of prediction models showed that participant characteristics preceding the start of care were poor predictors of at least 50 % improvement and the individual’s future pain intensity. Pain collected shortly after completion of 6 weeks of study intervention predicted future pain the best.

Keywords

Chronic low back pain Prediction model Spinal manipulation Chiropractic Dose–response Randomized controlled trial

Background

The most common cause of disability is low back pain with an estimated 1099 years of life lost to disability each year per 100,000 people, worldwide, in 2010 [1]. The prevalence of chronic low back pain (cLBP) is approximately 10 % [1, 2]. An effective approach to treating low back pain can include spinal manipulative therapy (SMT) [35]. Advantageously, treatment of cLBP with spinal manipulative therapy does not appear to increase the cost of treatment plus lost productivity [6]. The question remains though, about what kind of patient has a greater chance of benefit with efficacious conservative therapies such as SMT [79], mechanical lumbar traction [10], and a stabilization exercise program [11]. Our study is a step in this direction and to our knowledge, this is the only study to date that has sought to create prediction models of prognosis in individuals receiving a dose of SMT for the care of cLBP. This scientific inquiry is of great societal interest given today’s environment of prevention of opiate deaths in chronic pain management [1216].

There have been few studies evaluating determinants of outcomes in patients receiving SMT for the care of cLBP. Leboeuf-Yde et al., [17] looked at predictors in a cohort of chiropractic patients with persistent low back pain. They found sex, social benefit, severity of pain, duration of continuous pain at first consultation, and additional neck pain predictive of failure to recover in the short term. Most notable was being pain-free at the fourth visit was a strong predictor of recovery at 3 and 12 months. Dougherty et al. [18] modified the clinical prediction rule developed by Flynn et al. [7] so it could be tested in a randomized trial in a chronic patient population. The modified rule was not successful, a cautionary tale against using a prediction rule in individuals for which it was not specifically developed. A large practice-based observational study reported in Nyiendo et al.[8] and Haas et al. [9] found the following to be predictive of outcomes: baseline pain and disability, age, history of low back pain, duration of baseline LBP episode, pain below the knee, provider type, income, smoking, comorbidity, and chronic depression.

The aim of this secondary analysis was to build and attempt to internally validate prediction models for pain outcomes in cLBP patients treated with SMT in a randomized controlled trial. The purpose was prognosis related to a course of care, rather than development of a clinical prediction rule for selecting from among alternative interventions. Models were created separately using data from the two most natural time points for discussing prognosis with a patient in clinical practice: immediately prior to care (pre-treatment predictor models) and following completion of care (post-treatment predictor models). The two outcomes that were predicted by the models were the continuous pain score and a dichotomous responder indicator (50 % or greater improvement in pain score from the baseline). Our approach was unique for studies of SMT in that the sample was partitioned to both create and test the strength of the prediction models in one study.

Methods

Design

Data were obtained through a randomized clinical trial looking for the dose effect of spinal manipulative therapy for cLBP [3, 6]. Four hundred participants were randomized to 4 dose groups, 100 patients per group. Participants were treated 3 times per week for 6 weeks. They received 0, 1, 2, or 3 SMT sessions per week (0, 6, 12, or 18 total SMT visits) with control visits, consisting of light massage, on non-manipulation visits. All care was provided in the Portland metro area. Outcomes were collected at baseline and 6, 12, 18, 24, 39, and 52 weeks after randomization on. Randomization coincided with the first treatment visit.

Participants

All patients provided informed consent for participation in this study which was approved of by University of Western States Institutional Review Board. Patients were recruited through craigslist, mailers, and newspaper advertisements in the Portland Metro area to participate in our study. Details on subject enrollment, inclusion/exclusion criteria, design, and analysis are reported elsewhere [3]. Details on cost effectiveness and the doctor/patient interaction have also been reported [6, 19].

Patients were considered chronic if their low back pain was at least 3 months in duration and they had at least 30 days of low back pain in the last 6 weeks. A minimum pain score of 25 (0–100 scale) was required in order to prevent floor effects. Participants were excluded for contraindications to SMT, such as active cancer, spine pathology, inflammatory arthropathies, autoimmune disorders, and anti-coagulant conditions. Also excluded were those with potential confounders to pain and disability improvement; including neurodegenerative diseases, pain radiating below the knee, organic referred pain, and disability compensation [3].

Intervention

The intervention was SMT consisting of manual high velocity, low amplitude, thrust spinal manipulation [20]. The control appointments consisted of a brief light massage, shorter and lighter than what would be considered appropriate in a therapeutic massage practice [21, 22]. In addition to SMT or light massage, at each visit, all participants received 5 min of hot pack treatment to relax the back musculature and 5 min of very low intensity pulsed ultrasound (0.5 watts/cm2) to enhance credibility of care. This was deemed necessary for fidelity of care in patients receiving only or mostly a brief light massage.

Pain and disability scales

We assessed cLBP with the modified Von Korff pain and disability scales, validated by Underwood et al. [23]. The pain scale consisted of asking patients to rate their pain today, average pain over the last four weeks, and worst pain over the past 4 weeks each on a 0–10 pain scale with no pain and as bad as pain could be as the anchors. The disability scale consisted of asking patients to rate how much their low back pain interfered with their daily activities; changed their ability to take part in recreational, social, and family activities; and changed their ability to work (including house and yard work) with no interference/change and extreme interference/change as the anchors. For both pain and disability, the 3 0–10 scales were averaged and then multiplied by 10 to create the two modified Von Korff scores on 0 to 100 scales.

Outcome measures

Pain intensity was measured by the Modified Von Korff pain scale described above [23]. Pain outcomes were analyzed either as a continuous score (future pain intensity) or as a binary indicator for a responder. Choosing a dichotomous responder indicator is recommended by the NIH Task Force for Research Standards on Chronic Low Back Pain [24]. For a given follow-up visit, a responder was defined as having at least 50 % improvement relative to the baseline pain intensity [24, 25]. Fifty percent improvement in back pain is considered substantial [26]. Note that the same participant could be classified as a responder during one visit and not at another. The pretreatment predictor models were used to predict the outcomes at the 6, 12, 18, 24, 39 and 52-week follow-ups and the post-treatment predictor models were used to predict the outcomes at the 12, 18, 24, 39 and 52-week follow-ups.

Predictive measures

A more extensive discussion of the outcomes used in this study is provided in our paper on main results from our dose response study [3]. Potential pretreatment predictors were collected at baseline by questionnaire (Table 1). Demographics included age, gender, education, race, and ethnicity. Validated low back pain characteristics were the primary outcomes, pain intensity and disability [23], as well as pain unpleasantness [27]. Other common condition-specific variables we have used in the past were days with pain, days with disability, duration, and previous treatment [28, 29]. General Health status measures were comorbidity, smoking, and the validated EuroQol-5D [3032] and SF-12 [33, 34]. Psychosocial variables were the Fear Avoidance Beliefs Questionnaire [35] and previously used confidence in the success of SMT and light massage [28, 29]. Dose was determined during randomization at baseline and coded as 0, 1, 2, or 3 representing the multiple of 6 SMT sessions each study group received. Follow-up time point was used as a continuous variable measured in weeks from randomization.
Table 1

Pretreatment characteristics and univariate pain prediction modelsa

 

Baseline Mean (SD) (N = 391)

Responder model significance

Future Pain Intensity model significance

Dose (per 6 spinal manipulation visits)

 

Time (in weeks

   

Pain/Disability

   

 Pain intensity (0–100 scale)

51.6 (17.2)

 

 Functional disability (0–100)

45.3 (22.7)

 

 Pain unpleasantness (0–100)

41.4 (21.4)

 

 Days with pain (last 4 wk)

24.1 ( 5.2)

 

 Days with disability (last 4 wk)

6.8 ( 7.6)

 

 Duration (yr)

11.8 ( 9.8)

  

Sociodemographics

   

 Age (yr)

41.3 (14.1)

  

 Female, % (n)

50 % (196)

  

 Non-white or Hispanic, % (n)

15 % (58)

  

 College degree, % (n)

56 % (219)

  

 Income $40 K or less, % (n)

57 % (222)

  

General Health

   

 Comorbidities (#)

0.9 ( 1.1)

 

 Smoking, % (n)

11 % (43)

  

 SF-12 physical health componentb

43.3 ( 8.9)

 

 SF-12 mental health componentb

48.9 (10.5)

  

 EuroQol – VAS (0–100 visual analog scale)

70.9 (15.8)

 EuroQol 5D – mobility (1–3)

1.4 ( 0.5)

 

 EuroQol 5D – self-care (1–3)

1.2 ( 0.4)

 EuroQol 5D – usual activities (1–3)

1.7 ( 0.5)

 

 EuroQol 5D – pain (1–3)

2.0 ( 0.2)

 

 EuroQol 5D – anxiety/depression (1–3)

1.4 ( 0.5)

 

Psychosocial

   

 FABQ Work beliefs (0–100)

32.9 (21.8)

 

 FABQ Activity beliefs (0–100)

56.0 (20.3)

 

 Confidence in treatment success (−6 - +6)

0.2 ( 0.8)

  

Objective Physical Exam c

   

 Lumbar ROM: flexion

43.2 (16.3)

  

 Lumbar ROM: extension

15.1 (10.2)

  

 Lumbar ROM: right lateral bending

18.6 ( 9.4)

  

 Lumbar ROM: left lateral bending

19.1 ( 8.9)

  

 LBP: Flexion (0–10)

2.3 ( 2.4)

 LBP: Extension (0–10)

3.0 ( 2.4)

 

 LBP: Right lateral bending (0–10)

2.7 ( 2.4)

 LBP: Left lateral bending (0–10)

2.6 ( 2.3)

 LBP: sum for 4 lumbar ROM 0–10 pain scores

10.7 ( 7.9)

 LBP: maximum of 4 lumbar ROM pain scores

4.0 ( 2.3)

 

 LBP: right – left lateral bending

0.1 ( 1.9)

  

 LBP: |right – left lateral bending|

1.2 ( 1.5)

  

 LBP: sum for right and left lateral bending pain scores

5.3 ( 4.3)

 LBP: maximum of right and left lateral bending pain scores

3.2 ( 2.4)

 Modified Schober Test (cm)

5.7 ( 2.0)

  

 Lumbar hypomobility: L1, % (n)

54 % (209)

 

 Lumbar hypomobility: L2, % (n)

52 % (200)

  

 Lumbar hypomobility: L3, % (n)

49 % (189)

  

 Lumbar hypomobility: L4, % (n)

49 % (191)

  

 Lumbar hypomobility: L5, % (n)

64 % (248)

  

 Total hypomobile joints: L1 thru L5

2.7 ( 1.3)

  

 Pain Pressure Threshold: right L1-L2

6.1 ( 2.8)

  

 Pain Pressure Threshold: left L1-L2

6.2 ( 2.9)

  

 Pain Pressure Threshold: right L3-L4

5.9 ( 3.0)

  

 Pain Pressure Threshold: left L3-L4

6.0 ( 3.2)

  

 Pain Pressure Threshold: right L5-S1

5.8 ( 3.2)

 

 Pain Pressure Threshold: left L5-S1

5.7 ( 3.3)

 

 Pain Pressure Threshold: minimum of 6 measures

4.5 ( 2.5)

 

OR Odds ratio, r Pearson’s correlation coefficient, β regression coefficient, VAS visual analog scale, FABQ fear avoidance beliefs questionnaire, ROM range of motion, LBP low back pain

Variables with a statistically significant association with outcome, p-value < 0.05, after adjusting for dose

aLogistic and linear longitudinal regressions were adjusted for dose and were fitted using generalized estimating equations to account for correlation across time points. Only the statistically significant variables (p < .05) in this table are used as candidates for the subsequent inclusion into the relevant final multivariate prediction models

bScores are standardized to the US general population (mean = 50, SD = 10)

cROM was measured in degrees, LBP during ROM on a 0 to 10 scale for each of the 4 ROMs, and pain pressure threshold in kg. Hypomobility was identified using manual motion palpation

Objective physical exam measures included lumbar range of motion, segmental hypomobility, and pain pressure threshold. Lumbar range of motion was measured in flexion , extension, and lateral bending [3638]; reliability has been established by Keeley et al. [39]. The accompanying self-reported pain was evaluated using the validated 0 to 10 numeric pain rating scale [40]. The sum of these pain scores, the difference between right and left lateral bending pain scores, and maximum pain score were also of interest for prediction model development. A modified Schober test [41, 42] was conducted on each patient; test reliability has been established [4345]. Segmental hypomobility from L1 through L5 was determined by manual motion palpation [46, 47] and defined for this study as restriction of motion in any plane of motion. The total number of hypomobile joints were also recorded [46, 47]. Stochkendahl et al. [47] recently observed that global assessment of hypomobility has clinically acceptable reproducibility. Pain pressure thresholds (PPT) were assessed from L1 through S1 using a validated pressure algometer [48, 49].

Potential post-treatment predictors were collected several days after the end of treatment (6 weeks after randomization) and included pain intensity, disability, LBP unpleasantness, days with LBP, days with disability due to LBP, satisfaction with care [8, 19, 50, 51] and objective physical exam measures, as well as dose and time (follow-up time points listed under outcome measures).

Analysis

The data set was randomly split into two sets. The training set included all data for 75 % of the participants and the test set included all data for the remaining 25 % of the participants. The multivariate prediction models were developed on the training-set and the ability of the developed models to predict the outcomes was evaluated on the test set. This random split enabled internal validation of the developed model in the same study and addressed the potential issue of over-optimism (over-fitting) of the model in the training set.

For each of the 2 outcomes, we developed a pretreatment and a post-treatment predictor model using the variables identified above. The outcomes were modeled by logistic regression (responder outcome) and linear regression (future pain intensity outcome) using generalized estimating equations (with the AR1 correlation structure) to account for the repeated measures for the same subject [5254]. Analysis included all time points in aggregate (as repeated measures) to improve the stability and power of the estimated model and to avoid the complexity of reporting and interpreting results from separate predictor models for each time point. A sensitivity analysis (not shown) of models for individual time points showed results that were consistent with and warranting the repeated-measures analysis.

Model development

The prediction models were developed in 2 steps. First, a univariate analysis was performed on the set of all potential predictors with treatment dose forced into all models (to adjust for the primary purpose of the trial which was to evaluate the effect of dose). Second, statistically significant variables (p < .05) in the univariate analysis were then considered for inclusion into the multivariate models. The multivariate predictor models used a forward stepwise variable selection procedure with p < .05 required for entry into the model [55]. Observations with missing data were dropped.

For the responder models (binary data outcome), the odds ratio, 95 % confidence intervals, and p-values are reported. The odds ratio gives the increase in the odds of being a responder after adjusting for the other variables in the model. For the future pain intensity models (continuous data outcome), linear regression coefficients (β), 95 % confidence intervals, and p-values are reported. The coefficient β estimates the change in mean future pain intensity score expected for a unit change of the predictor variable after controlling for the other variables in the predictor model.

Model evaluation

The predictive ability of the models in the training and test sets are expressed by the area under receiver operating characteristics curve (AUC) for the binary responder models [56] and by both root mean square error (RMSE) and R2 for the continuous future pain intensity models [57]. The AUC measures the ability to predict a binary outcome on a 0 to 1 scale with AUC = 0.50 representing prediction expected by chance alone and AUC = 1 representing perfect prediction. The RMSE estimates the standard deviation of the prediction error, which is defined as the difference between the predicted and observed future pain scores. R2 is the proportion of the variation in the continuous outcome explained by the model.

Of 400 participants, the 391 who provided follow-up data were included in the analysis for the pretreatment models, while 385 with follow-up data were included in the post-treatment models. Follow-up missing data were imputed for these intention-to-treat analyses using linear interpolation and last point carried forward.

Analyses were conducted with Stata 11.0 (StataCorp, College Station, TX) and R version 3.1.0 (R Foundation for Statistical Computing, Vienna, Austria). All tests were two-sided and p < 0.05 was used to denote statistical significance. Multiple testing adjustments were not made because this was an exploratory analysis aimed at evaluating the feasibility of prediction.

Results

Pretreatment variables

Table 1 shows the descriptive statistics for the baseline variables that were considered for entry into pretreatment predictor models. On average, these patients were 41 years old, equally split in gender, and white non-Hispanic; they had moderate pain (52/100) and disability (45/100), chronic back pain of about 12 years in duration, moderate baseline health, previous care for their back problems, and confidence in success of study care [3]. Mean global lumbar range of motion was 43° flexion, 15° extension, and 19° lateral bending. Overall, reported pain with lumbar motion was mild to moderate (3/10). Pain pressure thresholds were reported to be about 6 kg/cm2 from L1 through L5, on average. Table 1 also notes the statistically significant variables in the univariate analyses (p < .05) that were considered for inclusion into the multivariate models.

There were no missing values in the variables considered in the responder multivariate model and <2 % missing values for variables considered in the multivariate model for future pain intensity. Less than <3 % of values were missing for the remaining variables. In aggregate, only 2.6 % observations had to be dropped due to missing values in variables for building the multivariate model for future pain intensity.

Pretreatment multivariate predictor models

The predictive ability of the pretreatment responder model in the training and test sets were quantified by the AUC statistic and illustrated visually by the receiver operating characteristics (ROC) curves (Table 2 and Fig. 1). While the model appeared to predict some risk of being a responder in the training set (AUC = 0.624), in the test set the model performed similarly to chance only (AUC = 0.479 vs. 0.500 for chance). Among the variables selected into the model, greater odds of 50 % improvement in pain were associated with greater dose of SMT, while poorer odds were associated with comorbidity, less tendency towards self-care, and greater pain with lumbar motion.
Table 2

Final multivariate pretreatment pain-prediction models and performance metricsa

 

Responders (N = 297/94)b

Future pain intensity (N = 289/94)b

Independent variables

OR

(95 % CI)

P-value

 

β

(95 % CI)

P-value

Dose (per 6 spinal manipulation visits)

1.27

(1.08, 1.49)

0.004

 

−1.86

(−3.35, −0.38)

0.014

Pain/Disability

       

 Pain intensity

    

4.77

(1.85, 7.70)

0.001

 Pain unpleasantness

    

3.29

(0.35, 6.24)

0.028

General Health

       

 Comorbidities

0.81

(0.67, 0.97)

0.025

    

 EuroQol – VAS

    

−2.20

(−4.00, −0.39)

0.017

 EuroQol 5D – self-care (1–3)

0.64

(0.41, 0.99)

0.044

    

Objective Physical Exam

       

 LBP: sum for 4 lumbar ROM pain scores

0.81

(0.67, 0.97)

0.024

 

2.89

(0.61, 5.16)

0.013

Performance metricsc

AUC

(95 % CI)

 

RMSE

(95 % CI)

R2

(95 % CI)

 Training set

0.624

  

17.4

 

.268

 

 Test set

0.479

(0.387, 0.575)

 

19.4

(17.0, 21.6)

.065

(−10.5, 21.9)

OR Odds ratio, PC part correlation, β regression coefficient, VAS visual analogue scale, AUC Area under the curve (receiver operating characteristic curve), RMSE root mean squared error (SD of prediction error), R 2 coefficient of determination, LBP low back pain

aVariables were selected into the regression models using forward selection among variables with p < .05 in the univariate analysis; dose was forced into the models. Independent variables were standardized except for dose (scale unit = 6 visits) and self-care (scale unit = 1 on a 1–3 scale). Lower scores were favorable for pain and self-care; higher scores for EuroQol VAS

bThe first number is the sample size for the model in the training set and the second number is the N for the test set

cChance performance is indicated by 0.5 for AUC. RMSE is the standard deviation of the error in prediction of future pain intensity evaluated on the 0 – 100 pain scale. R 2 is the proportion of the variance in pain intensity explained by the independent variables in the model. Confidence intervals for the performance metrics are given for the test set only

Fig. 1

Pretreatment model ROC curves. Receiver operating characteristic (ROC) curves for the final multivariate model for prediction of responders. The area under the curve (AUC) was 0.624 in the training set and 0.479 in the test set. Chance is shown by the diagonal line indicating AUC = 0.5

The ability to predict future pain intensity was also poor (Table 2 and Fig. 2). The model R2 was 0.268 for the training set and 0.065 for the test set. Hence, the model could account for only a trivial percentage of the variability in future pain. The prediction error (RMSE) was large for both the training and test set data (RMSE = 17.4 and 19.4 points, respectively). Predictors in the model associated with greater future pain were greater baseline pain intensity and unpleasantness, as well as greater pain with lumbar motion; less future pain was related to greater dose of SMT and greater general health evaluated with the EuroQol visual analogue scale.
Fig. 2

Pretreatment model scatterplots. Observed pain scores are plotted against predicted pain scores from the final multivariate model for prediction of follow-up pain. The diagonal line perfect agreement between predicted and observed values is shown for reference

Post-treatment variables

Patient characteristics evaluated within several days after completion of care (Table 3) show that, on average, patients were in mild to moderate pain (31/100) and disability (28/100). Patients generally agreed or strongly agreed, on average, that they were satisfied with care. Patient perception of provider confidence in treatment was more neutral because the study doctors were trained to show equipoise with regards to study interventions. Lumbar global range of motion increased only about 2°, on average, in each direction. Overall, patients had greater pain pressure thresholds than at baseline.
Table 3

Post-treatment (6-week) characteristics and univariate pain-prediction modelsa

Post-treatment (six-week) variables

Post-treatment Mean (SD) (N = 385)

Responder models

Future pain intensity models

Dose (per 6 spinal manipulation visits)

   

Time (in weeks)

   

Pain/Disability (6wk)

   

 Pain intensity (0–100)

30.9 (17.2)

 Functional disability (0–100)

27.7 (20.1)

 Perceived change in pain (1–6)

4.2 (0.9)

 Perceived change in pain score (−100 to 100)

−38.3 (33.0)

 Perceived change in disability (1–6)

3.9 (0.9)

 Number of outside care visits prior 4 weeks

0.1 (1.3)

 

Pain unpleasantness (0–100 scale)

19.6 (18.9)

 Days with pain (last 4 wk)

16.9 (10.5)

 Days with disability (last 4 wk)

1.6 ( 3.8)

 

Psychosocial (6wk)

   

 Satisfaction with chiropractor’s time listening (1–5)

4.7 ( 0.7)

 Satisfaction with chiropractor’s comfort treating LBP (1–5)

4.8 ( 0.5)

 

 Satisfaction with chiropractor’s enthusiasm for treatment (1–5)

4.5 ( 0.8)

 

 Satisfaction with chiropractor’s confidence in treatment (1–5)

3.5 ( 1.0)

 Mean satisfaction with chiropractor (1–5)

4.4 ( 0.5)

 Confidence treatment is working (1–7)

4.9 ( 1.7)

Objective Physical Exam (6wk)b

   

 Lumbar ROM: flexion

45.5 (17.6)

  

 Lumbar ROM: extension

17.0 ( 9.5)

  

 Lumbar ROM: right lateral bending

20.9 ( 9.7)

  

 Lumbar ROM: left lateral bending

20.5 ( 9.6)

  

 LBP: flexion (0–10)

1.1 ( 1.8)

 LBP: extension (0–10)

1.7 ( 2.0)

 LBP: right lateral bending (0–10)

1.4 ( 1.9)

 LBP: left lateral bending (0–10)

1.3 ( 1.7)

 LBP: sum for 4 lumbar ROMs pain scores (each 0–10)

5.4 ( 6.1)

 LBP: maximum of 4 lumbar ROMs pain scores

2.5 ( 2.2)

 LBP: right – left lateral bending

0.1 ( 1.4)

 LBP: |right – left lateral bending|

0.8 ( 1.2)

 LBP: sum for right and left lateral bending pain scores

2.7 ( 3.4)

 LBP: maximum of right and left lateral bending pain scores

1.7 ( 2.0)

 Modified Schober Test (cm)

21.0 ( 1.9)

  

 Lumbar hypomobility: L1 % (n)

43 % (157)

  

 Lumbar hypomobility: L2 % (n)

38 % (138)

  

 Lumbar hypomobility: L3 % (n)

27 % (100)

  

 Lumbar hypomobility: L4, % (n)

29 % (104)

 Lumbar hypomobility: L5, % (n)

43 % (155)

 Total hypomobile joints: L1 thru L5

1.8 ( 1.3)

 Pain Pressure Threshold: right L1-L2

6.8 ( 3.7)

  

 Pain Pressure Threshold: left L1-L2

6.8 ( 3.0)

 

 Pain Pressure Threshold: right L3-L4

6.7 ( 3.3)

 

 Pain Pressure Threshold: left L3-L4

6.8 ( 3.2)

 

 Pain Pressure Threshold: right L5-S1

6.7 ( 3.4)

 

 Pain Pressure Threshold: left L5-S1

6.8 ( 3.5)

 

 Pain Pressure Threshold: minimum of 6 measures

5.4 ( 2.7)

 

OR Odds ratio, r Pearson’s correlation coefficient, β regression coefficient, ROM range of motion, LBP low back pain

Variables with a statistically significant association with outcome, p-value < 0.05, after adjusting for dose

aLogistic and longitudinal linear regressions were adjusted for dose and were fitted using generalized estimating equations to account for correlation across time points. Only the statistically significant variables (p < .05) \in this table are used as candidates for the subsequent inclusion into the relevant final multivariate prediction models

bROM was measured in degrees, LBP during ROM on a 0 to 10 scale for each of the 4 ROMs, and pain pressure threshold in kg. Hypomobility was identified using manual motion palpation

There were no missing values in most variables considered in the post-treatment multivariate models and for those with missing values 8 % or fewer were missing. In aggregate a total of 13.8 and 9.2 % of observations had to be dropped due to missing values for building the multivariate model for responders and for future pain intensity, respectively.

Post-treatment multivariate predictor models

For responder prediction, AUC was 0.750 in the training set and 0.665 for the test set (Table 4, Fig. 3). Greater odds of 50 % improvement in pain intensity were associated with greater dose of SMT and the passage of time. Poorer odds were associated with greater pain intensity, days with pain, and difference in pain with left and right lateral bending.
Table 4

Final multivariate post-treatment pain-prediction models and performance metricsa

 

Responders (N = 249/93)b

Future pain intensity (N = 262/93)b

Independent variables

OR

(95 % CI)

P-value

 

β

(95 % CI)

P-value

Dose (per 6 spinal manipulation visits)

1.14

(0.95, 1.37)

0.150

 

−0.07

(−1.35, 1.21)

0.910

Time (in weeks)

1.08

(1.02, 1.14)

0.004

    

Pain/Disability

       

 Pain intensity

0.64

(0.51, 0.80)

<0.001

 

10.7

(8.84, 12.56)

<0.001

 Days with pain (last 4 weeks)

0.57

(0.46, 0.70)

<0.001

    

Objective Physical Exam

       

 LBP: right – left lateral bending

0.76

(0.63, 0.92)

0.005

    

 LBP: right lateral bending

    

2.95

(1.21, 4.69)

0.001

Performance metricsc

AUC

(95 % CI)

 

RMSE

(95 % CI)

R2

(95 % CI)

Training set

0.750

  

16.3

 

.366

 

Test set

0.665

(0.58, 0.74)

 

17.5

(15.0, 20.1)

.261

(7.5, 43.2)

OR Odds ratio, PC part correlation, β regression coefficient, ROM range of motion, AUC Area under the curve (receiver operating characteristic curve), RMSE root mean squared error (SD of prediction error), R 2 coefficient of determination, LBP low back pain

aVariables were selected into the regression models using forward selection among variables with p < .05 in the univariate analysis; dose was forced into the models. Independent variables were standardized except for dose (scale unit = 6 visits) and time (scale unit = 1 week). Lower scores were favorable for pain and days with pain

bThe first number is the sample size for the model in the training set and the second number is the N for the test set

cChance performance is indicated by 0.5 for AUC. RMSE is the standard deviation of the error in prediction of future pain intensity evaluated on the 0 – 100 pain scale. R 2 is the proportion of the variance in pain intensity explained by the independent variables in the model. Confidence intervals are given for the test set only

Fig. 3

Post-treatment model ROC curves. Receiver operating characteristic (ROC) curves for the final multivariate model for prediction of responders. The area under the curve (AUC) was 0.750 in the training set and 0.665 in the test set. Chance is shown by the diagonal line indicating AUC = 0.5

The ability to predict future pain in the study population can be considered moderate with an R2 of 0.366 for the training set and 0.261 for the test set. (Table 4, Fig. 4). However, the RMSE was fairly large, 16.3 points in the training set and 17.5 points in the test set. Greater future pain intensity was related to greater 6-week pain intensity and pain in right lateral bending.
Fig. 4

Post-treatment model scatterplots. Observed pain scores are plotted against predicted pain scores from the final multivariate model for prediction of follow-up pain. The diagonal line perfect agreement between predicted and observed values is shown for reference

In addition, we would like to point out that the 6-week pain intensity predictor alone had better ability to predict future pain score (R2 = 0.350 in the test set) compared to the full multivariate predictor model (R2 = 0.261 in the test set). The RMSE for the 6-week pain intensity only model was 16.5 points for the training set and 17.6 points for the test set, almost identical to the RMSE for the full multivariate model.

Discussion

Our analysis demonstrates the importance of full internal validation using an independent data set. The differences between the training set and test set in AUC and R2 clearly illustrate the problem of over-optimism (i.e., over-fitting) during model development (Tables 2 and 4) [58]. Whereas the performance of the pretreatment model to predict responders and future pain intensity might be considered promising in the training data set, the internal validation of the model in the test set showed failure to identify responders better than chance and failure to explain variation in pain intensity after baseline. For the post-treatment predictor models, performance during internal validation was not as strong as during model development, but still demonstrated some ability to predict both responders and future pain intensity.

A desirable feature for clinical practice is the simplicity of a prediction model. In our models, post-treatment pain intensity alone predicted future pain intensity as well as or better than the post-treatment model built using many more predictor variables. The predictive ability of the single variable model also did not decrease as drastically between the training and test sets as the predictive ability of the multivariate model. This reflects the fact that fitting a model to a single variable does not result in as much over-optimism (over-fitting) as building a multivariate model using a large number of predictors. The sustained predictive ability of the post-treatment, pain-intensity-only model suggests that this variable is very likely a true predictor of future pain and would very likely be predictive of pain intensity in further studies carried out in similar populations.

In our study, participants were treated for 18 visits over six weeks. Timelines of treatment vary across different predictive studies. For example, the Nordic Back Pain study only looked at patients for the first four visits over an unspecified period of time [59, 60]. They also looked at patients with and without sciatica while we excluded people with pain below the knee; and they looked at daily versus intermittent pain which we did not. Of the predictors in their initial model, only increased pain also predicted a worse prognosis in our model. LeBoeuf-Yde et al. [17] found that early recovery was a strong predictor for 12 month recovery. This matches our findings that those with less pain at the end of care were more likely to have successful treatment over the rest of the year. However, Leboeuf-Yde et al. had no test set and it is difficult to tell if their model would hold in a similar population.

Limitations

The goal in developing the multivariate predictor models was to predict pain outcome for individuals. This is a different objective from the development of a model that seeks to identify specific predictors that impact the outcomes. While our models do identify potential predictors of pain (in an exploratory sense), they cannot reliably claim that variables selected into the models are determinants of pain and variables not selected into the models are not. For example, functional disability might in general be expected to predict pain outcomes. We found that among single predictors other than pain, this variable had the strongest correlation with future pain intensity but it was not included in the multivariate model because of its correlation with the pain predictors that were included. In general, the effort of building a good predictive model does not necessarily lead to reliable identification of specific risk factors that truly impact the outcome. The ability to identify these variables is compromised by the large number of the predictors and by the correlation among them.

A related issue is the use of p-values for selection of the specific variables in our results. The univariate p-values should be interpreted as indicators of variable importance to guide variable selection, not as traditional hypothesis tests of statistical significance. The confidence intervals and p-values for the coefficients and odds ratios in the multivariate regression models must also be viewed as exploratory because both tend to be over-optimistic in forward variable selection. We considered only variables with univariate p < 0.05 in the multivariate analysis (a univariate filter) to reduce this issue at least to some extent. This choice may have theoretically eliminated some true predictors from the multivariate model. However, reducing the potential for over-fitting by considering fewer variables in the multivariate selection process was more desirable in our opinion.

We also would like to emphasize that we did not endeavor to create a clinical prediction rule. We did not have a meaningful clinical comparator such as another efficacious treatment or no treatment at all. We also did not have the power to evaluate potential effect modifiers of intervention including the interaction effect between treatment alternatives and the final prediction-rule recommendation [61]. However, the results from our study data do suggest that it may be difficult to develop a prediction rule to identify patients who might benefit from SMT for cLBP before they receive treatment.

The principal limitation was that the study was a randomized controlled trial with treatment mostly limited to SMT of the low back. The baseline models included a rigorously controlled treatment environment for the first six weeks. Even though the post-treatment models are based on the clinical course of cLBP, the influence of other practice characteristics on patient outcomes not included in the prediction models cannot be ruled out, such as full-spine manipulation, physical modalities, supervised exercise, supplements, and advice on posture, diet, stress management, and other lifestyle and self-care measures [62]. The observation that decreased future pain is associated with decreased pain with right lateral bending, and not left, is questionable and may simply be an artifact of assessing so many predictors. Pain with some form of lumbar motion does, however, appear in all models. External validation in common clinical practice applied to a variety of patient population is required to evaluate the robustness of applicability of our models.

Conclusion

Internal validation of prediction models showed that participant characteristics preceding the start of care were poor predictors of responders (at least 50 % improvement in pain intensity) and future pain intensity as well. Variables collected shortly after completion of 6 weeks of study intervention predicted future pain the best. The findings from this exercise of model development remind us that creating prediction models is difficult. We are also reminded of the importance of validating models. Our results suggest that the simplest model and the best predictor may be post-treatment pain.

Abbreviations

AUC: 

Area under the receiver operating curve

β: 

Regression coefficient

cLBP: 

Chronic low back pain

CI: 

Confidence interval

OR: 

Odds ratio

R2

Coefficient of determination

ROC: 

Receiver operating characteristics curve

SD: 

Standard deviation

RMSE: 

Root mean square error

SMT: 

Spinal manipulative therapy

VAS: 

Visual analogue scale

Declarations

Acknowledgments

The study, including all authors, was funded by the National Center for Complementary and Integrative Health (NCCIH) of the National Institutes of Health (U01 AT001908). NCCIH approved all study protocols. The contents of this publication are the sole responsibility of the authors and do not necessarily reflect the official views of NCCIH.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Authors’ Affiliations

(1)
University of Western States
(2)
The Mountain-Whisper-Light Statistics

References

  1. Vos T, Flaxman AD, Naghavi M, Lozano R, Michaud C, Ezzati M, et al. Years lived with disability (YLDs) for 1160 sequelae of 289 diseases and injuries 1990–2010: a systematic analysis for the Global Burden of Disease Study 2010. Lancet. 2012;380(9859):2163–96.View ArticlePubMedGoogle Scholar
  2. Freburger JK, Holmes GM, Agans RP, Jackman AM, Darter JD, Wallace AS, et al. The rising prevalence of chronic low back pain. Arch Intern Med. 2009;169(3):251–8.View ArticlePubMedPubMed CentralGoogle Scholar
  3. Haas M, Vavrek D, Peterson D, Polissar N, Neradilek MB. Dose–response and efficacy of spinal manipulation for care of chronic low back pain: a randomized controlled trial. Spine J. 2014;14(7):1106–16.View ArticlePubMedGoogle Scholar
  4. Bronfort G, Haas M, Evans R, Leininger B, Triano J. Effectiveness of manual therapies: the UK evidence report. Chiropr Osteopat. 2010;18:3.View ArticlePubMedPubMed CentralGoogle Scholar
  5. Goertz CM, Pohlman KA, Vining RD, Brantingham JW, Long CR. Patient-centered outcomes of high-velocity, low-amplitude spinal manipulation for low back pain: a systematic review. J Electromyogr Kinesiol. 2012;22(5):670–91.View ArticlePubMedGoogle Scholar
  6. Vavrek DA, Sharma R, Haas M. Cost analysis related to dose–response of spinal manipulative therapy for chronic low back pain: outcomes from a randomized controlled trial. J Manipulative Physiol Ther. 2014;37(5):300–11.View ArticlePubMedPubMed CentralGoogle Scholar
  7. Flynn T, Fritz J, Whitman J, Wainner R, Magel J, Rendeiro D, et al. A clinical prediction rule for classifying patients with low back pain who demonstrate short-term improvement with spinal manipulation. Spine. 2002;27(24):2835–43.View ArticlePubMedGoogle Scholar
  8. Nyiendo J, Haas M, Goldberg B, Sexton G. Pain, disability and satisfaction outcomes and predictors of outcomes: a practice-based study of chronic low back pain patients attending primary care and chiropractic physicians. J Manipulative Physiol Ther. 2001;24(7):433–9.View ArticlePubMedGoogle Scholar
  9. Haas M, Goldberg B, Aickin M, Ganger B, Attwood M. A practice-based study of patients with acute and chronic low back pain attending primary care and chiropractic physicians: two-week to 48-month follow-up. J Manipulative Physiol Ther. 2004;27(3):160–9.View ArticlePubMedGoogle Scholar
  10. Cai C, Pua YH, Lim KC. A clinical prediction rule for classifying patients with low back pain who demonstrate short-term improvement with mechanical lumbar traction. Eur Spine J. 2009;18(4):554–61.View ArticlePubMedPubMed CentralGoogle Scholar
  11. Hicks GE, Fritz JM, Delitto A, McGill SM. Preliminary development of a clinical prediction rule for determining which patients with low back pain will respond to a stabilization exercise program. Arch Phys Med Rehabil. 2005;86(9):1753–62.View ArticlePubMedGoogle Scholar
  12. Hall AJ, Logan JE, Toblin RL, Kaplan JA, Kraner JC, Bixler D, et al. Patterns of abuse among unintentional pharmaceutical overdose fatalities. JAMA. 2008;300(22):2613–20.View ArticlePubMedGoogle Scholar
  13. Graham NA, Gold MS, Goldberger BA. Prescription opioids and overdose deaths. JAMA. 2009;301(17):1766. author reply 1767–8.View ArticlePubMedGoogle Scholar
  14. Painkillers fuel growth in drug addiction. Opioid overdoses now kill more people than cocaine or heroin. Harv Ment Health Lett. 2011, 27(7):4–5.Google Scholar
  15. Kuehn BM. Methadone overdose deaths rise with increased prescribing for pain. JAMA. 2012;308(8):749–50.View ArticlePubMedGoogle Scholar
  16. Centers for Disease Control and Prevention (CDC). Vital signs: risk for overdose from methadone used for pain relief - United States, 1999–2010. MMWR Morb Mortal Wkly Rep. 2012;61(26):493–7.Google Scholar
  17. Leboeuf-Yde C, Gronstvedt A, Borge JA, Lothe J, Magnesen E, Nilsson O, et al. The Nordic back pain subpopulation program: demographic and clinical predictors for outcome in patients receiving chiropractic treatment for persistent low-back pain. J Manipulative Physiol Ther. 2004;27(8):493–502.View ArticlePubMedGoogle Scholar
  18. Dougherty PE, Karuza J, Savino D, Katz P. Evaluation of a modified clinical prediction rule for use with spinal manipulative therapy in patients with chronic low back pain: a randomized clinical trial. Chiropr Man Therap. 2014;22(1):41-014-0041-8. eCollection 2014.View ArticleGoogle Scholar
  19. Haas M, Vavrek D, Neradilek MB, Polissar N. A path analysis of the effects of the doctor-patient encounter and expectancy in an open-label randomized trial of spinal manipulation for the care of low back pain. BMC Complement Altern Med. 2014;14(1):16-6882-14-16.View ArticleGoogle Scholar
  20. Peterson DH, Bergmann TF. Chiropractic technique: principles and practice. St. Louis: Mosby; 2002.Google Scholar
  21. Furlan AD, Brosseau L, Imamura M, Irvin E. Massage for low back pain: a systematic review within the framework of the Cochrane collaboration back review group. Spine. 2002;27:1896–910.View ArticlePubMedGoogle Scholar
  22. Cherkin DC, Deyo RA, Sherman KJ, Hart G, Street JH, Hrbek A, et al. Characteristics of licensed acupuncturists, chiropractors, massage therapists and naturopathic physicians. J Am Board Fam Pract. 2002;15:463–72.PubMedGoogle Scholar
  23. Underwood MR, Barnett AG, Vickers MR. Evaluation of two time-specific back pain outcome measures. Spine. 1999;24(11):1104–12.View ArticlePubMedGoogle Scholar
  24. Deyo RA, Dworkin SF, Amtmann D, Andersson G, Borenstein D, Carragee E, et al. Report of the NIH Task Force on research standards for chronic low back pain. J Pain. 2014;15(6):569–85.View ArticlePubMedPubMed CentralGoogle Scholar
  25. McLeod LD, Coon CD, Martin SA, Fehnel SE, Hays RD. Interpreting patient-reported outcome results: US FDA guidance and emerging methods. Expert Rev Pharmacoecon Outcomes Res. 2011;11(2):163–9.View ArticlePubMedPubMed CentralGoogle Scholar
  26. Fritz JM, Hebert J, Koppenhaver S, Parent E. Beyond minimally important change: defining a successful outcome of physical therapy for patients with low back pain. Spine (Phila Pa 1976). 2009;34(25):2803–9.View ArticleGoogle Scholar
  27. Price DD. Psychological mechanisms of pain and analgesia. Seattle, WA: IASP Press; 1999.Google Scholar
  28. Haas M, Spegman A, Peterson D, Aickin M, Vavrek D. Dose response and efficacy of spinal manipulation for chronic cervicogenic headache: a pilot randomized controlled trial. Spine J. 2010;10(2):117–28.View ArticlePubMedPubMed CentralGoogle Scholar
  29. Haas M, Groupp E, Kraemer DF. Dose–response for chiropractic care of chronic low back pain. Spine J. 2004;4(1529–9430):574–83.View ArticlePubMedGoogle Scholar
  30. Luo N, Johnson JA, Shaw JW, Feeny D, Coons SJ. Self-reported health status of the general adult U.S. population as assessed by the EQ-5D and Health Utilities Index. Med Care. 2005;43(0025–7079; 11):1078–86.View ArticlePubMedGoogle Scholar
  31. van Agt HM, Essink-Bot ML, Krabbe PF, Bonsel GJ. Test-retest reliability of health state valuations collected with the EuroQol questionnaire. Soc Sci Med. 1994;39(11):1537–44.View ArticlePubMedGoogle Scholar
  32. Soer R, Reneman MF, Speijer BL, Coppes MH, Vroomen PC. Clinimetric properties of the EuroQol-5D in patients with chronic low back pain. Spine J. 2012;12(11):1035–9.View ArticlePubMedGoogle Scholar
  33. Ware JE, Kosinski M, Keller SD. A 12-item short-form health survey: construction of scales and preliminary tests of reliability and validity. Med Care. 1996;34(3):220–33.View ArticlePubMedGoogle Scholar
  34. Luo X, George ML, Kakouras I, Edwards CL, Pietrobon R, Richardson W, et al. Reliability, validity, and responsiveness of the short form 12-item survey (SF-12) in patients with back pain. Spine. 2003;28(15):1739–45.PubMedGoogle Scholar
  35. Waddell G, Newton M, Henderson I, Somerville D, Main CJ. A fear-avoidance beliefs questionnaire (FABQ) and the role of fear avoidance beliefs in chronic low back pain and disability. Pain. 1993;52(2):157–68.View ArticlePubMedGoogle Scholar
  36. Mayer TG, Tencer AF, Kristoferson S, Mooney V. Use of noninvasive techniques for quantification of spinal range-of-motion in normal subjects and chronic low-back dysfunction patients. Spine (Phila Pa 1976). 1984;9(6):588–95.View ArticleGoogle Scholar
  37. Loebl WY. Measurement of spinal posture and range of spinal movement. Ann Phys Med. 1967;9(3):103–10.PubMedGoogle Scholar
  38. Troup JD, Hood CA, Chapman AE. Measurements of the sagittal mobility of the lumbar spine and hips. Ann Phys Med. 1968;9(8):308–21.PubMedGoogle Scholar
  39. Keeley J, Mayer TG, Cox R, Gatchel RJ, Smith J, Mooney V. Quantification of lumbar function. Part 5: Reliability of range-of-motion measures in the sagittal plane and an in vivo torso rotation measurement technique. Spine (Phila Pa 1976). 1986;11(1):31–5.View ArticleGoogle Scholar
  40. Jensen MP, Karoly P, Braver S. The measurement of clinical pain intensity: a comparison of six methods. Pain. 1986;27:117–26.View ArticlePubMedGoogle Scholar
  41. Macrae IF, Wright V. Measurement of back movement. Ann Rheum Dis. 1969;28(6):584–9.View ArticlePubMedPubMed CentralGoogle Scholar
  42. Moll JM, Wright V. Normal range of spinal mobility. An objective clinical study. Ann Rheum Dis. 1971;30(4):381–6.View ArticlePubMedPubMed CentralGoogle Scholar
  43. Gill K, Krag M, Johnson G, Haugh L, Pope M. Repeatability of four clinical methods for assessment of lumbar spinal motion. Spine. 1988;13(1):50–3.View ArticlePubMedGoogle Scholar
  44. Williams R, Binkley J, Bloch R, Goldsmith CH, Minuk T. Reliability of the modified-modified Schöber and double inclinometer methods for measuring lumbar flexion and extension. Phys Ther. 1993;73:26–37.Google Scholar
  45. Robinson HS, Mengshoel AM. Assessments of lumbar flexion range of motion: intertester reliability and concurrent validity of 2 commonly used clinical tests. Spine (Phila Pa 1976). 2014;39(4):E270–5.View ArticleGoogle Scholar
  46. Childs JD, Fritz JM, Flynn TW, Irrgang JJ, Johnson KK, Majkowski GR, et al. A clinical prediction rule to identify patients with low back pain most likely to benefit from spinal manipulation: a validation study. Ann Intern Med. 2004;141(1539–3704; 12):920–8.View ArticlePubMedGoogle Scholar
  47. Stochkendahl MJ, Christensen HW, Hartvigsen J, Vach W, Haas M, Hestbaek L, et al. Manual examination of the spine: a systematic critical literature review of reproducibility. J Manipulative Physiol Ther. 2006;29(1532–6586; 6):475–85.View ArticlePubMedGoogle Scholar
  48. Fischer AA. Pressure algometry over normal muscles. Standard values, validity and reproducibility of pressure threshold. Pain. 1987;30:115–26.View ArticlePubMedGoogle Scholar
  49. Ohrback R, Gale EN. Pressure pain thresholds, clinical assessment and differential diagnosis: reliability and validity in patients with myogenic pain. Pain. 1989;39:157–62.View ArticleGoogle Scholar
  50. Cherkin DC, MacCornack FA. Patient evaluations of low back pain care from family physicians and chiropractors. West J Med. 1989;150(3):351–5.PubMedPubMed CentralGoogle Scholar
  51. Cherkin D, Deyo RA, Berg AO. Evaluation of a physician education intervention to improve primary care for low-back pain: II. Impact on patients. Spine. 1991;16(10):1173–8.View ArticlePubMedGoogle Scholar
  52. Liang KY, Zeger SL. Longitudinal data analysis using generalized linear models. Biometrika. 1986;71(1):13.View ArticleGoogle Scholar
  53. Zeger SL, Liang KY. Longitudinal data analysis for discrete and continuous outcomes. Biometrics. 1986;42(1):121–30.View ArticlePubMedGoogle Scholar
  54. Diggle PJ, Heagerty PJ, Liang KY, Zeger SL. Analysis of Longitudinal Data. 2nd ed. New York: Oxford University Press; 2002.Google Scholar
  55. Hocking RR. A biometrics invited paper. The analysis and selection of variables in linear regression. Biometrics. 1976;32(1):1–49.View ArticleGoogle Scholar
  56. Proceedings of the Proceedings of Graphics Interface 2005: Victoria, British Columbia. School of Computer Science, University of Waterloo, Waterloo, Ontario, Canada: Canadian Human-Computer Communications Society; 2005.Google Scholar
  57. Weisberg S. Applied linear regression. 3rd ed. Hoboken, N.J.: Wiley-Interscience; 2005.View ArticleGoogle Scholar
  58. Kuhn M, Johnson K. Applied predictive modeling. New York: Springer; 2013.View ArticleGoogle Scholar
  59. Axen I, Rosenbaum A, Robech R, Larsen K, Leboeuf-Yde C. The Nordic back pain subpopulation program: can patient reactions to the first chiropractic treatment predict early favorable treatment outcome in nonpersistent low back pain? J Manipulative Physiol Ther. 2005;28(3):153–8.View ArticlePubMedGoogle Scholar
  60. Axen I, Jones JJ, Rosenbaum A, Lovgren PW, Halasz L, Larsen K, et al. The Nordic Back Pain Subpopulation Program: validation and improvement of a predictive model for treatment outcome in patients with low back pain receiving chiropractic treatment. J Manipulative Physiol Ther. 2005;28(6):381–5.View ArticlePubMedGoogle Scholar
  61. Stanton TR, Hancock MJ, Maher CG, Koes BW. Critical appraisal of clinical prediction rules that aim to optimize treatment selection for musculoskeletal conditions. Phys Ther. 2010;90(6):843–54.View ArticlePubMedGoogle Scholar
  62. Christensen MG, Kollasch MW, Hyland JK. Practice analysis of chiropractic 2010. Greeley Colorado: National Board of Chiropractic Examiners; 2010.Google Scholar

Copyright

© Vavrek et al. 2015

Advertisement