Treatment compliance and effectiveness of a cognitive behavioural intervention for low back pain: a complier average causal effect approach to the BeST data set

A common problem in clinical trials, as well as clinical practice, is the failure of patients to fully comply with the allocated treatment. In trials of therapist-led intervention packages non-compliance occurs when individuals randomised to the intervention do not attend the number of sessions deemed sufficient for the intervention to deliver a benefit. In a conventionally used intention-to-treat (ITT) analysis all participants are analysed according to their treatment allocation even if they attend few or none of the therapy sessions. The intention-to-treat estimate provides an estimate for the effect of being offered the intervention when often our interest lies in the effect of receiving the treatment.

Complier-average causal effect (CACE) modelling is an analytic approach that provides a robust estimate of the treatment effect amongst compliant participants [1]. We specified a priori that compliance was likely to be an important contributor to the effect of group cognitive behavioural therapy, and designed a trial that was sufficiently pragmatic to allow estimation of these effects in a generalizable sample and range of settings [2]. In the absence of published guidance, we defined compliance as attendance at the assessment session plus three of the six group sessions as we hypothesised that this would enable the key components of the cognitive behavioural intervention to be delivered, although not necessarily re-enforced. In the original intention-to-treat analysis, we demonstrated that group cognitive therapy was effective at and beyond 12 months in a range of clinically relevant outcome measures, with standardised effect sizes mostly in the moderate range [3, 4].

Here our aim was to investigate the nature and impact of non-compliance on the outcomes of group based cognitive behavioural intervention reported by Lamb et al. [3]. In non-technical terms, the concept of CACE is predicated as follows. At the outset of a trial, we assume that all participants have an unobservable characteristic (or set of characteristics, known as a latent variable) that determines whether they would comply or not with the test intervention. With randomisation, we assume that these characteristics are equally distributed across each arm of the trial, and that the proportion of “would be compliers with the test intervention” is therefore also equally distributed across the trial arms [5, 6]. Only those randomised to the test arm have the opportunity to comply with the intervention and we are able to observe the proportion of compliers in the test arm. Because the proportion of would be compliers is equally distributed at the outset, we are able to estimate the proportion in the control group, from the proportion that are observed in the treatment group. We are also able to infer the unobserved mean of the non-compliers in the control group, from the observed average of the non-compliers in the treatment group. By representing the treatment difference observed through intention to treatment analysis as a product of the proportion of compliers and the complier averaged difference, and then re-organising the algebra, it is possible to estimate the complier averaged difference (CACE). Assumptions about missing data are important, and further refinement is added by inclusion of baseline co-variates through more sophisticated statistical techniques. An important assumption is that the offer of the intervention does not influence the outcome [7]. Non-compliers should, on average, have the same mean score in the intervention group as the control group [8, 9].

Few, if any, trials of low back pain interventions have considered the treatment effect amongst compliers using robust analyses. Both the complier average causal effect and linear regression analyses support the concept that compliance has an important role determining the size of treatment effect. People who achieved the compliance threshold achieved a larger benefit in clinical and health related quality of life outcomes in the year of follow up in comparison to advice only, than those who are less compliant.

The estimates from the CACE analysis provide the most robust indication of the treatment effect amongst compliers, although there are some limitations in the analysis we present. First is that there is no accepted method of defining compliance to cognitive behavioural interventions. We based our analysis on the compliance definition set out in the protocol and have examined different thresholds in further analyses due to potential limitations of this definition in the analysis model. Our definition of attendance at the initial assessment and three or more sessions was based on a collective judgement of the intervention designers that this would provide the essential components of the programme. This is broadly in keeping with other reports in the literature [20]. In those defined as non-compliers, some may have received some elements of the cognitive behavioural intervention, and this could possibly lead to an underestimate of the treatment effect amongst compliers [21]. All participants of the trial received a brief, best practice active management advice session from a nurse or physiotherapist. In the intervention arm, 71% of those deemed non-compliant had attended some part of the BeST intervention, most commonly the assessment +/- the first or second group session, and had received the session summary materials for all sessions. Despite this we still observed a substantial and statistically significant difference in the treatment effect estimate.

We quantified compliance in terms of treatment attendance. CACE analysis assumes that compliance is a pre-randomisation characteristic and hence equally distributed across the treatment arms. Our data supports the hypothesis that the characteristic of being a complier is associated with a larger treatment benefit than being a non-complier. Whether compliance is potentially modifiable, and whether modifying the attendance behaviour of patients results in better outcomes is less certain.

There are several potential mechanisms by which compliance is associated with a larger treatment effect. First is reverse causality, i.e. that compliance is driven by early response to treatment and that as opposed to compliance being important to response, the early response engenders compliance. High pain at study entry (i.e. a pre-randomisation variable) was associated with non-compliance. We cannot rule in or out this hypothesis as we did not take measures of response on a weekly basis. Future studies could address this issue, and this would be important in understanding what drives compliance in the context of low back pain. The second potential mechanism is that greater attendance results in re-enforcement of the key components of the intervention. In designing the intervention some of the concepts were repeated across sessions, and the therapists were encouraged to revisit and draw on the experiences and skills mastered in prior sessions. Participants undertook regular review of their treatment goals, and progression toward them. Finally, it is possible that the later sessions contain elements that are substantially important to the effect, or that the increasing social contact across repeated visits has a therapeutic value.

We are not able to conclude from our results whether a shorter programme (3 or less sessions) would be as good as six sessions, as we did not test this explicitly.

An important assumption of the CACE analysis, the exclusion restriction, is that for non-compliers in the intervention arm there is no additional benefit gained from the offer of treatment compared to participants randomised to the advice group [7, 9]. In this study participants that are randomised to the intervention are treated as non-compliers even if they might have attended one or two sessions as well as the individual assessment. We would expect that at least some of these participants would have received a partial benefit and so this could violate the exclusion restriction and introduce bias. We used a CACE model adjusted for covariates associated with outcome score and compliance which protects against bias if, as described above, there is violation of the exclusion restriction assumption [21]. It has also been shown that bias introduced by violation of the exclusion criteria is increased in the CACE estimate if the compliance rate is very low [21]. This was not the case; compliance within the group cognitive behavioural therapy was 63%.

Further CACE analyses show that when compliance is defined as attendance at a minimum of the individual assessment the treatment effect is still estimated to be greater than the ITT estimate at 12 months. This is not unexpected as, in a parallel qualitative study, a number of the participants of the intervention mentioned how valuable they found the assessment session independently of the group sessions [2]. The treatment effect estimates based on attendance at the individual assessment can be defined as the treatment effect of partial and complete compliers; it is the treatment effect excluding complete non-compliers. This provides an unbiased estimate of treatment effect and again demonstrates that the treatment effect estimated from an ITT analysis is an underestimate of the actual effect of treatment.

We have compared the estimates from the CACE model to ITT estimates so that it is comparable to our original reporting. An as-treated analysis is not reported as in the presence of non-compliance this method is considered to be misleading and so does not provide robust estimates [22].

Missing follow up data was shown to be related to compliance and so identification of non-compliers could be useful in managing the collection of follow up data. It also follows that if compliance rates can be increased through encouraging attendance at all sessions then this should lead to an increase in the completeness of follow up data. Under the missing at random assumption used in this CACE analysis the outcome response behaviour is permitted to differ between complier groups in the intervention arm [23]. There is also evidence from prior methodological work that CACE estimates produced from latent class modelling are insensitive to the missing data assumption used, missing at random or latent ignorability [24]. We undertook additional sensitivity analyses using multiply imputed datasets, and these suggest that the CACE results are insensitive to the missing data.

The latent class modelling approach used here to derive CACE estimates does not account for clustering effects such as therapist effects. It is possible to undertake analyses including clustering in a model accounting for non-compliance as demonstrated by B Jo, T Asparouhov and BO Muthén [25]. However, our original ITT analyses demonstrated therapist and group effects were negligible.

This analysis shows that the effectiveness of group cognitive behavioural intervention is increased in compliers. Whilst we cannot be sure that compliance is a modifiable trait, research from other areas suggest that it is, and issues such as family and work commitments in scheduling of sessions, or even in the mode of delivery (for example requiring attendance at a face to face meeting), may render better compliance [26]. We did not use specific behavioural techniques to encourage attendance at the sessions (i.e. attendance at sessions was not identified as a goal), and this approach may add to improved compliance.

There are examples in the literature of analysis of trials subject to non-compliance but these are mainly limited to social interventions and trials of psychological treatments [8, 22, 27]. This current paper is distinctive in providing a CACE analysis of a trial of sub-acute or chronic low-back pain. CACE estimates are potentially more clinically relevant than effect estimates derived from ITT analyses and give a better indication of the effect of receiving treatment.

Exploration of compliance data and use of a CACE analysis can aid the interpretation of the primary results of randomised trials. CACE analysis provides a means of estimating the effect of the receipt of treatment in a randomised trial in which not all patients were compliant. Supplementing estimates from an ITT analysis with CACE estimates helps to provide a more complete picture of the pragmatic effects of an intervention that can support clinical decision making.