Skip to main content

The cost-effectiveness of a treatment-based classification system for low back pain: design of a randomised controlled trial and economic evaluation

Abstract

Background

Systematic reviews have shown that exercise therapy and spinal manipulation are both more effective for low back pain (LBP) than no treatment at all. However, the effects are at best modest. To enhance the clinical outcomes, recommendations are to improve the patient selection process, and to identify relevant subgroups to guide clinical decision-making. One of the systems that has potentials to improve clinical decision-making is a treatment-based classification system that is intended to identify those patients who are most likely to respond to direction-specific exercises, manipulation, or stabilisation exercises.

Methods/Design

The primary aim of this randomised controlled trial will be to assess the effectiveness of a classification-based system. A sample of 150 patients with subacute and chronic LBP who attend a private physical therapy clinic for treatment will be recruited. At baseline, all participants will undergo a standard evaluation by trained research physical therapists and will be classified into one of the following subgroups: direction-specific exercises, manipulation, or stabilisation. The patient will not be informed about the results of the examination. Patients will be randomly assigned to classification-based treatment or usual care according to the Dutch LBP guidelines, and will complete questionnaires at baseline, and 8, 26, and 52 weeks after the start of the treatment. The primary outcomes will be general perceived recovery, functional status, and pain intensity. Alongside this trial, an economic evaluation of cost-effectiveness and cost-utility will be conducted from a societal perspective.

Discussion

The present study will contribute to our knowledge about the effectiveness and cost-effectiveness of classification-based treatment in patients with LBP.

Trial registration

Trial registration number: NTR1176

Peer Review reports

Background

Low back pain (LBP) is a common condition. It has a lifetime prevalence of over 70% in industrialised countries, and accounts for considerable healthcare and socioeconomic costs [1, 2]. It is often not possible to make a specific diagnosis based on patho-anatomical causes, and 85-95% of the cases are diagnosed as "non specific" LBP [3]. The lack of a clear patho-anatomical basis for LBP has resulted in a large variation in LBP diagnoses, and a multitude of poorly studied interventions [4]. It has been assumed that randomised controlled trials (RCTs) could provide answers to questions such as 'which intervention is most effective for which patient', but after the publication of more than 1,000 RCTs focusing on LBP, there is still a lack of evidence regarding the most effective strategies for matching individual patients to specific interventions. The Cochrane Back Review Group acknowledged the limited role of RCTs in providing useful information on aspects of LBP management other than efficacy and effectiveness, and stated that additional etiological, diagnostic, and prognostic studies are needed to identify varieties of LBP, natural courses, or more homogeneous subgroups of patients with LBP [5]. The identification of homogenous subgroups according to evidence-based classification systems was determined as a priority for primary care research on LBP as early as 1996 [6].

Over the years, many attempts have been made to classify patients with LBP into more homogeneous subgroups, based on the specific interventions that they are likely to respond to. In 2007, Billis et al. identified 39 diagnostic and treatment-based classification systems [7]. Most of the classification systems were based on biomedical patient characteristics, and seldom on psychosocial or biopsychosocial features. The majority of the classification systems was based on a judgemental approach, relying on clinical experience and intuition, and a minority was based on a statistical approach and prospective study designs.

One of the few classification systems that has potentials to improve outcomes is a treatment-based classification approach, originally proposed by Delitto et al. [8]. This approach is based on the patient's history, clinical presentation, and a physical examination. It identifies subgroups of patients who are most likely to respond to the following interventions: (1) direction-specific exercises, (2) manipulation, (3) stabilisation, and (4) traction. Since the early nineties, several derivation and validation studies have focused on these four interventions and the entire classification system (Additional file 1) [9–20]. Two studies have investigated the validity of the entire classification system, but because the studies used different RCT designs, they answered different research questions. Fritz et al. [12] investigated the effectiveness of the overall treatment approach of the classification system (classification decision-making and treatment protocols), whereas Brennan et al. [18] focused on the effectiveness of the classification decision-making. In a RCT, Fritz et al. [12] compared treatment according to the classification system with treatment according to the clinical guidelines (low-stress aerobic exercises and advice to remain active) for 78 patients with acute, work-related LBP. They found statistically significant better results for the outcomes of disability and return to work in patients who were receiving classification-based treatment at the four-week follow-up, but not at the one-year follow-up. Their research addressed the efficacy of the overall treatment approach of the classification system. However, the results may have been influenced by the treatment protocols that were used, and not the classification decision-making process. Brennan et al. [18] classified 123 patients with a duration of LBP of less than 90 days according to the classification system. All patients were randomised to receive direction-specific exercises, manipulation or stabilisation, regardless their classification. Outcomes were compared between patients who had received 'matched treatment' (according to the classification) and 'unmatched' (non-classification) treatment. The authors found a statistically significant reduction in disability, favouring the matched treatment group after 4 and 52 weeks.

The reliability of the classification system has been assessed in four studies. In a study of 43 patients with acute LBP, Fritz et al. [21] investigated the inter-rater reliability of the classification of patients into one of the subgroups by 7 physical therapists (PTs) who were familiar with the system. They achieved a kappa value of 0.49. Kiesel et al. [22] achieved a slightly higher kappa value (0.65) with 8 PTs who were familiar with the classification system and 30 patients with LBP. Heiss et al. [23], in a study in which 45 acute LBP patients were classified by four PTs who were unfamiliar with the classification system, found a low kappa value (0.15). Finally, Fritz et al. [24] investigated the inter-rater reliability of the system in a vignette study with 30 PTs and 123 LBP patients, and reported a kappa value of 0.60. These findings therefore provide preliminary evidence that PTs who are familiar with the classification system can obtain an acceptable level of intra-rater reliability.

It is essential for a treatment-based classification approach to be dynamic, and to incorporate new evidence. Since the original proposal, made by Delitto and colleagues, the criteria for allocating a patient to a specific intervention, and also the intervention procedures, have been revised and updated. However, although this continued evolution of a classification system is necessary in order to incorporate emerging evidence, it does reduce the comparability of studies using different versions of the treatment-based classification approach, and should motivate continued research on the approach.

In summary, the classification system proposed by Delitto and colleagues [8] has evolved considerably, and some promising results have been reported. However, the evidence for its clinical usefulness is still limited. Therefore, the aim of the present study is to evaluate the cost-effectiveness of classification-based treatment, compared to usual care according to national clinical guidelines, in patients with subacute and chronic LBP in primary care in the Netherlands. The present study will focus on a revised version of the Delitto et al. treatment-based classification system [8], based on updated evidence outlined in previous RCTs [25], but is consistent with the concept that sub-groups of patients can be identified, and that the primary subgroups are patients who are likely to respond to direction-specific exercises, stabilisation or manipulation. The aim of this paper is to describe the rationale and design of the study.

Methods/Design

Design

A randomised controlled trial will be conducted, together with a full economic evaluation. Figure 1 provides an overview of the study design.

Figure 1
figure 1

Design of the study.

Setting

Up to 300 private physical therapy clinics, with more than 1,000 PTs in the city of Amsterdam and the surrounding (rural) area (< 50-kilometre radius), will be invited to participate.

Ethical approval

In February 2008, the Medical Ethics Committee of the VU University Medical Centre in Amsterdam approved the study protocol (registration number: 2008\5).

Study population

The study population will consist of patients who seek care in a participating private physical therapy clinic. The inclusion criteria will be: LBP as primary complaint, with or without associated leg pain, age between 18 and 65 years, current episode >6 weeks, able to speak and understand Dutch, and willing to give written informed consent. The exclusion criteria will be: known or suspected specific LBP (e.g. cauda equina compression, fractures), severe radiculopathy, spondylolisthesis (grade 2 or more), serious co-morbidity, serious psychopathology, lumbar spinal surgery in the previous year, more than one low back operation, more than one year of absence from work due to LBP, pregnancy or given birth in the past three months, inability to attend 6 or more regular physical therapy appointments, moderate complaints about one or more items of the Urogenital Distress Inventory (UDI 6, short form) or inability to demonstrate any LBP symptoms during mechanical examination.

Recruitment of patients

PTs will inform all eligible LBP patients who attend their clinic about the study. Subsequently, the principal investigator (AA) will phone potentially eligible patients to explain the trial procedures in detail and to check the selection criteria. If a patient fulfils the initial requirements for eligibility, an appointment will be made at the private physical therapy clinic that the patient attended, where the patient will be asked to sign an informed consent form and to complete baseline questionnaires. The patient will then undergo a physical examination.

The PTs will also be asked to collect basic data from eligible patients who are not willing to participate, and to record their reasons for declining.

Research physical therapists

All patients will be examined by one of four research PTs, all of whom have followed a post-graduate course in manipulation and mobilisation techniques, and are certified manual therapists (MTs) in the Netherlands. Two research PTs have a Mechanical Diagnosis and Therapy certificate (MDT or McKenzie method). The two research PTs with no MDT certificate attended a one-day MDT training session supervised by a senior educator of the MDT Institute (HvH) to familiarise them with the MDT system. Prior to the study, the research PTs held meetings to standardise the examination protocol, to train in examination techniques and to examine patients together. The research PTs will examine and classify the patients, but will not be involved in providing the intervention.

Classification system

The classification system that will be used in this study is a revised version of the system used by Brennan et al. [18]. The classification system is a decision-making procedure that is used to allocate patients to one of the following treatment categories: direction-specific exercises, manipulation, or stabilisation (Figure 2).

Figure 2
figure 2

Modified treatment-based classification system for patients with LBP (>6 weeks) as used in this RCT.

Baseline examination

The examination procedure will start with a standardised history-taking, which will include questions on the frequency of episodes, current duration, intensity, and location of the symptoms. Subsequently, a standardised comprehensive physical assessment will be made, together with a neurological examination if indicated. The examination starts with the determination of a directional preference, based on the definition proposed in the MDT classification method (step 1 in Figure 2) [26]. This involves the monitoring of symptomatic and mechanical responses during repetitive end-range lumbar test movements (flexion, extension, side-gliding, or rotation) and/or prolonged positioning. A directional preference is established if certain postures or repeated end-range movements in one direction (e.g. flexion) cause the symptoms to improve or centralise (pain moves from a distal to a proximal area), and in the opposite direction (e.g. extension) cause the symptoms to worsen or peripheralise (pain moves from a proximal to a distal area). Priority is given to peripheral pain locations and their response, regardless of the response of the central pain intensity. The improved or centralised symptoms must be retained in a neutral standing position. A directional preference is not established if the symptoms and the pain immediately reappear. For the sake of clarity, patients with a directional preference do not necessarily have centralisation of the symptoms. After this assessment, several other physical tests, such as the prone instability test, thigh thrust test, and the active straight leg raise test will be performed (step 2 in Figure 2). The operational definitions of these tests are summarised in the Appendix (Additional file 2). The classification decision-making has a hierarchical structure, however, all patients will undergo the entire examination procedure. In cases of uncertainty about the primary classification, extra criteria, such as three or more prior LBP episodes in the medical history and a visible lateral shift during the examination, will be used to determine the patient's subgroup (step 3 in Figure 2). Directly after this examination the research PT will determine the patient's classification and the level of firmness of the conclusion: level 1. there is an obvious primary classification; level 2. two of the three classifications are plausible; and level 3. none or all of the three classifications are plausible. The patient will not be informed about the results of the examination. Reasons for exclusion, such as inability to demonstrate any LBP symptoms during the examination, will be registered. It is the intention that the principal investigator (AA) will examine all patients. To identify patients reliably, one of the three other research PTs, depending on availability, will attend the examination procedure. When two research PTs examine a patient, the primary classification will be reached by consensus.

Treatment allocation

To conceal the treatment allocation, an independent researcher who is not involved in the selection of the patients will randomise each patient to one of the two treatment groups. Randomisation lists will be generated by computer before the study starts, pre-stratified for the duration of the complaints (6-12 weeks or >12 weeks) and disability (Oswestry Disability Index [ODI] <25% or ≥25%) to prevent unequal distribution. To prevent unequal treatment-group sizes, the patients will be randomised according to a stratified block randomisation method, in blocks of four. No treatment will be given prior to randomisation.

Directly after the randomisation procedure, the PT who will provide treatment will be informed about the group allocation. Patients will be scheduled for their first treatment session as soon as possible in the private practice they attended.

Physical therapists in the participating clinics

All patients will be treated by PTs in the participating clinics. In each clinic, at least one PT will be instructed about the classification system, and at least one PT will be instructed about the usual care procedure, depending on preference and expertise. PTs who will provide manipulation treatment must be licensed, post-graduate MTs or in the 4th and final year of the MT master education program at the Educational Center for Musculoskeletal Therapies (SOMT) in the Netherlands. No extra qualifications will be required to treat patients according to the classifications of direction-specific exercises and stabilisation. The number of years of experience of the PTs, and their post-graduate education in MT and/or MDT, will be recorded for both treatment groups. To minimise contamination, the PTs who will provide the classification-based treatment will not be allowed to provide usual care, and vice versa.

The PTs who will provide the classification-based treatment will follow a five-hour training session in which the treatment protocols will be described and discussed. Experts on direction-specific exercises (HvH) and stabilisation will provide the training. The PTs will receive a detailed manual of the study, and a translation of the registration forms and treatment protocols used in the Brennan et al. study [18]. The PTs providing usual care will receive a two-hour training course in the rationale and treatment components described in the LBP guidelines issued by the Royal Dutch College for Physical Therapy (KNGF) [27, 28].

Intervention according to the Dutch LBP guidelines

Patients assigned to usual care will receive PT or MT according to the current Dutch LBP guidelines [27, 28]. The PTs providing usual care will not be informed about the physical examination and classification procedure, and will therefore reassess the patient to obtain the relevant information for treatment. However, to avoid inconvenience for the patient, and to increase the chance that the patient remains unaware of the treatment allocation, the PT will receive a written description of the patient's medical history from the research PTs. The patients will receive supervised, individually tailored treatment based on their clinical presentation. Treatment techniques vary within and among PTs, and depend on the preferences and expertise of the PTs. The focus of the treatment will be on improving function and participation. Interventions that are commonly used in Dutch primary care are 'hands-on' muscular mobilisation techniques, specific articular mobilisation and manipulation techniques, strength and stability exercises, massage, and cognitive, respondent and operant techniques, such as relaxation exercises and graded activity programmes. Decisions about the frequency and the number of sessions are left to the discretion of the PT. The number of treatment sessions, the treatment modalities, the treatment goals, any deviations from the protocol, and reasons for prematurely terminating treatment, will be recorded.

Interventions according to the classification system

Before providing any treatment, the PTs in this group will be informed about the results of the classification procedure. Patients will receive either direction-specific exercises, spinal manipulation, or stabilisation.

I. Direction-specific exercises

Patients will perform exercises in the direction that matches their directional preference: extension, flexion, lateral (side-gliding right or left) or flexion-rotation (right or left). For each direction is a treatment protocol which contains patient-generated forces (sustained positions and/or repeated movements) and PT-generated forces (mobilisations in the direction of preference). The protocol can be adapted to the patient's pain responses. All patients will receive a booklet containing information about directional preference and centralisation, the exercises, and the correct performance and frequency of each exercise. The patients will be instructed to perform the exercises four times a day, to correct their posture in the preferred direction, and to avoid activities and positions that increase the intensity or peripheralisation of symptoms. The patients will also be asked to keep an exercise logbook to monitor their compliance.

II. Spinal manipulation

Patients will be treated with high velocity thrusts directed at the spinal levels from T12 to L5 and/or the sacroiliac region. Decisions about the choice of the technique (long lever to manipulate multiple levels or short lever to manipulate one specific level) and the location of the forces are left to the discretion of the PTs. The most common spinal manipulation technique in the Netherlands is one of the two technique options in the Brennan et al. study [18], with the patient lying on his/her side. The symptomatic side is manipulated first. If the patient cannot specify a symptomatic side, the PT can select a side. During each session, the PT can make a maximum of four attempts (two on each side) to achieve a cavitation (i.e. 'a pop') heard or felt by the PT or the patient. Each session will end with 10 active flexion and extension movements of the lumbar spine in a supine or quadruped position within the limits of pain. The choice of this flexion-extension exercise will be made by the PT. Patients will be instructed to perform this active exercise once a day. The PT will record the number of attempts, whether or not a cavitation is heard or felt, and the technique that is used (long or short lever). The PT is allowed to provide spinal manipulation treatment for a maximum of 6 sessions.

III. Stabilisation

First, the patients will be taught to activate their local stability system to control neutral joint position in a supine position [29]. This (slight) co-activation of the lumbar multifidus with the transversus abdominus ('drawing in' the stomach) must be accompanied by minimal trunk movements and relaxation of global muscles such as the upper abdominal muscles. The patients will then be instructed to perform these abdominal bracing exercises ('drawing in' the stomach with normal breathing) during strengthening exercises in standing, quadruped, and side-support positions. Apart from these standardised exercises, activity of the local and global muscle systems will be trained in positions and movements that aggravate the patient's pain, in order to restore functional capacity and to improve dynamic control. The patients will also receive an instruction booklet, and will be instructed to perform the exercises once a day and to keep self-monitoring exercise diaries.

For all classification-based interventions, the treatment frequency, amount and type of exercises, and the duration and progression of the programme will be recorded, and directed by the judgment of the PT. The treatment frequency will only be restricted for spinal manipulation (maximum of 6 sessions). The patients will be treated according to their primary classification for a minimum of 4 weeks, during which the PT will be allowed to change the treatment strategy, if (1) the functionality of the patient improves significantly (a reduction of >30% on the ODI, compared to the score at baseline) or (2) the symptoms are severely aggravated. After the first 4 weeks, the PT will be allowed to provide follow-up interventions. At the end of each patient's treatment period, the PT will note the degree of satisfaction with the classification-based treatment, measured on a 5-point Likert scale ranging from very satisfied to very dissatisfied.

In both treatment groups, the patients are requested not to receive any co-interventions apart from the physical therapy (such as complementary and/or alternative medicine [CAM]) during the intervention period. All patients will be asked to stay active, and to engage in general exercise activities (e.g. a sports club). The principal investigator will regularly monitor the PTs for compliance with the treatment protocols, by means of visits, phone calls and e-mail contact.

Blinding

The PTs will be asked not to disclose the treatment allocation to the patients, and the success of the blinding will be evaluated at the 8-week follow-up. The research assistants and the participating statistician will be blinded for the patient's group assignment. The research PTs and the treating PTs cannot be blinded for the treatment allocation, but they will not be involved in the outcome date-collection.

Measurements

The baseline questionnaire will include questions about socio-demographic characteristics (e.g. age, gender, marital status, etc.) and all primary and secondary outcome measures.

Primary outcome measures

Global perceived recovery will be measured by self-assessment on a 7-point Likert scale ranging from "completely recovered" to "worse than ever" [30]. This will be dichotomised into success (complete and much recovered) and non-success (slightly recovered, no change, slightly worse, much worse and worse than ever). This measure has been found to be useful in LBP research [31].

Functional status will be measured according to the Dutch translation of the ODI, version 2.1a [32, 33]. The ODI is sub-divided into 10 sections related to activities of daily living. Each section consists of 6 graded responses, scored 0-5, and the total score can range from 0 (no difficulty at all) to 50 (maximal difficulty). The ODI is widely used in LBP studies, and has also been used as a primary outcome measure in studies focusing on the treatment-based classification approach [12, 18].

Pain intensity over the previous week will be measured on an 11-point numerical rating scale (0 = no pain to 10 = worst imaginable pain), with reliability, validity, and sensibility to change that are commonly accepted in LBP research [34–36].

Secondary outcome measures

General health status will be evaluated with the Short Form 36 (SF-36) [37]. It consists of 36 questions that can be aggregated to form eight sub-scales (physical functioning, mental health, general health perceptions, pain, role limitations physical, role limitations emotional, social functioning, and vitality) and two sum-scales (physical and mental component scales). The scores on all sub-scales range from 0-100, with higher scores indicating better outcomes. It is a widely used measurement instrument with satisfactory validity, reproducibility and responsiveness to change [38], and the Dutch translation has been found to be sufficiently valid [39].

Health-related quality of life will be measured with the EuroQol (EQ-5D) [40]. This questionnaire assesses 5 dimensions (mobility, self-care, usual activities, pain/discomfort and anxiety/depression) on a 3-point scale; no problems, moderate problems and severe problems. The questionnaire is appropriate for estimating quality-adjusted life-years (QALYs), and can be used for cost-utility analysis. The total score is expressed in utilities according to the Dolan model [41]. The questionnaire has been found to have adequate psychometric properties, and is commonly used in LBP research [42]. QALYs will be calculated by multiplying the utility of a health state by the time spent in this health state, based on the Dutch valuation tariff [43].

Fear-avoidance beliefs will be measured with the Dutch version of the Fear-Avoidance Beliefs Questionnaire (FABQ) [44]. It consists of two multi-item scales, measuring the patient's beliefs about how physical activity (FABQPA, 5 questions) and work (FABQW, 11 questions) affect their LBP. Following the recommendations of Waddell et al. [44], only 4 items will be included to score the FABQPA sub-scale (range 0-24) and 7 items to score the FABQW sub-scale (range 0-42). Higher scores indicate more fear-avoidance beliefs.

Psychosocial status will be measured with the Örebro Musculoskeletal Pain Screening Questionnaire (ÖMPSQ) [45]. This questionnaire contains 25 items, divided into five groups (function, pain, psychological factors, fear-avoidance, and miscellaneous), 21 of which are scored on a 0-10 scale. The scores of the items are summed to provide a total score range of 0-210, with higher scores indicating a higher risk of a poor outcome. The ÖMSPQ has been found to be a reliable and valid predictor of outcome for patients with acute, subacute and chronic LBP [46, 47].

Work absenteeism and presenteeism will be measured with the Productivity and Disease Questionnaire (PRODISQ) [48]. This questionnaire was developed and validated in samples of patients and workers in the Netherlands, and covers all relevant aspects of the relationship between health and productivity.

Direct and indirect costs will be measured by means of self-completed cost diaries [49]. The following costs will be evaluated: i) health care costs (e.g. primary care, medical specialist care, prescription of medication, professional home care and hospitalisation); ii) patient and family costs (out-of-pocket expenses, such as over-the-counter medication and the costs of paid and unpaid help); and iii) costs of loss of productivity (work absenteeism and presenteeism, and costs due to days of inactivity for patients without a paid job).

Table 1 presents an overview of the data-collection procedures.

Table 1 Overview of the data-collection

Sample size

Power calculations, based on the studies carried out by Brennan et al. [18] and Fritz et al. [12], were performed for the 3 main outcomes (for all: power 0.9; alpha 0.05). To detect a clinically relevant mean difference between the classification group and the usual care group of 9 points on the ODI (standard deviation [SD] 16), 55 patients are needed per group. To detect a clinically relevant mean difference of 2 points (SD 2) for pain (11-point numerical rating scale), 2 groups of 21 patients are needed. To detect a 20% difference in the dichotomised global perceived recovery (recovered vs. not recovered), 68 patients are needed per group. Anticipating a potential drop-out of 10%, 75 participants per treatment group (total n = 150) will be recruited.

Data-analysis

Baseline characteristics of the two intervention groups with regard to the most important prognostic indicators and main outcome measures will be compared to assess the adequacy of the randomisation. If necessary, adjustments will be made for baseline characteristics. The primary analysis will be an intention-to-treat analysis. A per protocol analysis will be performed to estimate the extent to which protocol deviations might have biased the results. The data will be analysed in a linear mixed model, with responses at 0 (baseline), 8, 26 and 52 weeks. In this model, the effect of interest is the time × treatment interaction. Estimates of treatment effects, with 95% confidence intervals between baseline and follow-up measurements, will be calculated and compared between the two treatment groups. Multivariable regression analysis will be used to adjust for possible baseline differences, and multilevel analysis will be performed, with PT, patient, and time of measurement as levels. Additionally, as a secondary analysis, a responder analysis will be performed according to the cut-off values as proposed by a consensus panel [50]. If there are many missing data, the missing data will be imputed with multiple imputation techniques.

Economic evaluation from a societal perspective

The cost-effectiveness analysis will be carried out from a societal perspective. The total back-pain-related costs for patients in both intervention groups will be compared with the health effects of the two groups, and cost-effectiveness ratios will be calculated for the primary outcome measures. The economic evaluation will be performed according to the intention-to-treat principle. Costs will be valued according to the guidelines published in the updated handbook for economic evaluation in the Netherlands [51]. The costs of production losses due to LBP will be estimated for both paid and unpaid labour [52]. For paid labour, the costs will be calculated, using both the human capital approach and the friction cost approach. For unpaid labour, the indirect costs will be estimated as the costs of production losses due to LBP, e.g. voluntary work and household work, based on shadow prices. The costs will be summed for each individual patient, and bootstrapping will be used to make a pair-wise comparison of the mean differences in total costs between the two groups. Confidence intervals will be obtained by bias-corrected and accelerated bootstrapping, using 2000 replications [53].

Cost-utility and cost-effectiveness ratios will be estimated, using bootstrapping techniques. The differences in costs will be compared, and related to the differences in effects (global perceived recovery, functional status and pain intensity) and differences in QALYs. Bootstrapped cost-effect pairs will be used to estimate cost-effectiveness planes and cost-effectiveness acceptability curves. Cost-effectiveness acceptability curves indicate the probability that a treatment is cost-effective at a specific ceiling ratio, which is the amount of money society is willing to pay to gain one extra unit of effect. Sensitivity analysis will be performed on the most important cost-drivers, in order to assess the robustness of the results.

Discussion

There is an increasing interest in classification systems for LBP patients that can be used to guide treatment. One of the classification systems that has potentials to improve the treatment decision-making process and clinical outcomes is a classification system that was originally proposed by Delitto et al. [8]. To critically examine and test the strength of this system, we designed an RCT, together with an economic evaluation to investigate whether treatment according to this system is more cost-effective than usual care in reducing pain and improving functioning in patients with subacute and chronic LBP in primary care.

Contrast between interventions

In both intervention groups personalised, individually tailored sessions will be given, and some of the same examination and treatment techniques will be used. However, some important features will account for the contrast between the two groups. Patients in the classification-based treatment group will be classified according to a standardised examination protocol, and will be treated according to standardised treatment protocols. In the usual care group however, the examination techniques are expected to vary greatly among the PTs [54], and the same examination findings can lead to a wide range of treatment goals and strategies.

Modifications

The classification system is not a static system, and some modifications have been made, based on new evidence [25], and based on the context in which physical therapy is provided within the Dutch health care system.

Patients with moderate complaints about one or more items of the Urogenital Distress Inventory (UDI 6, short form) will be excluded from the study. There are indications that patients with a combination of LBP and continence disorders require tailored treatment that is partially beyond the scope of the classification system [55]. The UDI 6 contains 6 questions on symptoms of irritation, stress, and obstruction/discomfort [56]. Patients can indicate whether a symptom is present, and if so, the degree to which it bothers them (not at all, slightly, moderately or greatly). The UDI 6 is a widely used, scientifically feasible questionnaire for urinary incontinence evaluation purposes, and has been translated and validated in Dutch [57].

Previous studies, carried out by Fritz et al. [12] and Brennan et al. [18], required that patients had at least a moderate level of disability due to LBP. Fritz et al. [12] required that patients had work restrictions, while Brennan et al. [18] required that patients had an ODI score >25%. In the present study, the level of disability is not an inclusion criterion, and therefore the participants may have less disability due to LBP than the participants in previous studies.

In the original approach developed by Delitto et al. [8], and in updated versions used by Fritz et al. [12] and Brennan et al. [18], centralisation was considered as the primary criterion for the classification of direction-specific exercises. In the present study however, directional preference has been recognised and implemented as a key examination finding for this classification [15, 25]. Directional preference is a broader construct than centralisation, and it is therefore expected that in the present study more patients will be classified for direction-specific exercises than in previous studies, in which centralisation was the primary examination criterion [12, 18].

The criteria for manipulation classification have been revised, based on the studies carried out by Flynn et al. [11] and Childs et al. [14], and in the present study four of these five criteria that they identified will be applied. The criterion of a recent onset of symptoms (<16 days) will be omitted in the present study, since the Dutch guidelines for general practitioners [58] and the multidisciplinary guidelines [59] discourage PT in the acute phase of LBP.

Patients who will be treated according to the manipulation treatment protocol, will be treated with manipulation techniques only, because recent research suggests that these patients may only be responsive to manipulation [11, 14, 60]. In previous studies, patients in this subgroup could also be treated with mobilisation techniques [12, 18]. There is no current evidence for the supremacy of one single manipulation technique [25], so the choice of the manipulation technique will be left to the discretion of the PTs.

No patients in the present study will be classified in the traction subgroup. Although this subgroup is likely to be quite small [19], the exclusion of this treatment may influence the effectiveness of the approach.

Finally, the classification system, which was originally developed for patients with acute low back pain [8], has undergone some validation studies for patients with subacute and chronic LBP [14, 15, 18, 20]. Researchers have suggested that it is more appropriate to subgroup and treat patients with chronic LBP according to psychosocial symptoms instead of the biomechanical signs and symptoms used in the classification system [61, 62]. However, although it is well recognised that some key psychosocial factors are overall predictors of outcome [63], there is limited and controversial evidence that these factors can indicate a specific type of treatment [64]. This study will make it possible to investigate the usefulness of the classification system for patients with more chronic LBP symptoms.

The results of this trial will be available in November 2010.

References

  1. Biering-Sorensen F: A prospective study of low back pain in a general population. I. Occurrence, recurrence and aetiology. Scand J Rehabil Med. 1983, 15: 71-79.

    CAS  PubMed  Google Scholar 

  2. van Tulder MW, Koes BW, Bouter LM: A cost-of-illness study of back pain in the Netherlands. Pain. 1995, 62: 233-240. 10.1016/0304-3959(94)00272-G.

    Article  CAS  PubMed  Google Scholar 

  3. Waddell G: Subgroups within "nonspecific" low back pain. J Rheumatol. 2005, 32: 395-396.

    PubMed  Google Scholar 

  4. Poitras S, Blais R, Swaine B, Rossignol M: Management of work-related low back pain: a population-based survey of physical therapists. Phys Ther. 2005, 85: 1168-1181.

    PubMed  Google Scholar 

  5. Bouter LM, Pennick V, Bombardier C: Cochrane back review group. Spine. 2003, 28: 1215-1218. 10.1097/00007632-200306150-00002.

    PubMed  Google Scholar 

  6. Borkan JM, Cherkin DC: An agenda for primary care research on low back pain. Spine. 1996, 21: 2880-2884. 10.1097/00007632-199612150-00019.

    Article  CAS  PubMed  Google Scholar 

  7. Billis EV, McCarthy CJ, Oldham JA: Subclassification of low back pain: a cross-country comparison. Eur Spine J. 2007, 16: 865-879. 10.1007/s00586-007-0313-2.

    Article  PubMed  PubMed Central  Google Scholar 

  8. Delitto A, Erhard RE, Bowling RW: A treatment-based classification approach to low back syndrome: identifying and staging patients for conservative treatment. Phys Ther. 1995, 75: 470-485.

    CAS  PubMed  Google Scholar 

  9. McKenzie R, May S: The lumbar spine. Mechanical diagnosis & therapy. 2003, Waikanae: Spinal Publications New Zealand Ltd, 2

    Google Scholar 

  10. Delitto A, Cibulka MT, Erhard RE, Bowling RW, Tenhula JA: Evidence for use of an extension-mobilization category in acute low back syndrome: a prescriptive validation pilot study. Phys Ther. 1993, 73: 216-222.

    CAS  PubMed  Google Scholar 

  11. Flynn T, Fritz J, Whitman J, Wainner R, Magel J, Rendeiro D, Butler B, Garber M, Allison S: A clinical prediction rule for classifying patients with low back pain who demonstrate short-term improvement with spinal manipulation. Spine. 2002, 27: 2835-2843. 10.1097/00007632-200212150-00021.

    Article  PubMed  Google Scholar 

  12. Fritz JM, Delitto A, Erhard RE: Comparison of classification-based physical therapy with therapy based on clinical practice guidelines for patients with acute low back pain: a randomized clinical trial. Spine. 2003, 28: 1363-1371. 10.1097/00007632-200307010-00003.

    PubMed  Google Scholar 

  13. Schenk R, Jozefczyk C, Kopf A: A randomized trial comparing interventions in patients with lumbar posterior drangement. J Man Manip Ther. 2003, 11: 95-102.

    Article  Google Scholar 

  14. Childs JD, Fritz JM, Flynn TW, Irrgang JJ, Johnson KK, Majkowski GR, Delitto A: A clinical prediction rule to identify patients with low back pain most likely to benefit from spinal manipulation: a validation study. Ann Intern Med. 2004, 141: 920-928.

    Article  PubMed  Google Scholar 

  15. Long A, Donelson R, Fung T: Does it matter which exercise? A randomized control trial of exercise for low back pain. Spine. 2004, 29: 2593-2602. 10.1097/01.brs.0000146464.23007.2a.

    Article  PubMed  Google Scholar 

  16. Hicks GE, Fritz JM, Delitto A, McGill SM: Preliminary development of a clinical prediction rule for determining which patients with low back pain will respond to a stabilization exercise program. Arch Phys Med Rehabil. 2005, 86: 1753-1762. 10.1016/j.apmr.2005.03.033.

    Article  PubMed  Google Scholar 

  17. Fritz JM, Piva SR, Childs JD: Accuracy of the clinical examination to predict radiographic instability of the lumbar spine. Eur Spine J. 2005, 14: 743-750. 10.1007/s00586-004-0803-4.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Brennan GP, Fritz JM, Hunter SJ, Thackeray A, Delitto A, Erhard RE: Identifying subgroups of patients with acute/subacute "nonspecific" low back pain: results of a randomized clinical trial. Spine. 2006, 31: 623-631. 10.1097/01.brs.0000202807.72292.a8.

    Article  PubMed  Google Scholar 

  19. Fritz JM, Lindsay W, Matheson JW, Brennan GP, Hunter SJ, Moffit SD, Swalberg A, Rodriquez B: Is there a subgroup of patients with low back pain likely to benefit from mechanical traction? Results of a randomized clinical trial and subgrouping analysis. Spine. 2007, 32: E793-E800. 10.1097/BRS.0b013e31815d001a.

    Article  PubMed  Google Scholar 

  20. Browder DA, Childs JD, Cleland JA, Fritz JM: Effectiveness of an extension-oriented treatment approach in a subgroup of subjects with low back pain: a randomized clinical trial. Phys Ther. 2007, 87: 1608-1618.

    Article  PubMed  Google Scholar 

  21. Fritz JM, George S: The use of a classification approach to identify subgroups of patients with acute low back pain. Interrater reliability and short-term treatment outcomes. Spine. 2000, 25: 106-114. 10.1097/00007632-200001010-00018.

    Article  CAS  PubMed  Google Scholar 

  22. Kiesel KB, Underwood FB, Mattacola CG, Nitz AJ, Malone TR: A comparison of select trunk muscle thickness change between subjects with low back pain classified in the treatment-based classification system and asymptomatic controls. J Orthop Sports Phys Ther. 2007, 37: 596-607.

    Article  PubMed  Google Scholar 

  23. Heiss DG, Fitch DS, Fritz JM, Sanchez WJ, Roberts KE, Buford JA: The interrater reliability among physical therapists newly trained in a classification system for acute low back pain. J Orthop Sports Phys Ther. 2004, 34: 430-439.

    Article  PubMed  Google Scholar 

  24. Fritz JM, Brennan GP, Clifford SN, Hunter SJ, Thackeray A: An examination of the reliability of a classification algorithm for subgrouping patients with low back pain. Spine. 2006, 31: 77-82. 10.1097/01.brs.0000193898.14803.8a.

    Article  PubMed  Google Scholar 

  25. Fritz JM, Cleland JA, Childs JD: Subgrouping patients with low back pain: evolution of a classification approach to physical therapy. J Orthop Sports Phys Ther. 2007, 37: 290-302.

    Article  PubMed  Google Scholar 

  26. Donelson R, Grant W, Kamps C, Medcalf R: Pain response to sagittal end-range spinal motion. A prospective, randomized, multicentered trial. Spine. 1991, 16: S206-S212. 10.1097/00007632-199106001-00006.

    Article  CAS  PubMed  Google Scholar 

  27. Heijmans WFGL, Hendriks HJM, Esch van der M, Pool-Goudwaard A, Scholten GGM, van Tulder MW, de Wijer A, Oostendorp RAB: KNGF-richtlijn manuele therapie bij lage-rupijn. [National practice guidelines for manual therapy in patients with low back pain]. Ned Tijdschr Fysiother. 2003, 113: 1-12.

    Google Scholar 

  28. Bekkering GE, Hendriks HJM, Koes BW, Oostendorp RAB, Ostelo RWJG, Thomassen JMC, van Tulder MW: KNGF-richtlijn fysiotherapie bij lage-rupijn [National practice guidelines for physical therapy in patients with low back pain]. Ned Tijdschr Fysiother. 2005, 115: 1-32.

    Google Scholar 

  29. Jull GA, Richardson CA: Motor control problems in patients with spinal pain: a new direction for therapeutic exercise. J Manipulative Physiol Ther. 2000, 23: 115-117. 10.1016/S0161-4754(00)90079-4.

    Article  CAS  PubMed  Google Scholar 

  30. Feinstein AR: Clinimetrics. 1987, New Haven and London: Yale University Press

    Google Scholar 

  31. Bombardier C: Outcome assessments in the evaluation of treatment of spinal disorders: summary and general recommendations. Spine. 2000, 25: 3100-3103. 10.1097/00007632-200012150-00003.

    Article  CAS  PubMed  Google Scholar 

  32. Fairbank JC, Pynsent PB: The Oswestry Disability Index. Spine. 2000, 25: 2940-2952. 10.1097/00007632-200011150-00017.

    Article  CAS  PubMed  Google Scholar 

  33. Fairbank JC: Use and abuse of Oswestry Disability Index. Spine. 2007, 32: 2787-2789. 10.1097/BRS.0b013e31815b9e9d.

    Article  PubMed  Google Scholar 

  34. Carlsson AM: Assessment of chronic pain. I. Aspects of the reliability and validity of the visual analogue scale. Pain. 1983, 16: 87-101. 10.1016/0304-3959(83)90088-X.

    Article  CAS  PubMed  Google Scholar 

  35. Revill SI, Robinson JO, Rosen M, Hogg MI: The reliability of a linear analogue for evaluating pain. Anaesthesia. 1976, 31: 1191-1198. 10.1111/j.1365-2044.1976.tb11971.x.

    Article  CAS  PubMed  Google Scholar 

  36. Sriwatanakul K, Kelvie W, Lasagna L, Calimlim JF, Weis OF, Mehta G: Studies with different types of visual analog scales for measurement of pain. Clin Pharmacol Ther. 1983, 34: 234-239.

    Article  CAS  PubMed  Google Scholar 

  37. McHorney CA, Ware JE, Raczek AE: The MOS 36-Item Short-Form Health Survey (SF-36): II. Psychometric and clinical tests of validity in measuring physical and mental health constructs. Med Care. 1993, 31: 247-263. 10.1097/00005650-199303000-00006.

    Article  CAS  PubMed  Google Scholar 

  38. Wittink H, Turk DC, Carr DB, Sukiennik A, Rogers W: Comparison of the redundancy, reliability, and responsiveness to change among SF-36, Oswestry Disability Index, and Multidimensional Pain Inventory. Clin J Pain. 2004, 20: 133-142. 10.1097/00002508-200405000-00002.

    Article  PubMed  Google Scholar 

  39. Zee van der K, Sanderman R, Heyink J: De psychometrische kwaliteiten van de MOS 36-item Short Form Health Survey (SF-36) in een Nederlandse populatie [The psychometric properties of the SF-36 in a Dutch population]. Tijdschr Soc Gez. 1993, 71: 183-191.

    Google Scholar 

  40. EuroQol--a new facility for the measurement of health-related quality of life. The EuroQol Group. Health Policy. 1990, 16: 199-208. 10.1016/0168-8510(90)90421-9.

  41. Dolan P: Modeling valuations for EuroQol health states. Med Care. 1997, 35: 1095-1108. 10.1097/00005650-199711000-00002.

    Article  CAS  PubMed  Google Scholar 

  42. Deyo RA, Battie M, Beurskens AJ, Bombardier C, Croft P, Koes B, Malmivaara A, Roland M, von Korff M, Waddell G: Outcome measures for low back pain research. A proposal for standardized use. Spine. 1998, 23: 2003-2013. 10.1097/00007632-199809150-00018.

    Article  CAS  PubMed  Google Scholar 

  43. Lamers LM, Stalmeier PF, McDonnell J, Krabbe PF, van Busschbach JJ: Kwaliteit van leven meten in economische evaluaties: het Nederlands EQ-5D-tarief [Measuring the quality of life in economic evaluations: the Dutch EQ-5D tariff]. Ned Tijdschr Geneeskd. 2005, 149: 1574-1578.

    CAS  PubMed  Google Scholar 

  44. Waddell G, Newton M, Henderson I, Somerville D, Main CJ: A Fear-Avoidance Beliefs Questionnaire (FABQ) and the role of fear-avoidance beliefs in chronic low back pain and disability. Pain. 1993, 52: 157-168. 10.1016/0304-3959(93)90127-B.

    Article  CAS  PubMed  Google Scholar 

  45. Linton SJ, Hallden K: Can we screen for problematic back pain? A screening questionnaire for predicting outcome in acute and subacute back pain. Clin J Pain. 1998, 14: 209-215. 10.1097/00002508-199809000-00007.

    Article  CAS  PubMed  Google Scholar 

  46. Westman A, Linton SJ, Ohrvik J, Wahlen P, Leppert J: Do psychosocial factors predict disability and health at a 3-year follow-up for patients with non-acute musculoskeletal pain? A validation of the Örebro Musculoskeletal Pain Screening Questionnaire. Eur J Pain. 2008, 12: 641-649. 10.1016/j.ejpain.2007.10.007.

    Article  CAS  PubMed  Google Scholar 

  47. Grotle M, Vollestad NK, Brox JI: Screening for yellow flags in first-time acute low back pain: reliability and validity of a Norwegian version of the Acute Low Back Pain Screening Questionnaire. Clin J Pain. 2006, 22: 458-467. 10.1097/01.ajp.0000208243.33498.cb.

    Article  PubMed  Google Scholar 

  48. Koopmanschap MA: PRODISQ: a modular questionnaire on productivity and disease for economic evaluation studies. Expert Rev Pharmacoecon Outcomes Res. 2005, 5: 23-28. 10.1586/14737167.5.1.23.

    Article  PubMed  Google Scholar 

  49. Goossens ME, Rutten-van Molken MP, Vlaeyen JW, Linden van der SM: The cost diary: a method to measure direct and indirect costs in cost-effectiveness research. J Clin Epidemiol. 2000, 53: 688-695. 10.1016/S0895-4356(99)00177-8.

    Article  CAS  PubMed  Google Scholar 

  50. Ostelo RW, Deyo RA, Stratford P, Waddell G, Croft P, Von KM, Bouter LM, de Vet HC: Interpreting change scores for pain and functional status in low back pain: towards international consensus regarding minimal important change. Spine. 2008, 33: 90-94. 10.1097/BRS.0b013e31815e3a10.

    Article  PubMed  Google Scholar 

  51. Oostenbrink JB, Boumans CAM, Koopmanschap MA, Rutten FFH: Handleiding voor kostenonderzoek: Methoden en standaard kostprijzen voor economische evaluaties in de gezondheidszorg, geactualiseerde versie [Manual for costing: Methods and standard costs for economic evaluations in health care, updated version]. 2004, Diemen: College voor zorgverzekeringen

    Google Scholar 

  52. Koopmanschap MA, Rutten FF: A practical guide for calculating indirect costs of disease. Pharmacoeconomics. 1996, 10: 460-466. 10.2165/00019053-199610050-00003.

    Article  CAS  PubMed  Google Scholar 

  53. Efron B, Tibshirani R: An Introduction to the Bootstrap. 1993, London: Chapman & Hall

    Book  Google Scholar 

  54. Foster NE, Thompson KA, Baxter GD, Allen JM: Management of nonspecific low back pain by physiotherapists in Britain and Ireland. A descriptive questionnaire of current clinical practice. Spine. 1999, 24: 1332-1342. 10.1097/00007632-199907010-00011.

    Article  CAS  PubMed  Google Scholar 

  55. Grewar H, McLean L: The integrated continence system: a manual therapy approach to the treatment of stress urinary incontinence. Man Ther. 2008, 13: 375-386. 10.1016/j.math.2008.01.003.

    Article  PubMed  Google Scholar 

  56. Uebersax JS, Wyman JF, Shumaker SA, McClish DK, Fantl JA: Short forms to assess life quality and symptom distress for urinary incontinence in women: the Incontinence Impact Questionnaire and the Urogenital Distress Inventory. Continence Program for Women Research Group. Neurourol Urodyn. 1995, 14: 131-139. 10.1002/nau.1930140206.

    Article  CAS  PubMed  Google Scholar 

  57. Vaart van der CH, de LJR, Roovers JP, Heintz AP: De invloed van urine-incontinentie op de kwaliteit van leven bij thuiswonende Nederlandse vrouwen van 45-70 jaar [The influence of urinary incontinence on the quality of life in community-dwelling, 45-70 year-old women in the Netherlands]. Ned Tijdschr Geneeskd. 2000, 144: 894-897.

    PubMed  Google Scholar 

  58. Chavannes AW, Mens JMA, Koes BW, Lubbers WJ, Ostelo R, Spinnewijn WEM, Kolnaar BGM: NHG-Standaard aspecifieke lagerugpijn (Eerste herziening) [Dutch College of General Practitioners' practice guideline low back pain (first update)]. Huisarts Wet. 2005, 48: 113-123.

    Google Scholar 

  59. Kwaliteitsinstituut voor de gezondheidszorg CBO: Richtlijn aspecifieke lage rugklachten [The Dutch Institute for Health Care Improvement: Guideline aspecific low back complaints]. 2002, Alphen aan de Rijn: van Zuiden Communications

    Google Scholar 

  60. Hancock MJ, Maher CG, Latimer J, Herbert RD, McAuley JH: Independent evaluation of a clinical prediction rule for spinal manipulative therapy: a randomised controlled trial. Eur Spine J. 2008, 17: 936-943. 10.1007/s00586-008-0679-9.

    Article  PubMed  PubMed Central  Google Scholar 

  61. Burton AK, Tillotson KM, Main CJ, Hollis S: Psychosocial predictors of outcome in acute and subchronic low back trouble. Spine. 1995, 20: 722-728. 10.1097/00007632-199503150-00014.

    Article  CAS  PubMed  Google Scholar 

  62. Gatchel RJ, Turk DC: Criticisms of the biopsychosocial model in spine care: creating and then attacking a straw person. Spine. 2008, 33: 2831-2836. 10.1097/BRS.0b013e31817d24ad.

    Article  PubMed  Google Scholar 

  63. Pincus T, Burton AK, Vogel S, Field AP: A systematic review of psychological factors as predictors of chronicity/disability in prospective cohorts of low back pain. Spine. 2002, 27: E109-E120. 10.1097/00007632-200203010-00017.

    Article  PubMed  Google Scholar 

  64. Smeets RJ, Maher CG, Nicholas MK, Refshauge KM, Herbert RD: Do psychological characteristics predict response to exercise and advice for subacute low back pain?. Arthritis Rheum. 2009, 61: 1202-1209. 10.1002/art.24731.

    Article  PubMed  Google Scholar 

Pre-publication history

Download references

Acknowledgements

The study received a grant from the Netherlands Organisation for Health Research and Development (ZonMw, project no:170882401).

The authors wish to thank Mark Hancock for his comments on an earlier draft, and all the PTs and patients for their participation in this project.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Adri T Apeldoorn.

Additional information

Competing interests

The authors declare that they have no competing interests.

Authors' contributions

AA was responsible for developing, translating and implementing the physical examination and treatment protocols, and the preparation of the manuscript. RWO, MWvT and HCWdV were responsible for the study design, the acquisition of funding for the study and the supervising of the protocols. HvH participated in establishing the physical examination and treatment protocols, and instructed the participating physical therapists in direction-specific exercises. JMF contributed to the design of the study, and provided the treatment protocols. All authors have read, revised and approved the final manuscript.

Electronic supplementary material

12891_2010_804_MOESM1_ESM.DOC

Additional file 1: Table S1. Treatment-based classification studies. It contains relevant publications of the treatment-based classification system that we are investigating. (DOC 64 KB)

12891_2010_804_MOESM2_ESM.DOC

Additional file 2:Operational definitions for the physical measurements (= Appendix). It contains the tests that we will use to classify the patients. (DOC 42 KB)

Authors’ original submitted files for images

Below are the links to the authors’ original submitted files for images.

Authors’ original file for figure 1

Authors’ original file for figure 2

Rights and permissions

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions

About this article

Cite this article

Apeldoorn, A.T., Ostelo, R.W., van Helvoirt, H. et al. The cost-effectiveness of a treatment-based classification system for low back pain: design of a randomised controlled trial and economic evaluation. BMC Musculoskelet Disord 11, 58 (2010). https://doi.org/10.1186/1471-2474-11-58

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/1471-2474-11-58

Keywords