Skip to main content
Download PDF
  • Research article
  • Open access
  • Published:

Pilot randomized controlled trials in the orthopaedic surgery literature: a systematic review

BMC Musculoskeletal Disorders volume 19, Article number: 412 (2018) Cite this article

Abstract

Background

The primary objective of this systematic review is to examine the characteristics of pilot randomized controlled trials (RCTs) in the orthopaedic surgery literature, including the proportion framed as feasibility trials and those that lead to definitive RCTs. This review aim to answer the question of whether pilot RCTs lead to definitive RCTs, whilst investigating the quality, feasibility and overall publication trends of orthopaedic pilot trials.

Methods

Pilot RCTs in the orthopaedic literature were identified from three electronic databases (EMBASE, MEDLINE, and Pubmed) searched from database inception to January 2018. Search criteria included the evaluation of at least one orthopaedic surgical intervention, research on humans, and publication in English. Two reviewers independently screened the pool of pilot trials, and conducted a search for corresponding definitive trials. Screened pilot RCTs were assessed for feasibility outcomes related to efficiency, cost, and/or timeliness of a large-scale clinical trial involving a surgical intervention. The quality of the pilot and definitive trials were assessed using the Checklist to Evaluate a Report of a Non-Pharmacological Trial (CLEAR NPT).

Results

The initial search for pilot RCTs yielded 3857 titles, of which 49 articles were relevant for this review. 73.5% (36/49) of the orthopaedic pilot RCTs were framed as feasibility trials. Of these, 5 corresponding definitive trials (10.2%) were found, of which four were published and one ongoing. Based on author responses, the lack of a definitive RCT following the pilot trial was attributed to a lack of funding, inadequacies in recruitment, and belief that the pilot RCT sufficiently answered the research question.

Conclusions

Based on this systematic review, most pilot RCTs were characterized as feasibility trials. However, the majority of published pilot RCTs did not lead to definitive trials. This discrepancy was mainly attributed to poor feasibility (e.g. poor recruitment) and lack of funding for an orthopaedic surgical definitive trial. In recent years this discrepancy may be due to researchers saving on time and cost by rolling their pilot patients into the definitive RCT rather than publish a separate pilot trial.

Peer Review reports

Background

Definitive randomized controlled trials (RCTs) exist to demonstrate unmistakable evidence of a certain inventions benefit on a patient [1]. Although they are very impactful for clinical practice are typically expensive and time-consuming [2]. Given the resources and time, investigators often conduct pilot trials designed with an aim to demonstrate the feasibility of the larger-scale definitive trial [3]. Pilot trials can identify possible challenges, predict costs, and fine-tune study design. In addition, by demonstrating feasibility, a successful pilot trial can be used to leverage momentum and definitive trial funding [2].

Effective pilot trials have a well-defined set of objectives to assess feasibility.3 Feasibility is assessed in terms of whether the intervention of interest, trial design, and protocol can be successfully implemented and completed by the researchers [3]. Feasibility can be determined at the program level, study level, and site or investigator level. Program level feasibilities include determining the prevalence of particular diseases in a particular region and include clinical and epidemiological trials [4]. Study level feasibilities are centered on assessing whether a specific clinical trial can be conducted in a country or region [3]. Site or investigator level feasibility trials focus on identifying challenges and probable solutions with respect to the investigator and clinical aspects of the trial (drug dosages, actual study population, recruitment and follow-up, usage of assessment tools, etc.) [4].

Despite the benefits of pilot trials, previous literature has demonstrated that they do not always lead to a definitive trial. In 2004, Lancaster et al reviewed four general medicine journals and three specialist journals and identified 90 pilot studies published from 2000 to 2001; of which 45 reported the intention to carry out further work [1]. However, in 2010, Arain et al. found that only eight out of the 45 were followed by a larger, definitive study [5]. The impact of pilot data and subsequent research remains to be evaluated in the orthopaedic surgical literature.

This systematic review assessed the quality of pilot RCTs and frequency of ensuing definitive RCTs in the orthopaedic surgical literature. The primary objectives of this review were to: 1) assess feasibility outcomes across pilot trials in the orthopaedic surgery literature; 2) identify the proportion of pilot trials that lead to and how they inform definitive RCTs, and 3) evaluate the quality and frequency of pilot trials over time.

Methods

Identification of RCTs

EMBASE, MEDLINE and Pubmed were searched for relevant articles published from database inception until January 25, 2018 (Additional file 1). All search results were imported into the Mendeley Reference Manager software (Elsevier Publishing, 2013) to remove all duplicate trials.

Once the final pool of included pilot RCTs was determined, an additional search was conducted in the same electronic databases in an attempt to find corresponding definitive trials. If a literature search of titles was unsuccessful, other trials conducted by at least one of the authors after the pilot were considered. The secondary search was conducted using key terms used in the pilot RCT. Additionally, clinicaltrials.gov, an online database of ongoing clinical trials, was reviewed to determine if the previously identified pilot RCTs had a definitive trial in progress. Finally, if no definitive trial was found using these methods, the pilot RCT authors were contacted by email and asked whether a definitive trials was ongoing, published, or submitted for publication.

Eligibility criteria

Trails had to be defined explicitly and reported as pilot trials within the paper itself to be included in this review. Trials reported as pilot RCTs were deemed eligible for this review if they: 1) included an orthopaedic surgical intervention, 2) included a drug that was used intra-operatively at the site of surgery/fracture, or 3) evaluated the difference between two surgical interventions or surgical vs. non-surgical orthopaedic interventions. Only clinical trials in humans published in English were included. RCTs were excluded if they were: 1) non-pilot RCT designs (including small trials not reported as pilot trials) 2) trial interventions were exclusively non-surgical including physiotherapy, exercise regimens, post-operative rehabilitation, anesthesia, post-operative pain management interventions, or 3) trial interventions were surgical procedures not related to orthopaedics (e.g. oral, urology, and ocular surgeries), and 4) drugs and supplements administered orally (intravenously administered during surgery were included).

Screening

Articles were independently screened in duplicate at the title, abstract, and full-text stage by decisions were independently recorded in a spreadsheet (Microsoft Excel, 2015). In order to ensure comprehensive screening, an article was progressed to the next screening stage if at least one reviewer had noted that the article should be included, and illustrated as a flow diagram and checklist in Fig. 1 below. All disagreements were resolved by consensus during the full-text screening phase in consultation with a third senior reviewer (AD).

Fig. 1
figure 1

Flow diagram of the search and screening strategies to define the final pool of trails

Full size image

Data abstraction

Pilot and definitive trial data, such as the country, number of patients in the RCT, orthopaedic condition being treated, orthopaedic intervention(s), controls, primary and secondary outcomes, percentage of patients that were lost to follow-up, follow-up schedule, and feasibility objectives were abstracted. In addition, for the definitive trials, any changes in interventions, controls, primary and secondary outcomes, or patient sample from the pilot trial were noted. For definitive RCTs, the time elapsed between the date of publication of the pilot and definitive trial and whether or not the sample size was calculated based on event rates from the pilot trial were determined.

Assessment of feasibility

Feasibility trials were defined as trials with a primary purpose of piloting the protocol to inform a definitive trial. In order to distinguish between pilot trials created solely for investigating the efficacy of interventions compared to feasibility trials, specific reference to feasibility objectives were evaluated. Feasibility objectives include determining the preliminary efficacy of a surgical intervention as well as the safety of the intervention, accurate event rates for a definitive sample size, the cost of a large scale clinical trial, patient recruitment rates, trial design, randomization procedure, and ability to maintain blinding.

Assessment of methodological quality

The reviewers (BD, VD, ALS, and SS) independently assessed the quality of each included pilot and definitive trial in duplicate using the Checklist to Evaluate A Report of Non-Pharmacological Trial (CLEAR NPT). The CLEARN NPT is designed for the critical appraisal of RCTs in nonpharmacological and surgical trials [6]. The original checklist was modified, where the question regarding patient adherence was omitted as all our trials evaluated a single, one-time surgical intervention. As the original checklist did not provide a scoring method, the criteria employed by Somford et al. was adopted to provide a modified CLEAR NPT (Additional file 2) [6]. The maximum CLEAR NPT score was 18, whereby a score of 0–6 indicated a low quality trial, 7–12 indicated a medium quality trial, and 13–18 indicated a high quality trial.

Statistical analysis

A kappa (κ) statistic was used to determine agreement at all stages of article screening with 95% confidence intervals (CI) [7]. An intraclass correlation coefficient (ICC) was calculated for the purpose of evaluating inter-rater reliability for the CLEAR NPT quality assessment. Agreement for both the κ and ICC was categorized as follows: > 0.90 indicated an almost perfect level of agreement, 0.80 < 0.90 strong agreement, 0.60 < 0.79 moderate agreement, 0.40 < 0.59 weak agreement, 0.21 < 0.39 minimal agreement, and 0.0 < 0.20 no agreement [8].

A t-test was performed using an online statistical calculator (Vassal Stats) to compare trial quality between pilot and definitive trials and a Pearson’s r correlation was calculated to determine if there was a relationship between number of studies and quality of pilot RCTs over time. A p-value less than 0.05 was considered significant. Descriptive statistics including means, proportions, standard deviations, and CIs are reported. A meta-analysis was not performed given the broad heterogeneity of the trial designs, interventions, and outcome measures.

Results

Screening

The initial screening of online databases yielded 3857 articles after the removal of 2230 duplicates. After title, abstract, and full text screening, 49 pilot RCTs were included (Fig. 1 and Additional file 3). Of these, we identified five definitive trials (one of which is still ongoing) that corresponded to the original published pilot trial. Inter-reviewer agreement was high at all stages of screening (title, κ = 0.886 (95% CI 0.878 to 0.893); abstract, κ = 0.740 (95% CI 0.693 to 0.780); and full text, κ = 0.792 (95% CI 0.737 to 0.835)).

Pilot trial characteristics

Pilot trials were commonly published from the UK and Canada (22 and 16%, respectively) (Tables 1 and 2). A total of 2117 patients were recruited across all pilot RCTs, and 5.84 ± 10.9% of patients, on average, were lost to follow up. The greatest proportion of pilot trials (59.2%, 29/49) focused on surgical fracture repair, including long bone, knee, spinal, foot and hip fractures (Fig. 2). As classified by the World Bank, 40 of the pilot trails were conducted in high-income countries, 6 were classified as middle income and 3 were classified as low income [9].

Table 1 Characteristics of the Included Pilot RCTs
Full size table
Table 2 Characteristics of Included Definitive trials
Full size table
Fig. 2
figure 2

Frequency of various types of pilot RCT interventions in each intervention category

Full size image

Primary and secondary outcomes of the pilot trials were divided into physician-reported and patient-reported outcomes. 65.3% (32/49) of all pilot RCTs used radiographic analysis, such as x-rays, MRIs, ultrasounds and CT scans. Patient-reported outcomes were recorded through self-reporting or interview style questionnaires. Questionnaires addressed outcomes such as quality of life, pain, function/independence, and emotional health. Of the pilot trials, 67.3% (33/49) made use of patient-reported questionnaires as tools for monitoring trial outcomes.

Overall, 73.5% (36/49) of the pilot RCTs found in the orthopaedic surgery literature were framed as feasibility trials (Table 1). The two most commonly explored feasibility objectives were safety and efficacy of an orthopaedic surgical intervention (Fig. 3). 26.5% (13/49) of pilot RCTs explored more than one feasibility objective. The pilot trials CLEAR NPT rating varied from 10 to 18. Only 3 of the 5 definitive RCTs included in this review determined their sample size based on their corresponding pilot trial. None of the definitive RCTs enrolled the pilot patients into the definitive trial. Additionally, 22.4% (11/49) of the pilot trials listed the efficacy/effectiveness of the surgical intervention as a primary outcome. Of these, only one led to a definitive trial.

Fig. 3
figure 3

Number of RCTs that define each of these feasibility objectives in their pilot RCT

Full size image

Definitive trial characteristics

Of the 49 identified pilot RCTs, five (10.2%) corresponding definitive RCTs were found (Table 2). On average, definitive trials were published at a mean of 4.25 years (3–7 years) after the pilot trial. The sample size of the pilot trial was 7.2% of the definitive trials. The total number of patients recruited to definitive RCTs was 4016, with one trial still recruiting participants (Table 2). Only one of these definitive trials was ongoing according to clinicaltrials.gov (31). Authors from 17 pilot trials (34.7%) responded to our email confirming that a definitive trial had not been published. Of these, 8 authors cited the following reasons for not conducting a definitive RCT: a lack of funding (12.5%), inability to meet recruitment targets (12.5%), preliminary efficacy of the intervention was not demonstrated (25.0%), the pilot study was thought to yield reliable results therefore eliminating the need for further investigation (50.0%).

Trial quality

There was no correlation (r = − 0.1508, p = 0.5655) between number of studies and quality of pilot RCTs over time (Table 3). The overall quality of the pilot RCTs was relatively high (mean CLEAR NPT score 15.9 ± 1.53). Based on the CLEAR NPT scale, the highest quality pilot RCTs involved the treatment of arthrodesis and repair of knee fractures. All of the definitive RCTs were given a score of 18 and were therefore 2.6 points higher on the CLEAR NPT scale than their corresponding pilot trials (p < 0.01). The agreement among reviewers for the quality assessment was very high (ICC = 0.969 (95% CI 0.948 to 0.982)).

Table 3 Number and average quality of pilot RCTs over time of publication
Full size table

Discussion

Results from this systematic review demonstrate that the majority of orthopaedic surgical pilot RCTs were framed as feasibility trials, and that the pilot trials mostly evaluated site or investigator level feasibility. As expected, the quality of the corresponding definitive RCTs was higher than their respective pilot trial. Despite the majority (87%) of pilot RCTs being conducted in the high-income countries, the majority of the included pilot trials however, did not lead to a definitive RCT. In these cases, reasons cited included: a lack of funding, inadequate sample sizes, and that research questions were sufficiently answered in the pilot phase.

Similar to other fields of medicine, the majority of orthopaedic surgical pilot trials were not followed by a definitive trial. Arain et al. reviewed seven medical journals, including four general medicine journals (British Medical Journal, Lancet, the New England Journal of Medicine and the Journal of American Medical Association) and three specialist journals (British Journal of Surgery, British Journal of Cancer, British Journal of Obstetrics and Gynecology) to identify 54 pilot studies [5]. The authors reported a very low number of follow up studies, wherein only 14.8% (8/54) pilot studies yielded published definitive studies. Additionally, a systematic review published in 2017 by Kaur et al., looked at the quality of pilot studies within the Clinical Rehabilitation journal over the past 30 years, and they concluded that only 12% of their pilot studies led to a definitive trial [10].

The limited number of published pilot trials and corresponding definitive trials may be attributed to numerous factors. Firstly, the pilot may have demonstrated that a definitive trial was not feasible based on criteria established a priori (e.g. ability to recruit patients). However, we would expect that in some of these cases, researchers would amend their trial design, interventions, and outcomes to ensure feasibility in the definitive trial. Secondly, if found to be feasible, investigators may refrain from publishing their pilot trial and instead, roll the pilot patients into the definitive RCT to help save on time and costs. Trial methods papers and online registries are often used to first describe these trials. Thirdly, based on author responses in this review, definitive trials may not be feasible due to a lack of funding. In one case, the authors noted that their research question was answered by the pilot trial [11]. However, the published pilot did not provide a sample size calculation, and therefore, we cannot determine if the statistical power threshold was met for the primary outcome [12].

The majority of the orthopaedic surgical pilot trials found in this review posited feasibility objectives and were of relatively high quality. The first published pilot surgical trial was found in 1996, and since then, there has been an increase in the number of pilot RCTs published over time, with a relatively constant quality of trials up until 2013, with a decline in publications up until 2016. From 2016 to the end of our search in 2018, there were no orthopaedic surgical pilot RCTs published. This may be due to a more recent trend of trialists to roll their pilot patients into a definitive trial to save on costs and maximize recruitment. There may also be a lag in pilot publications in the past 3 years.

Strengths and limitations

Strengths of this review include a broad systematic search and high agreement at all stages of screening and quality assessment. The main limitation is the minimal data available regarding the reasons why pilot trials have not led to definitive RCTs. There was a lack of response from authors, limiting further insight into barriers to definitive trials. Within the past 5 years, 13 of the 49 pilot RCTs and 4 of the 5 definitive trials were published. Thus, the inclusion of more recent pilot RCTs may be a limitation, as their current definitive trials may be underway, and/or not yet published. This potential source of bias was mitigated by searching the clinical trial registry, clinicaltrials.gov, for any records of ongoing definitive RCTs.

This review includes the use of the CLEAR NPT checklist to evaluate each pilot trial. Specifically within orthopedic literature, the quality of reporting RCTs using the CLEAR NPT is suboptimal, and that there is a need for improved surgical reporting [13]. However, in comparison to the CONSORT statement, the CLEAR NPT scale proves to be more useful in its analysis in interventions that require technical skill, with unique considerations in both conducting and reporting trials [14]. In this review, to account for methodological considerations, a modified CLEAR NPT scale was used instead to increase reliability and remove the necessity of including the Cochrane Risk of Bias Tool. The CLEAR NPT scale was modified, tested and optimized for orthopaedic trials, which was the focus of this paper.

Conclusion

While the majority of pilot RCTs found in the surgical orthopaedic literature are framed as feasibility trials, most did not lead to definitive trials. The reported reasons include: minimal funding, the inability to recruit an adequate sample size and that the research questions were sufficiently answered in the pilot phase. Although, most pilot RCTs did not result in a definitive trial, this does not diminish the value of the pilot trial in determining feasibility.

Abbreviations

CI:

Confidence intervals

CLEAR NPT:

Checklist to Evaluate A Report of Non-Pharmacological Trial

ICC:

Intraclass correlation coefficient

n/a:

Not applicable

RCT:

Randomized controlled trial

References

  1. Hujoel P. Definitive vs. exploratory periodontal trials: a survey of published studies. J Dent Res. 1995;74(8):1453–8.

    Article  CAS  Google Scholar 

  2. Lancaster GA, Dodd S, Williamson PR. Design and analysis of pilot studies: recommendations for good practice. J Eval Clin Pract. 2004;10(2):307–12.

    Article  Google Scholar 

  3. Feeley N, Cossette S, Côté J, Héon M, Stremler R, Martorella G, et al. The importance of piloting an RCT intervention. Can J Nurs Res. 2009;41(2):84–99.

    Google Scholar 

  4. Rajadhyaksha V. Conducting feasibilities in clinical trials: an investment to ensure a good study. Perspect Clin Res. 2010;1(3):106–9.

    PubMed  PubMed Central  Google Scholar 

  5. Arain M, Campbell MJ, Cooper CL, Lancaster GA. What is a pilot or feasibility study? A review of current practice and editorial policy. BMC Med Res Methodol. 2010;10(1):67.

    Article  Google Scholar 

  6. Somford MP, Van Deurzen DF, Ostendorf M, Eygendaal D, Van den Bekerom MP. Quality of research and quality of reporting in elbow surgery trials. J Shoulder Elb Surg. 2015;24(10):1619–26.

    Article  Google Scholar 

  7. McGinn T, Wyer P, Newman T, Keitz S, Leipzig R. Tips for learners of evidence-based medicine: 3. Measures of observer variability (kappa statistic). CMAJ. 2004;171(11):1369–73.

    Article  Google Scholar 

  8. Sackett DL, Haynes RB, Guyatt GH, Tugwell P. Clinical Epidemiology. 2nd ed. Boston: Little, Brown and Company; 1991.

    Google Scholar 

  9. How does the World Bank classify countries. The World Bank. 2018.

  10. Kaur N, Figueiredo S, Bouchard V, Moriello C, Mayo N. Where have all the pilot studies gone? A follow-up on 30 years of pilot studies in clinical rehabilitation. Clin Rehabil. 2017;31(9):1238–48.

    Article  Google Scholar 

  11. De Sèze MP, Bonhomme C, Daviet JC, Burguete E, Machat H, Rousseaux M, et al. Effect of early compensation of distal motor deficiency by the chignon ankle-foot orthosis on gait in hemiplegic patients: a randomized pilot study. Clin Rehabil. 2011;25(11):989–98.

    Article  Google Scholar 

  12. Nayak B. Understanding the relevance of sample size calculation. Indian J Ophthalmol. 2010;58(6):469.

    Article  Google Scholar 

  13. Mundi R, Chaudhry H, Singh I, Bhandari M. Checklists to improve the quality of orthopaedic literature. IJO. 2008;42(2):150.

    Google Scholar 

  14. Chan S, Bhandari M. The quality of reporting of orthopaedic randomized trials with use of a checklist for nonpharmacological therapies. JBJS. 2007;89(9):1970–8.

    Google Scholar 

  15. Hamdy R, Montpetit K, Raney E, Aiona M, Fillman R, MacKenzie W, et al. Botulinum toxin type a injection in alleviating postoperative pain and improving quality of life in lower extremity limb lengthening and deformity correction: a pilot study. J Pediatr Orthop. 2009;29(5):427–34.

    Article  Google Scholar 

  16. Vaccaro AR, Patel T, Fischgrund J, Anderson DG, Truumees E, Herkowitz H, et al. A 2-year follow-up pilot study evaluating the safety and efficacy of op-1 putty (rhbmp-7) as an adjunct to iliac crest autograft in posterolateral lumbar fusions. Eur Spine J. 2005;14(7):623–9.

    Article  Google Scholar 

  17. Ekrol I, Hajducka C, Court-Brown C, McQueen MM. A comparison of RhBMP-7 (OP-1) and autogenous graft for metaphyseal defects after osteotomy of the distal radius. Injury. 2008;39:S73–82.

    Article  Google Scholar 

  18. Lerner T, Bullmann V, Schulte TL, Schneider M, Liljenqvist U. A level-1 pilot study to evaluate of ultraporous β-tricalcium phosphate as a graft extender in the posterior correction of adolescent idiopathic scoliosis. Eur Spine J. 2009;18(2):170–9.

    Article  Google Scholar 

  19. Marks P, O'Donnell S, Yee G. A pilot clinical evaluation comparing the Mitek bone–tendon–bone cross pin and bioabsorbable screw in anterior cruciate ligament reconstruction fixation, a randomized double blind controlled trial. Knee. 2008;15(3):168–73.

    Article  Google Scholar 

  20. Shamji MF, Roffey DM, Young DK, Reindl R, Wai EK. A pilot evaluation of the role of bracing in stable thoracolumbar burst fractures without neurological deficit. J Spinal Disord Tech. 2014;27(7):370–5.

    Article  Google Scholar 

  21. Carson JL, Terrin ML, Barton FB, Aaron R. Greenburg, AG, heck DA, et al. a pilot randomized trial comparing symptomatic vs. hemoglobin-level-driven red blood cell transfusions following hip fracture. Transfusion. 1998;38(6):522–9.

    Article  CAS  Google Scholar 

  22. Vaccaro AR, Patel T, Fischgrund J, Anderson DG, Truumees E, Herkowitz H, et al. A pilot safety and efficacy study of OP-1 putty (rhBMP-7) as an adjunct to iliac crest autograft in posterolateral lumbar fusions. Eur Spine J. 2003;12(5):495–500.

    Article  Google Scholar 

  23. Vaccaro AR, Patel T, Fischgrund J, Anderson DG, Truumees E, Herkowitz HN, et al. A pilot study evaluating the safety and efficacy of OP-1 putty (rhBMP-7) as a replacement for iliac crest autograft in posterolateral lumbar arthrodesis for degenerative spondylolisthesis. Spine. 2004;29(17):1885–92.

    Article  Google Scholar 

  24. Glazebrook M, Younger A. Lalonde, KAA prospective pilot study of B2A-coated ceramic granules (Amplex) compared to autograft for ankle and hindfoot arthrodesis. Foot Ankle Int. 2013;34(8):1055–63.

    Article  Google Scholar 

  25. Mauffrey C, McGuinness K, Parsons N, Achten J, Costa ML. A randomised pilot trial of “locking plate” fixation versus intramedullary nailing for extra-articular fractures of the distal tibia. J Bone Joint Surg Br. 2012;94(5):704–8.

    Article  CAS  Google Scholar 

  26. Darmanis S, Toms A, Durman R, Moore D, Eyres K. A technical innovation for improving identification of the trackers by the LED cameras in navigation-assisted total knee arthroplasty. Comput Aided Surg. 2007;12(4):247–51.

    Article  Google Scholar 

  27. Buse GL, Bhandari M, Sancheti P, Rocha S, Winemaker M, Adili A, et al. Accelerated care versus standard care among patients with HIP fracture: the HIP ATTACK pilot trial. CMAJ. 2013;186(1):E52–60.

    Google Scholar 

  28. Adolfsson L, Lindau T, Arner M. Acutrak screw fixation versus cast immobilisation for undisplaced scaphoid waist fractures. J Hand Surg. 2001;26(3):192–5.

    Article  CAS  Google Scholar 

  29. Kang P, Pei F, Shen B, Zhou Z, Yang J. Are the results of multiple drilling and alendronate for osteonecrosis of the femoral head better than those of multiple drilling? A pilot study. Joint Bone Spine. 2012;79(1):67–72.

    Article  Google Scholar 

  30. Moseley JB, Wray NP, Kuykendall D, Willis K, Landon G. Arthroscopic treatment of osteoarthritis of the knee: a prospective, randomized, placebo-controlled trial results of a pilot study. Am J Sports Med. 1996;24(1):28–34.

    Article  Google Scholar 

  31. Chhabra HS, Lima C, Sachdeva S, Mittal A, Nigam V, Chaturvedi D, et al. Autologous mucosal transplant in chronic spinal cord injury: an Indian pilot study. Spinal Cord. 2009;47(12):887.

    Article  CAS  Google Scholar 

  32. Wang Y, Zhou J, Yan F, Li G, Duan X, Pan H, et al. Comparison of arthrodesis with total contact casting for midfoot ulcerations associated with charcot neuroarthropathy. Med Sci Monit. 2015;21:2141–8.

    Article  Google Scholar 

  33. Zou J, Zhang W, Zhang CQ. Comparison of minimally invasive percutaneous plate osteosynthesis with open reduction and internal fixation for treatment of extra-articular distal tibia fractures. Injury. 2013;44(8):1102–6.

    Article  Google Scholar 

  34. Zehir S, Zehir R, Şahin E, Çalbıyık M. Comparison of novel intramedullary nailing with mini-invasive plating in surgical fixation of displaced midshaft clavicle fractures. Arch Orthop Trauma Surg. 2015;135(3):339–44.

    Article  Google Scholar 

  35. Pang HN, Yeo SJ, Chong HC, Chin PL, Ong J, Lo NN. Computer-assisted gap balancing technique improves outcome in total knee arthroplasty, compared with conventional measured resection technique. Knee Surg Sports Traumatol Arthrosc. 2011;19(9):1496–503.

    Article  Google Scholar 

  36. Wondrasch B, Zak L, Welsch GH, Marlovits S. Effect of accelerated weightbearing after matrix-associated autologous chondrocyte implantation on the femoral condyle on radiographic and clinical outcome after 2 years a prospective, randomized controlled pilot study. Am J Sports Med. 2009;37(1-suppl):88–96.

    Article  Google Scholar 

  37. Kraus VB, Birmingham J, Stabler TV, Feng S, Taylor DC, Moorman CT, et al. Effects of intraarticular IL1-Ra for acute anterior cruciate ligament knee injury: a randomized controlled pilot trial. Osteoarthr Cartil. 2012;20(4):271–8.

    Article  CAS  Google Scholar 

  38. Investigators F. Fluid lavage of open wounds (FLOW): a multicenter, blinded, factorial pilot trial comparing alternative irrigating solutions and pressures in patients with open fractures. J Trauma Acute Care Surg. 2011;71(3):596–606.

    Article  Google Scholar 

  39. Lindsey RW, Wood GW, Sadasivian KK, Stubbs HA, Block JE. Grafting long bone fractures with demineralized bone matrix putty enriched with bone marrow: pilot findings. Orthopedics. 2006;29(10):939–41.

    PubMed  Google Scholar 

  40. Costa ML, Shepstone L, Darrah C, Marshall T, Donell ST. Immediate full-weight-bearing mobilisation for repaired Achilles tendon ruptures: a pilot study. Injury. 2003;34(11):874–6.

    Article  CAS  Google Scholar 

  41. Mahowald ML, Krug HE, Singh JA, Dykstra D. Intra-articular botulinum toxin type a: a new approach to treat arthritis joint pain. Toxicon. 2009;54(5):658–67.

    Article  CAS  Google Scholar 

  42. Paterson K, Nicholls M, Bates D. Intra-articular injection of platelet-rich plasma in patients with knee osteoarthritis: a randomised controlled pilot study. J Sci Med Sport. 2013;16:e90.

    Article  Google Scholar 

  43. Zehir S, Calbiyik M, Zehir R, Ipek D. Intramedullary repair device against volar plating in the reconstruction of extra-articular and simple articular distal radius fractures; a randomized pilot study. Int J Orthop. 2014;38(8):1655–60.

    Article  Google Scholar 

  44. Hey HWD, Hong CC, Long AS, Hee HT. Is hybrid surgery of the cervical spine a good balance between fusion and arthroplasty? Pilot results from a single surgeon series. Eur Spine J. 2013;22(1):116–22.

    Article  Google Scholar 

  45. Kuo LC, Yang TH, Hsu YY, Wu PT, Lin CL, Hsu HY, et al. Is progressive early digit mobilization intervention beneficial for patients with external fixation of distal radius fracture? A pilot randomized controlled trial. Clin Rehabil. 2013;27(11):983–93.

    Article  Google Scholar 

  46. Abbott A, Halvorsen M, Dedering Å. Is there a need for cervical collar usage post anterior cervical decompression and fusion using interbody cages? A randomized controlled pilot trial. Physiother Theory Pract. 2013;29(4):290–300.

    Article  Google Scholar 

  47. McMorland G, Suter E, Casha S, du Plessis SJ, Hurlbert RJ. Manipulation or microdiskectomy for sciatica? A prospective randomized clinical study. JMPT. 2010;33(8):576–84.

    PubMed  Google Scholar 

  48. Ringel F, Stüer C, Reinke A, Preuss A, Behr M, Meyer B, et al. Accuracy of robot-assisted placement of lumbar and sacral pedicle screws: a prospective randomized comparison to conventional freehand screw implantation. Spine. 2012;37(8):E496–501.

    Article  Google Scholar 

  49. Boonen S, Rosen C, Bouillon R, Sommer A, McKay M, Rosen D, et al. Musculoskeletal effects of the recombinant human IGF-I/IGF binding protein-3 complex in osteoporotic patients with proximal femoral fracture: a double-blind, placebo-controlled pilot study. J Clin Endocrinol Metab. 2002;87(4):1593–9.

    Article  CAS  Google Scholar 

  50. Griffin D, Parsons N, Shaw E, Kulikov Y, Hutchinson C, Thorogood M, et al. Operative versus non-operative treatment for closed, displaced, intra-articular fractures of the calcaneus: randomised controlled trial. BMJ. 2014;349:g4483.

    Article  Google Scholar 

  51. Guo Q, Shen Y, Zong Z, Zhao Y, Liu H, Hua X, et al. Percutaneous compression plate versus proximal femoral nail anti-rotation in treating elderly patients with intertrochanteric fractures: a prospective randomized study. J Orthop Sci. 2013;18(6):977–86.

    Article  Google Scholar 

  52. Storey P, Armstrong D, Dear H, Bradley M, Burke F. Pilot randomised controlled trial comparing C-Trac splints with beta wrist braces for the management of carpal tunnel syndrome. Hand Ther. 2013;18(2):35–41.

    Article  Google Scholar 

  53. Zhang YZ, Chen B, Lu S, Yang Y, Zhao JM, Liu R, et al. Preliminary application of computer-assisted patient-specific acetabular navigational template for total hip arthroplasty in adult single development dysplasia of the hip. Int J Med Robot. 2011;7(4):469–74.

    Article  Google Scholar 

  54. Nejrup K, de Fine ON, Jacobsen JL, Siersma V. Randomised controlled trial of extraarticular gold bead implantation for treatment of knee osteoarthritis: a pilot study. Clin Rheumatol. 2008;27(11):1363–9.

    Article  Google Scholar 

  55. Eskander MBF, Limb D, Stone MH, Furlong AJ, Shardlow D, Stead D, et al. Sequential mechanical and pharmacological thromboprophylaxis in the surgery of hip fractures. Int Orthop. 1997;21(4):259–61.

    Article  CAS  Google Scholar 

  56. Jordan R, Hao J, Fader R, Gibula D, Mauffrey C. Study protocol: trial of inflation osteoplasty in the management of tibial plateau fractures. EJOST. 2014;24(5):647–53.

    CAS  Google Scholar 

  57. Kearney RS, Achten J, Parsons NR, Costa ML. The comprehensive cohort model in a pilot trial in orthopaedic trauma. BMC Med Res Methodol (2011);11(1):1.

  58. Sabeti M, Schmidt M, Ziai P, Graf A, Nemecek E, Schueller-Weidekamm C. The intraoperative use of ultrasound facilitates significantly the arthroscopic debridement of calcific rotator cuff tendinitis. Arch Orthop and Trauma Surg. 2014;134(5):651–6.

    Article  CAS  Google Scholar 

  59. Dutton T, De-Souza R, Parsons N, Costa ML. The timing of tourniquet release and ‘retransfusion’drains in total knee arthroplasty: a stratified randomised pilot investigation. Knee. 2012;19(3):190–2.

    Article  CAS  Google Scholar 

  60. Sardar Z, Alexander D, Oxner W, Plessis SD, Yee A, Wai EK, et al. Twelve-month results of a multicenter, blinded, pilot study of a novel peptide (B2A) in promoting lumbar spine fusion. J Neurosurg Spine. 2015;22(4):358–66.

    Article  Google Scholar 

  61. Capa-Grasa A, Rojo-Manaute JM, Rodríguez FC, Martín JV. Ultra minimally invasive sonographically guided carpal tunnel release: an external pilot study. OTSR. 2014;100(3):287–92.

    CAS  PubMed  Google Scholar 

  62. Okcu G, Ozkayin N, Okta C, Topcu I, Aktuglu K. Which implant is better for treating reverse obliquity fractures of the proximal femur: a standard or long nail? Clin Orthop Relat Res. 2013;471(9):2768–75.

    Article  Google Scholar 

  63. Manach YL, Collins G, Bhandari M, Bessissow A, Boddaert J, Khiami F, et al. Outcomes after hip fracture surgery compared with elective total hip replacement. JAMA. 2015;314(11):1159.

    Article  Google Scholar 

  64. Hamdy RC, Montpetit K, Aiona MD, MacKenzie WG, van Bosse HJ, Narayanan U, et al. Safety and efficacy of botulinum toxin a in children undergoing lower limb lengthening and deformity correction: results of a double-blind, multicenter, randomized controlled trial. J Pediatr Orthop. 2016;36(1):48–55.

    PubMed  Google Scholar 

  65. Bhandari M, Jeray KJ, Petrisor BA, Devereaux PJ, Heels-Ansdell D, Schemitsch EH, et al. A trial of wound irrigation in the initial management of open fracture wounds. N Engl J Med. 2015;373(27):2629.

    Article  CAS  Google Scholar 

  66. Costa ML, MacMillan K, Halliday D, Chester R, Shepstone L, Robinson AH, et al. Randomised controlled trials of immediate weight-bearing mobilisation for rupture of the tendo Achillis. J Bone J Surg Br. 2006;88(1):69–77.

    Article  CAS  Google Scholar 

  67. Rojo-Manaute J, Capa-Grasa A, Chana-Rodríguez F, Perez-Mañanes R, Rodriguez-Maruri G, Sanz-Ruiz P, et al. Ultra-minimally invasive ultrasound-guided carpal tunnel release. J Ultrasound Med. 2016;35(6):1149–57.

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank Ajaykumar Shanmugaraj for his assistance with the article’s submission.

Funding

Not applicable.

Availability of data and materials

All data generated or analysed during this study are included in this published article, found in the Additional file.

Author information

Authors and Affiliations

  1. Faculty of Health Sciences, McMaster University, Hamilton, ON, Canada

    Bijal Desai, Veeral Desai & Amr Saleh

  2. Schulich School of Medicine and Dentistry, Western University, London, ON, Canada

    Shivani Shah

  3. Faculty of Medicine, University of Toronto, Toronto, ON, Canada

    Archita Srinath

  4. Department of Health Research Methods, Evidence, and Impact (HEI), Division of Orthopaedic Surgery, McMaster University, Hamilton, ON, Canada

    Nicole Simunovic, Sheila Sprague & Mohit Bhandari

  5. Department of Surgery, Division of Orthopaedic Surgery, McMaster University, Hamilton, ON, Canada

    Nicole Simunovic, Andrew Duong, Sheila Sprague & Mohit Bhandari

Authors
  1. Bijal Desai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Veeral Desai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shivani Shah
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Archita Srinath
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Amr Saleh
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Nicole Simunovic
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Andrew Duong
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Sheila Sprague
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Mohit Bhandari
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

BD, VD, SS, AS and AS all contributed to paper screening, data abstraction, data validation, and writing of the paper. NS, AD, SS, and MB contributed to study design, manuscript revision, study supervision, and have critically reviewed all content during the preparation of the final paper. All authors have made substantial contributions to the paper, including study conduct, manuscript drafting, have given final approval for this version to be published, and have agreed to be accountable for all aspects of the work.

Corresponding author

Correspondence to Nicole Simunovic.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Additional files

Additional file 1:

Search Strategy. (DOCX 63 kb)

Additional file 2:

Modified Scoring of the CLEAR NPT Scale. (DOCX 87 kb)

Additional file 3:

References of Included Pilot and Definitive RCTs. (DOCX 30 kb)

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Desai, B., Desai, V., Shah, S. et al. Pilot randomized controlled trials in the orthopaedic surgery literature: a systematic review. BMC Musculoskelet Disord 19, 412 (2018). https://doi.org/10.1186/s12891-018-2337-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s12891-018-2337-7

Keywords

BMC Musculoskeletal Disorders

ISSN: 1471-2474

Contact us