- Research article
- Open Access
- Open Peer Review
Sources of information influencing decision-making in orthopaedic surgery - an international online survey of 1147 orthopaedic surgeons
BMC Musculoskeletal Disordersvolume 14, Article number: 96 (2013)
Manufacturers of implants and materials in the field of orthopaedics use significant amounts of funding to produce informational material to influence the decision-making process of orthopaedic surgeons with regards to choice between novel implants and techniques. It remains unclear how far orthopaedic surgeons are really influenced by the materials supplied by companies or whether other, evidence-based publications have a higher impact on their decision-making. The objective was to evaluate the subjective usefulness and usage of different sources of information upon which orthopaedic surgeons base their decisions when acquiring new implants or techniques.
We undertook an online survey of 1174 orthopaedic surgeons worldwide (of whom n = 305 were head of their department). The questionnaire included 34 items. Sequences were randomized to reduce possible bias. Questions were closed or semi-open with single or multiple answers. The usage and relevance of different sources of information when learning about and selecting orthopaedic treatments were evaluated. Orthopaedic surgeons and trainees were targeted, and were only allowed to respond once over a period of two weeks. Baseline information included country of workplace, level of experience and orthopaedic subspecialisation. The results were statistically evaluated.
Independent scientific proof had the highest influence on decisions for treatment while OEM (Original Equipment Manufacturer) driven activities like newsletters, white papers or workshops had the least impact. Comparison of answers from the three best-represented countries in this study (Germany, UK and USA) showed some significant differences: Scientific literature and congresses are significantly more important in the US than in the UK or Germany, although they are very important in all countries.
Independent and peer-reviewed sources of information are preferred by surgeons when choosing between methods and implants. Manufacturers of medical devices in orthopaedics employ a considerable workforce to inform or influence hospital managers and leading doctors with marketing activities. Our results indicate that it might be far more effective to channel at least some of these funds into peer-reviewed research projects, thereby assuring significantly higher acceptance of the related products.
Manufacturers of implants and materials in the field of orthopaedics and traumatology produce a large number of leaflets, white papers and publications each year to influence the decision-making process of orthopaedic surgeons with regards to choice between novel implants and techniques. This puts a large financial burden on these companies [1, 2]. On the other hand, the amount of money that companies are willing to spend on clinical research funding and evidence-based research projects appears to be steadily decreasing . Manufacturers often quote compliance rules and internal cost saving regulations as justification. Meanwhile, the same manufacturers are increasingly investing in direct-to-consumer advertising, despite the fact that surgeons feel an overall negative impact on their practice and their interaction with patients .
It remains unclear how much orthopaedic surgeons are really influenced by the materials supplied by companies or whether more evidence-based publications have a higher impact on their decision-making. If we could determine whether the latter was a more effective way to guide surgeons, it might be possible to increase the amount of support gained from manufacturers for funding in the area of evidence-based orthopaedics. The funding of research by industry per se is not under public scrutiny, but the fact that sources have not been properly declared in all cases in the past is an issue . Orthopaedic surgeons are increasingly demanding data supporting improvement of expensive new technology and new (and often expensive) treatments .
There have been numerous surveys regarding different aspects of decision-making in orthopaedic surgery in the past decades (e.g. [7–14]), and a few of these focused on the decision-making process based on different forms of information , mostly related to evidence-based practice [16, 17]. It is known that online surveys have a lower response rate than mail questionnaires [17, 18], that the response rates to online surveys have been declining since the advent of the internet , and that questionnaires should be preceded by a postal letter or pre-contact to increase response rates .
On the other hand, there is no proof that increasing the response rate increases study validity in orthopaedics . We therefore decided to use a strictly e-mail and online-based questionnaire without prior postal contact. This approach made it financially possible to contact a large number of surgeons.
The rationale for this study was to gain knowledge about the subjective usefulness and usage of different sources of information upon which orthopaedic surgeons base their decisions when acquiring new implants or techniques.
Study design: internet-based cohort survey
We developed a questionnaire focusing primarily on the factors influencing the deployment of new techniques and implants. Although no patient-related data was used in this study, we asked the local ethical committee to review the study protocol and documentation to assure data safety. After some alterations to the protocol regarding anonymisation and measures against intentional data fraud, the protocol was approved (Ethik-Kommission Lübeck; vote number 10–181). The software keyIngress (Ingress oHG, Norderstedt, Germany) was used for online survey management, including the randomization of items. Reports were generated using the software module keyIngress report from the same company.
The questionnaire included 34 items. These were provided in blocks of items concerning the same topic, within which the sequence of items was randomized to reduce possible bias by the online survey system. Questions were closed or semi-open with single or multiple answers. A progress bar that stated the percentage of successfully finished pages (14 over all, see Figure 1) was included, and progress to the next page was only possible when all questions had been answered. All questions included the option of a “no answer” reply. A unique link to the questionnaire was provided in the e-mail. The questionnaire could only be filled in once and only within 14 days. After 7 days, a reminder e-mail was sent to all non-responders. All correspondence and the questionnaire were in English. The e-mails were sent out between September 14 and November 18 2010 in three waves.
Overall 12,005 orthopaedic surgeons received an e-mail including a short letter explaining the request to participate in an online survey questionnaire. Contact e-mail addresses were provided by national and international orthopaedic societies and were strictly anonymized. There is thereby no data available about the population initially contacted apart from the fact that all of them were orthopaedic surgeons. This also was the only criterion for eligibility to participate in the study.
The number of fully completed questionnaires and thereby sample size was n = 1174 (for details see Table 1).
Surgeons were asked in which country they were currently working, and not their own nationality. The largest sample stemmed from the United States (20%), followed by the United Kingdom (14%) and Germany (10%) (for details see Figure 2).
Seventy percent of respondents gave their workplace as a teaching hospital or university hospital, 17% were employed in non-academic hospitals and 14% in private practice. For details of the sample see Table 2.
The main fields of work were ‘Trauma’ (42%), ‘Total Joint Reconstruction’ (26%) and ‘Knee Surgery’ (22%) (multiple answers, Figure 3). Twenty percent of respondents were undergoing training, and 26% were team leaders or heads of department. Thirty-eight percent of respondents had more than 15 years experience in orthopaedics (see Figure 4). For details of the sample see Table 2.
Statistical analysis was conducted using SPSS (version 19, IBM SPSS, Chicago, USA). Frequency analysis, cross-tabs with bivariate Chi2-tests, T-tests for paired samples (repeated measurement) and univariate variance analysis (ANOVA) were employed.
Overall n = 1174 (13.6% of all successful mail contacts) fully responded to the questionnaire. An incomplete questionnaire was found in n = 197 (2.2%) participants. These had to be excluded according to the study protocol.
The first closed single option question asked how often the different online and print media were read in order to help identify treatments for patients. Scientific literature, e.g. journal article, was read most often with 21% of respondents reading more than 15 articles per month and 14% reading between 11 and 15 articles per month. Newsletters or white papers provided by industrial sources were least frequently read with 56% reading 1 to 5 per month and 31% reading none. There was a statistically significant difference in the average number of journal articles (p = 0.02) and manufacturers’ newsletters (p = 0.01) read between department heads and non-heads with department heads reading more of each source of information.
The participants were asked to rate different sources of information with regards choosing a treatment method for patients. Scientific literature, independent training courses and scientific meetings were highly rated. Full details are given in Figure 5. Although the internet was ranked low as a source, 43% of respondents still found it to be a “Good” or “Very Good” way of gaining knowledge. Newsletters and white papers were ranked lowest with the largest group having a “Neutral” position. There were no detectable statistical differences between level of experience or type of hospital.
The next item block enquired about different types of event at which medical knowledge might be gained. The most frequently attended activities were local meetings with colleagues followed by scientific congresses. Training courses or workshops by OEM were frequented regularly by a smaller proportion of respondents (for details see Figure 6). There was no difference between the types of hospital. Department heads attended congresses, courses and workshops significantly more often than other members of staff (p = 0.000 for each item), but there was no difference regarding local meetings.
In the next step, participants were asked which factors would encourage them to choose a more expensive treatment option. The majority would base their decision on scientific evidence of a better outcome. Thirty-three percent stated that they definitely or probably would consider a more expensive treatment option if a company newsletter or white paper stated that the outcome was better (for details see Figure 7). There were no detectable statistical differences regarding type of hospital or level of experience. Statistical differences were found between the three nations with the most participants. Ninety-eight percent of participants from the United States (n = 232) responded to the question ‘would they consider a more expensive treatment if it is scientifically shown to have a better outcome’ with definitely or probably yes compared to just 93% for German participants (n = 106; p = 0.028). Thirty-four percent of participants from the United Kingdom (n = 161) answered the question ‘would they consider a more expensive treatment if the better outcome was documented in a company white paper or newsletter’ with definitely or probably yes compared to just 19% for participants from the United States (p = 0.031). Other trans-national comparisons showed no statistical significant differences.
Online surveys have been successfully used in orthopaedics in recent times either to interview surgeons [21–24] or patients . The response rate of just 13.6% illustrated that it is not easy to persuade a large proportion of surgeons to participate in an online survey. Nevertheless, more than a thousand participants did respond, assuring a good dataset. Furthermore, a low response rate does not necessarily mean low validity .
The main aim of our study was to assess the impact of different forms of information in the decision-making process of surgeons. Orthopaedic surgeons are provided with vast amounts of information . Sources like the internet and manufacturers’ leaflets are easy to access and free of charge. Training courses require time and the surgeon incurs costs, a factor that might explain their relatively low usage by junior surgeons . Journal articles also require time to find and study the relevant articles. As scientific literature is often written in English, we cannot exclude a certain language bias.
Nevertheless we clearly found that the sources of information that are preferred by surgeons remain peer-reviewed research articles, presentations at congresses and training courses by independent organisations. Ninety-four percent of respondents judged peer-reviewed scientific sources as a good or very good way of gaining important knowledge in order to identify novel treatment methods. Eighty-seven percent thought the same about independent courses and workshops. Meetings and congresses, as a source of information, were seen as important by 86%. In contrast, manufacturer-initiated publications were only seen by 20% to be a good or very good source. This is in accordance with studies that found evidence-based medicine to be of increasing importance for surgeons. Their impact in terms of changing knowledge, attitudes and uptake in clinical practice has been researched in the past with the aim of improving the use of research findings by individual healthcare professionals . The role of the internet and material provided by manufacturers as a source for decision-making and implementation of new techniques on the other hand has rarely been studied, and even then with suboptimal methodology .
One of the limitations of this study is that we did not enquire about which organisations are seen as independent by orthopaedic surgeons. It would be interesting to determine whether organisations that are officially independent but are directly funded by manufacturers are accepted as “independent”. In addition, we did not specify exactly what respondents judged as “information from the internet”.
Statistical evaluation of the three largest participating nations showed small but significant differences that we cannot fully explain. A possible explanation for the relatively high rate of acceptance of “manufacturers’ newsletters and white papers” in the United Kingdom might be attributed to the term “white papers”, which is the name often given to governmental publications or announcements by the national health service [30–33]. This result should be interpreted with caution, as bias cannot therefore be excluded. The language bias is another restriction of our study, we can not be sure that number of participants and results would have been different if the survey would have been multilingual [34, 35]. Knobloch described a general tendency to publish positive results in English , resulting in a selection bias. The potential limitation of response bias has been widely described [37–42]. In a recent German trial, internet based data sampling showed an additional response bias but had no direct effect on outcome scales . We cannot exclude response bias for our study.
Manufacturers certainly require large monetary resources each year to plan, create, print and distribute material like websites, newsletters, workshops and white papers. Clearly the aim of these materials is to influence decision-making processes in orthopaedic surgery. The results of our study indicate that it might be far more effective to channel at least some of these funds into peer-reviewed research projects, thereby assuring significantly higher acceptance of their products.
Synthes Audited Full Year. 2011,http://www.synthes.com/sites/intl/InvestorsMedia/FinancialReports/Pages/2009-10.aspx, Financial Statements,
Lissy DA: Orthopaedic Product Development: Steps to Success. OrthoTec. vol. Volume 2. 2011, Fairfield, NJ: UBM Canon
Shore BJ, Nasreddine AY, Kocher MS: Overcoming the funding challenge: the cost of randomized controlled trials in the next decade. J Bone Joint Surg. 2012, 94 (Suppl 1(E)): 101-106. 10.2106/JBJS.L.00193.
Bozic KJ, Smith AR, Hariri S, Adeoye S, Gourville J, Maloney WJ, Parsley B, Rubash HE: The impact of direct-to-consumer advertising in orthopaedics. Clin Orthop Relat Res. 2007, 458: 202-219.
Gelberman RH, Samson D, Mirza SK, Callaghan JJ, Pellegrini VD: Orthopaedic surgeons and the medical device industry: the threat to scientific integrity and the public trust. J Bone Joint Surg. 2010, 92 (3): 765-10.2106/JBJS.I.01164.
Ranawat A, Nunley R, Bozic K: Executive summary: value-based purchasing and technology assessment in orthopaedics. Clin Orthop Relat Res. 2009, 467 (10): 2556-2560. 10.1007/s11999-009-0908-5.
Borkhoff CM, Hawker GA, Kreder HJ, Glazier RH, Mahomed NN, Wright JG: The effect of patients’ sex on physicians’ recommendations for total knee arthroplasty. CMAJ. 2008, 178 (6): 681-687.
Bednarska E, Bryant D, Devereaux PJ: Orthopaedic surgeons prefer to participate in expertise-based randomized trials. Clin Orthop Relat Res. 2008, 466 (7): 1734-1744. 10.1007/s11999-008-0273-9.
Immerman I, Kubiak EN, Zuckerman JD: Resident work-hour rules: a survey of residents’ and program directors’ opinions and attitudes. Am J Orthop (Belle Mead NJ). 2007, 36 (12): E172-E179. discussion E179
Zlowodzki M, Bhandari M, Keel M, Hanson BP, Schemitsch E: Perception of Garden’s classification for femoral neck fractures: an international survey of 298 orthopaedic trauma surgeons. Arch Orthop Trauma Surg. 2005, 125 (7): 503-505. 10.1007/s00402-005-0022-4.
Irwin ZN, Hilibrand A, Gustavel M, McLain R, Shaffer W, Myers M, Glaser J, Hart RA: Variation in surgical decision making for degenerative spinal disorders. Part I: lumbar spine. Spine (Phila Pa 1976). 2005, 30 (19): 2208-2213. 10.1097/01.brs.0000181057.60012.08.
Bhandari M, Devereaux PJ, Tornetta P, Swiontkowski MF, Berry DJ, Haidukewych G, Schemitsch EH, Hanson BP, Koval K, Dirschl D: Operative management of displaced femoral neck fractures in elderly patients. An international survey. J Bone Joint Surg Am. 2005, 87 (9): 2122-2130. 10.2106/JBJS.E.00535.
Marx RG, Jones EC, Angel M, Wickiewicz TL, Warren RF: Beliefs and attitudes of members of the American Academy of Orthopaedic Surgeons regarding the treatment of anterior cruciate ligament injury. Arthroscopy. 2003, 19 (7): 762-770. 10.1016/S0749-8063(03)00398-0.
Sheehan J, Mohamed F, Reilly M, Perry IJ: Secondary prevention following fractured neck of femur: a survey of orthopaedic surgeons practice. Ir Med J. 2000, 93 (4): 105-107.
Bhandari M, Devereaux PJ, Swiontkowski MF, Schemitsch EH, Shankardass K, Sprague S, Guyatt GH: A randomized trial of opinion leader endorsement in a survey of orthopaedic surgeons: effect on primary response rates. Int J Epidemiol. 2003, 32 (4): 634-636. 10.1093/ije/dyg112.
Hanson BP, Bhandari M, Audige L, Helfet D: The need for education in evidence-based orthopedics: an international survey of AO course participants. Acta Orthop Scand. 2004, 75 (3): 328-332. 10.1080/00016470410001277.
Sprague S, Quigley L, Bhandari M: Survey design in orthopaedic surgery: getting surgeons to respond. J Bone Joint Surg Am. 2009, 91 (Supplement_3): 27-34.
Grava-Gubins I, Scott S: Effects of various methodologic strategies. Can Fam Physician. 2008, 54 (10): 1424-1430.
Cook C, Heath F, Thompson RL: A meta-analysis of response rates in web- or internet-based surveys. Educ Psychol Meas. 2000, 60 (6): 821-836. 10.1177/00131640021970934.
Hing CB, Smith TO, Hooper L, Song F, Donell ST: A review of how to conduct a surgical survey using a questionnaire. Knee. 2011, 18 (4): 209-213. 10.1016/j.knee.2010.10.003.
Hariri S, York SC, O’Connor MI, Parsley BS, McCarthy JC: A resident survey study of orthopedic fellowship specialty decision making and views on arthroplasty as a career. J Arthroplasty. 2011, 26 (6): 961-968. 10.1016/j.arth.2010.09.012. e961
Ratliff JK, Lebude B, Albert T, Anene-Maidoh T, Anderson G, Dagostino P, Maltenfort M, Hilibrand A, Sharan A, Vaccaro AR: Complications in spinal surgery: comparative survey of spine surgeons and patients who underwent spinal surgery. J Neurosurg Spine. 2009, 10 (6): 578-584. 10.3171/2009.2.SPINE0935.
Scherl SA, Rathjen KE, Gerardi J, Kiefer G, Georgopoulos G, Murphy-Zane MS, Blasier RD, Schoenecker PL, Epps H: Opinion survey regarding pediatric orthopaedic trauma call and emergency trauma management. J Pediatr Orthop. 2008, 28 (4): 393-396. 10.1097/BPO.0b013e318173f76c.
White J, Toy P, Gibbs P, Enneking W, Scarborough M: The current practice of orthopaedic oncology in North America. Clin Orthop Relat Res. 2010, 468 (11): 2840-2853. 10.1007/s11999-010-1413-6.
Bird S, McGill N: Blood conservation and pain control in scoliosis corrective surgery: an online survey of UK practice. Paediatr Anaesth. 2011, 21 (1): 50-53. 10.1111/j.1460-9592.2010.03443.x.
Purvis JM, Alexander AH, Einhorn TA, Griffin LY: American Orthopaedic Association symposium: Evaluating the flood of orthopaedic media and marketing information. J Bone Joint Surg Am. 2005, 87 (6): 1392-1398. 10.2106/JBJS.E.00191.
Harvey JR, Thomas NP: Regional orthopaedic courses–rationale and practice. Ann R Coll Surg Engl. 2004, 86 (6): 451-454. 10.1308/147870804759.
Eccles M, Grimshaw J, Walker A, Johnston M, Pitts N: Changing the behavior of healthcare professionals: the use of theory in promoting the uptake of research findings. J Clin Epidemiol. 2005, 58 (2): 107-112. 10.1016/j.jclinepi.2004.09.002.
LaRocca R, Yost J, Dobbins M, Ciliska D, Butt M: The effectiveness of knowledge translation strategies used in public health: a systematic review. BMC Public Health. 2012, 12 (1): 751-10.1186/1471-2458-12-751.
Halligan A: The NHS White Paper: licence for radical cultural reform of the NHS?. J R Soc Med. 2011, 104 (4): 146-148. 10.1258/jrsm.2011.11k008.
Bogg D: The NHS white paper and GP consortia offer opportunities for positive change. 2010, Brighton, England: Mental health today, 21-
McLellan A: White paper. Conflicting messages from the hint at growing resistance. Health Serv J. 2010, 120 (6224): 3-
Aitken M: White paper whiteout. J R Soc Med. 2010, 103 (11): 467-468. 10.1258/jrsm.2010.100275.
Jargin SV: Limited access to the international medical literature in Russia. Wien Med Wochenschr. 2012, 162 (11–12): 272-275.
de Granda-Orive JI, Solano-Reina S, Jimenez-Ruiz C: A new example of scientific bias of the English language: the American guide to stopping smoking treatment. Arch Bronconeumol. 2010, 46 (2): 104-105.
Knobloch K, Yoon U, Vogt PM: Preferred reporting items for systematic reviews and meta-analyses (PRISMA) statement and publication bias. J Craniomaxillofac Surg. 2011, 39 (2): 91-92. 10.1016/j.jcms.2010.11.001.
Hunter L: Challenging the reported disadvantages of e-questionnaires and addressing methodological issues of online data collection. Nurse Res. 2012, 20 (1): 11-20.
Mayr A, Gefeller O, Prokosch HU, Pirkl A, Frohlich A, de Zwaan M: Web-based data collection yielded an additional response bias–but had no direct effect on outcome scales. J Clin Epidemiol. 2012, 65 (9): 970-977. 10.1016/j.jclinepi.2012.03.005.
Shelley AM, Brunton P, Horner K: Questionnaire surveys of dentists on radiology. Dentomaxillofac Radiol. 2012, 41 (4): 267-275. 10.1259/dmfr/58627082.
Plante C, Jacques L, Chevalier S, Fournier M: Comparability of Internet and telephone data in a survey on the respiratory health of children. Can Respir J. 2012, 19 (1): 13-18.
Lippert S, Callaham ML, Lo B: Perceptions of conflict of interest disclosures among peer reviewers. PLoS One. 2011, 6 (11): e26900-10.1371/journal.pone.0026900.
Derby DC, Haan A, Wood K: Data quality assurance: an analysis of patient non-response. Int J Health Care Qual Assur. 2011, 24 (3): 198-210. 10.1108/09526861111116642.
The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2474/14/96/prepub
The authors thank Dr. Friedrich Pahlke, Biostatistician from Lübeck (Germany) for support during this research project.
The authors declare that they have no competing interests.
APS was involved in the conception and design of the study, data acquisition and drafting the manuscript; AJ was involved in the conception and design of the study and revising the manuscript critically for important intellectual content; RK was involved in evaluation of the data and drafting the manuscript; PJ was involved in evaluation of the data and revising the manuscript critically for important intellectual content; MB was involved in conception and design, statistical evaluation of the data and drafting the manuscript. All authors have given final approval of the version to be published.
Authors’ original submitted files for images
Below are the links to the authors’ original submitted files for images.