Sources of information influencing decision-making in orthopaedic surgery - an international online survey of 1147 orthopaedic surgeons
© Schulz et al.; licensee BioMed Central Ltd. 2013
Received: 22 February 2012
Accepted: 28 February 2013
Published: 14 March 2013
Manufacturers of implants and materials in the field of orthopaedics use significant amounts of funding to produce informational material to influence the decision-making process of orthopaedic surgeons with regards to choice between novel implants and techniques. It remains unclear how far orthopaedic surgeons are really influenced by the materials supplied by companies or whether other, evidence-based publications have a higher impact on their decision-making. The objective was to evaluate the subjective usefulness and usage of different sources of information upon which orthopaedic surgeons base their decisions when acquiring new implants or techniques.
We undertook an online survey of 1174 orthopaedic surgeons worldwide (of whom n = 305 were head of their department). The questionnaire included 34 items. Sequences were randomized to reduce possible bias. Questions were closed or semi-open with single or multiple answers. The usage and relevance of different sources of information when learning about and selecting orthopaedic treatments were evaluated. Orthopaedic surgeons and trainees were targeted, and were only allowed to respond once over a period of two weeks. Baseline information included country of workplace, level of experience and orthopaedic subspecialisation. The results were statistically evaluated.
Independent scientific proof had the highest influence on decisions for treatment while OEM (Original Equipment Manufacturer) driven activities like newsletters, white papers or workshops had the least impact. Comparison of answers from the three best-represented countries in this study (Germany, UK and USA) showed some significant differences: Scientific literature and congresses are significantly more important in the US than in the UK or Germany, although they are very important in all countries.
Independent and peer-reviewed sources of information are preferred by surgeons when choosing between methods and implants. Manufacturers of medical devices in orthopaedics employ a considerable workforce to inform or influence hospital managers and leading doctors with marketing activities. Our results indicate that it might be far more effective to channel at least some of these funds into peer-reviewed research projects, thereby assuring significantly higher acceptance of the related products.
KeywordsOrthopaedics Survey Decision-making process Evidence-based medicine Online evaluation Opinion Internet-based
Manufacturers of implants and materials in the field of orthopaedics and traumatology produce a large number of leaflets, white papers and publications each year to influence the decision-making process of orthopaedic surgeons with regards to choice between novel implants and techniques. This puts a large financial burden on these companies [1, 2]. On the other hand, the amount of money that companies are willing to spend on clinical research funding and evidence-based research projects appears to be steadily decreasing . Manufacturers often quote compliance rules and internal cost saving regulations as justification. Meanwhile, the same manufacturers are increasingly investing in direct-to-consumer advertising, despite the fact that surgeons feel an overall negative impact on their practice and their interaction with patients .
It remains unclear how much orthopaedic surgeons are really influenced by the materials supplied by companies or whether more evidence-based publications have a higher impact on their decision-making. If we could determine whether the latter was a more effective way to guide surgeons, it might be possible to increase the amount of support gained from manufacturers for funding in the area of evidence-based orthopaedics. The funding of research by industry per se is not under public scrutiny, but the fact that sources have not been properly declared in all cases in the past is an issue . Orthopaedic surgeons are increasingly demanding data supporting improvement of expensive new technology and new (and often expensive) treatments .
There have been numerous surveys regarding different aspects of decision-making in orthopaedic surgery in the past decades (e.g. [7–14]), and a few of these focused on the decision-making process based on different forms of information , mostly related to evidence-based practice [16, 17]. It is known that online surveys have a lower response rate than mail questionnaires [17, 18], that the response rates to online surveys have been declining since the advent of the internet , and that questionnaires should be preceded by a postal letter or pre-contact to increase response rates .
On the other hand, there is no proof that increasing the response rate increases study validity in orthopaedics . We therefore decided to use a strictly e-mail and online-based questionnaire without prior postal contact. This approach made it financially possible to contact a large number of surgeons.
The rationale for this study was to gain knowledge about the subjective usefulness and usage of different sources of information upon which orthopaedic surgeons base their decisions when acquiring new implants or techniques.
Study design: internet-based cohort survey
We developed a questionnaire focusing primarily on the factors influencing the deployment of new techniques and implants. Although no patient-related data was used in this study, we asked the local ethical committee to review the study protocol and documentation to assure data safety. After some alterations to the protocol regarding anonymisation and measures against intentional data fraud, the protocol was approved (Ethik-Kommission Lübeck; vote number 10–181). The software keyIngress (Ingress oHG, Norderstedt, Germany) was used for online survey management, including the randomization of items. Reports were generated using the software module keyIngress report from the same company.
Overall 12,005 orthopaedic surgeons received an e-mail including a short letter explaining the request to participate in an online survey questionnaire. Contact e-mail addresses were provided by national and international orthopaedic societies and were strictly anonymized. There is thereby no data available about the population initially contacted apart from the fact that all of them were orthopaedic surgeons. This also was the only criterion for eligibility to participate in the study.
Details of the final study sample size
Mail-outs and response rates
Total number of email addresses for mail-outs
n = 12,005
Incorrect email addresses or out-of-office replies
n = 3397
Mail-outs sent correctly
n = 8608
n = 1174
Unfinished questionnaires (incompletely answered)
n = 197
n = 7235
Description of the sample of 1174 participants
Which background does the hospital you are currently working have?
What is the level of your experience in orthopaedic surgery?
Junior assistant/registrar (1–3 years experience)
Senior assistant/registrar (4–6 years experience)
Consultant (7–15 years experience)
Senior consultant (>15 years experience)
Are you the head of your departmend (chief surgeon)?
Statistical analysis was conducted using SPSS (version 19, IBM SPSS, Chicago, USA). Frequency analysis, cross-tabs with bivariate Chi2-tests, T-tests for paired samples (repeated measurement) and univariate variance analysis (ANOVA) were employed.
Overall n = 1174 (13.6% of all successful mail contacts) fully responded to the questionnaire. An incomplete questionnaire was found in n = 197 (2.2%) participants. These had to be excluded according to the study protocol.
The first closed single option question asked how often the different online and print media were read in order to help identify treatments for patients. Scientific literature, e.g. journal article, was read most often with 21% of respondents reading more than 15 articles per month and 14% reading between 11 and 15 articles per month. Newsletters or white papers provided by industrial sources were least frequently read with 56% reading 1 to 5 per month and 31% reading none. There was a statistically significant difference in the average number of journal articles (p = 0.02) and manufacturers’ newsletters (p = 0.01) read between department heads and non-heads with department heads reading more of each source of information.
Online surveys have been successfully used in orthopaedics in recent times either to interview surgeons [21–24] or patients . The response rate of just 13.6% illustrated that it is not easy to persuade a large proportion of surgeons to participate in an online survey. Nevertheless, more than a thousand participants did respond, assuring a good dataset. Furthermore, a low response rate does not necessarily mean low validity .
The main aim of our study was to assess the impact of different forms of information in the decision-making process of surgeons. Orthopaedic surgeons are provided with vast amounts of information . Sources like the internet and manufacturers’ leaflets are easy to access and free of charge. Training courses require time and the surgeon incurs costs, a factor that might explain their relatively low usage by junior surgeons . Journal articles also require time to find and study the relevant articles. As scientific literature is often written in English, we cannot exclude a certain language bias.
Nevertheless we clearly found that the sources of information that are preferred by surgeons remain peer-reviewed research articles, presentations at congresses and training courses by independent organisations. Ninety-four percent of respondents judged peer-reviewed scientific sources as a good or very good way of gaining important knowledge in order to identify novel treatment methods. Eighty-seven percent thought the same about independent courses and workshops. Meetings and congresses, as a source of information, were seen as important by 86%. In contrast, manufacturer-initiated publications were only seen by 20% to be a good or very good source. This is in accordance with studies that found evidence-based medicine to be of increasing importance for surgeons. Their impact in terms of changing knowledge, attitudes and uptake in clinical practice has been researched in the past with the aim of improving the use of research findings by individual healthcare professionals . The role of the internet and material provided by manufacturers as a source for decision-making and implementation of new techniques on the other hand has rarely been studied, and even then with suboptimal methodology .
One of the limitations of this study is that we did not enquire about which organisations are seen as independent by orthopaedic surgeons. It would be interesting to determine whether organisations that are officially independent but are directly funded by manufacturers are accepted as “independent”. In addition, we did not specify exactly what respondents judged as “information from the internet”.
Statistical evaluation of the three largest participating nations showed small but significant differences that we cannot fully explain. A possible explanation for the relatively high rate of acceptance of “manufacturers’ newsletters and white papers” in the United Kingdom might be attributed to the term “white papers”, which is the name often given to governmental publications or announcements by the national health service [30–33]. This result should be interpreted with caution, as bias cannot therefore be excluded. The language bias is another restriction of our study, we can not be sure that number of participants and results would have been different if the survey would have been multilingual [34, 35]. Knobloch described a general tendency to publish positive results in English , resulting in a selection bias. The potential limitation of response bias has been widely described [37–42]. In a recent German trial, internet based data sampling showed an additional response bias but had no direct effect on outcome scales . We cannot exclude response bias for our study.
Manufacturers certainly require large monetary resources each year to plan, create, print and distribute material like websites, newsletters, workshops and white papers. Clearly the aim of these materials is to influence decision-making processes in orthopaedic surgery. The results of our study indicate that it might be far more effective to channel at least some of these funds into peer-reviewed research projects, thereby assuring significantly higher acceptance of their products.
The authors thank Dr. Friedrich Pahlke, Biostatistician from Lübeck (Germany) for support during this research project.
- Synthes Audited Full Year. 2011,http://www.synthes.com/sites/intl/InvestorsMedia/FinancialReports/Pages/2009-10.aspx, Financial Statements,
- Lissy DA: Orthopaedic Product Development: Steps to Success. OrthoTec. vol. Volume 2. 2011, Fairfield, NJ: UBM Canon
- Shore BJ, Nasreddine AY, Kocher MS: Overcoming the funding challenge: the cost of randomized controlled trials in the next decade. J Bone Joint Surg. 2012, 94 (Suppl 1(E)): 101-106. 10.2106/JBJS.L.00193.View ArticlePubMed
- Bozic KJ, Smith AR, Hariri S, Adeoye S, Gourville J, Maloney WJ, Parsley B, Rubash HE: The impact of direct-to-consumer advertising in orthopaedics. Clin Orthop Relat Res. 2007, 458: 202-219.PubMed
- Gelberman RH, Samson D, Mirza SK, Callaghan JJ, Pellegrini VD: Orthopaedic surgeons and the medical device industry: the threat to scientific integrity and the public trust. J Bone Joint Surg. 2010, 92 (3): 765-10.2106/JBJS.I.01164.View ArticlePubMed
- Ranawat A, Nunley R, Bozic K: Executive summary: value-based purchasing and technology assessment in orthopaedics. Clin Orthop Relat Res. 2009, 467 (10): 2556-2560. 10.1007/s11999-009-0908-5.PubMed CentralView ArticlePubMed
- Borkhoff CM, Hawker GA, Kreder HJ, Glazier RH, Mahomed NN, Wright JG: The effect of patients’ sex on physicians’ recommendations for total knee arthroplasty. CMAJ. 2008, 178 (6): 681-687.PubMed CentralView ArticlePubMed
- Bednarska E, Bryant D, Devereaux PJ: Orthopaedic surgeons prefer to participate in expertise-based randomized trials. Clin Orthop Relat Res. 2008, 466 (7): 1734-1744. 10.1007/s11999-008-0273-9.PubMed CentralView ArticlePubMed
- Immerman I, Kubiak EN, Zuckerman JD: Resident work-hour rules: a survey of residents’ and program directors’ opinions and attitudes. Am J Orthop (Belle Mead NJ). 2007, 36 (12): E172-E179. discussion E179
- Zlowodzki M, Bhandari M, Keel M, Hanson BP, Schemitsch E: Perception of Garden’s classification for femoral neck fractures: an international survey of 298 orthopaedic trauma surgeons. Arch Orthop Trauma Surg. 2005, 125 (7): 503-505. 10.1007/s00402-005-0022-4.View ArticlePubMed
- Irwin ZN, Hilibrand A, Gustavel M, McLain R, Shaffer W, Myers M, Glaser J, Hart RA: Variation in surgical decision making for degenerative spinal disorders. Part I: lumbar spine. Spine (Phila Pa 1976). 2005, 30 (19): 2208-2213. 10.1097/01.brs.0000181057.60012.08.View Article
- Bhandari M, Devereaux PJ, Tornetta P, Swiontkowski MF, Berry DJ, Haidukewych G, Schemitsch EH, Hanson BP, Koval K, Dirschl D: Operative management of displaced femoral neck fractures in elderly patients. An international survey. J Bone Joint Surg Am. 2005, 87 (9): 2122-2130. 10.2106/JBJS.E.00535.View ArticlePubMed
- Marx RG, Jones EC, Angel M, Wickiewicz TL, Warren RF: Beliefs and attitudes of members of the American Academy of Orthopaedic Surgeons regarding the treatment of anterior cruciate ligament injury. Arthroscopy. 2003, 19 (7): 762-770. 10.1016/S0749-8063(03)00398-0.View ArticlePubMed
- Sheehan J, Mohamed F, Reilly M, Perry IJ: Secondary prevention following fractured neck of femur: a survey of orthopaedic surgeons practice. Ir Med J. 2000, 93 (4): 105-107.PubMed
- Bhandari M, Devereaux PJ, Swiontkowski MF, Schemitsch EH, Shankardass K, Sprague S, Guyatt GH: A randomized trial of opinion leader endorsement in a survey of orthopaedic surgeons: effect on primary response rates. Int J Epidemiol. 2003, 32 (4): 634-636. 10.1093/ije/dyg112.View ArticlePubMed
- Hanson BP, Bhandari M, Audige L, Helfet D: The need for education in evidence-based orthopedics: an international survey of AO course participants. Acta Orthop Scand. 2004, 75 (3): 328-332. 10.1080/00016470410001277.View ArticlePubMed
- Sprague S, Quigley L, Bhandari M: Survey design in orthopaedic surgery: getting surgeons to respond. J Bone Joint Surg Am. 2009, 91 (Supplement_3): 27-34.View ArticlePubMed
- Grava-Gubins I, Scott S: Effects of various methodologic strategies. Can Fam Physician. 2008, 54 (10): 1424-1430.PubMed CentralPubMed
- Cook C, Heath F, Thompson RL: A meta-analysis of response rates in web- or internet-based surveys. Educ Psychol Meas. 2000, 60 (6): 821-836. 10.1177/00131640021970934.View Article
- Hing CB, Smith TO, Hooper L, Song F, Donell ST: A review of how to conduct a surgical survey using a questionnaire. Knee. 2011, 18 (4): 209-213. 10.1016/j.knee.2010.10.003.View ArticlePubMed
- Hariri S, York SC, O’Connor MI, Parsley BS, McCarthy JC: A resident survey study of orthopedic fellowship specialty decision making and views on arthroplasty as a career. J Arthroplasty. 2011, 26 (6): 961-968. 10.1016/j.arth.2010.09.012. e961View ArticlePubMed
- Ratliff JK, Lebude B, Albert T, Anene-Maidoh T, Anderson G, Dagostino P, Maltenfort M, Hilibrand A, Sharan A, Vaccaro AR: Complications in spinal surgery: comparative survey of spine surgeons and patients who underwent spinal surgery. J Neurosurg Spine. 2009, 10 (6): 578-584. 10.3171/2009.2.SPINE0935.View ArticlePubMed
- Scherl SA, Rathjen KE, Gerardi J, Kiefer G, Georgopoulos G, Murphy-Zane MS, Blasier RD, Schoenecker PL, Epps H: Opinion survey regarding pediatric orthopaedic trauma call and emergency trauma management. J Pediatr Orthop. 2008, 28 (4): 393-396. 10.1097/BPO.0b013e318173f76c.View ArticlePubMed
- White J, Toy P, Gibbs P, Enneking W, Scarborough M: The current practice of orthopaedic oncology in North America. Clin Orthop Relat Res. 2010, 468 (11): 2840-2853. 10.1007/s11999-010-1413-6.PubMed CentralView ArticlePubMed
- Bird S, McGill N: Blood conservation and pain control in scoliosis corrective surgery: an online survey of UK practice. Paediatr Anaesth. 2011, 21 (1): 50-53. 10.1111/j.1460-9592.2010.03443.x.View ArticlePubMed
- Purvis JM, Alexander AH, Einhorn TA, Griffin LY: American Orthopaedic Association symposium: Evaluating the flood of orthopaedic media and marketing information. J Bone Joint Surg Am. 2005, 87 (6): 1392-1398. 10.2106/JBJS.E.00191.View ArticlePubMed
- Harvey JR, Thomas NP: Regional orthopaedic courses–rationale and practice. Ann R Coll Surg Engl. 2004, 86 (6): 451-454. 10.1308/147870804759.PubMed CentralView ArticlePubMed
- Eccles M, Grimshaw J, Walker A, Johnston M, Pitts N: Changing the behavior of healthcare professionals: the use of theory in promoting the uptake of research findings. J Clin Epidemiol. 2005, 58 (2): 107-112. 10.1016/j.jclinepi.2004.09.002.View ArticlePubMed
- LaRocca R, Yost J, Dobbins M, Ciliska D, Butt M: The effectiveness of knowledge translation strategies used in public health: a systematic review. BMC Public Health. 2012, 12 (1): 751-10.1186/1471-2458-12-751.PubMed CentralView ArticlePubMed
- Halligan A: The NHS White Paper: licence for radical cultural reform of the NHS?. J R Soc Med. 2011, 104 (4): 146-148. 10.1258/jrsm.2011.11k008.PubMed CentralView ArticlePubMed
- Bogg D: The NHS white paper and GP consortia offer opportunities for positive change. 2010, Brighton, England: Mental health today, 21-
- McLellan A: White paper. Conflicting messages from the hint at growing resistance. Health Serv J. 2010, 120 (6224): 3-
- Aitken M: White paper whiteout. J R Soc Med. 2010, 103 (11): 467-468. 10.1258/jrsm.2010.100275.PubMed CentralView ArticlePubMed
- Jargin SV: Limited access to the international medical literature in Russia. Wien Med Wochenschr. 2012, 162 (11–12): 272-275.View ArticlePubMed
- de Granda-Orive JI, Solano-Reina S, Jimenez-Ruiz C: A new example of scientific bias of the English language: the American guide to stopping smoking treatment. Arch Bronconeumol. 2010, 46 (2): 104-105.View ArticlePubMed
- Knobloch K, Yoon U, Vogt PM: Preferred reporting items for systematic reviews and meta-analyses (PRISMA) statement and publication bias. J Craniomaxillofac Surg. 2011, 39 (2): 91-92. 10.1016/j.jcms.2010.11.001.View ArticlePubMed
- Hunter L: Challenging the reported disadvantages of e-questionnaires and addressing methodological issues of online data collection. Nurse Res. 2012, 20 (1): 11-20.View ArticlePubMed
- Mayr A, Gefeller O, Prokosch HU, Pirkl A, Frohlich A, de Zwaan M: Web-based data collection yielded an additional response bias–but had no direct effect on outcome scales. J Clin Epidemiol. 2012, 65 (9): 970-977. 10.1016/j.jclinepi.2012.03.005.View ArticlePubMed
- Shelley AM, Brunton P, Horner K: Questionnaire surveys of dentists on radiology. Dentomaxillofac Radiol. 2012, 41 (4): 267-275. 10.1259/dmfr/58627082.PubMed CentralView ArticlePubMed
- Plante C, Jacques L, Chevalier S, Fournier M: Comparability of Internet and telephone data in a survey on the respiratory health of children. Can Respir J. 2012, 19 (1): 13-18.PubMed CentralPubMed
- Lippert S, Callaham ML, Lo B: Perceptions of conflict of interest disclosures among peer reviewers. PLoS One. 2011, 6 (11): e26900-10.1371/journal.pone.0026900.PubMed CentralView ArticlePubMed
- Derby DC, Haan A, Wood K: Data quality assurance: an analysis of patient non-response. Int J Health Care Qual Assur. 2011, 24 (3): 198-210. 10.1108/09526861111116642.View ArticlePubMed
- The pre-publication history for this paper can be accessed here:http://www.biomedcentral.com/1471-2474/14/96/prepub
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.