Consensus on draft OMERACT core domains for clinical trials of Total Joint Replacement outcome by orthopaedic surgeons: a report from the International consensus on outcome measures in TJR trials (I-COMiTT) group
BMC Musculoskeletal Disorders volume 18, Article number: 45 (2017)
There are no core outcome domain or measurement sets for Total Joint Replacement (TJR) clinical trials. Our objective was to achieve an International consensus by orthopaedic surgeons on the OMERACT core domain/area set for TJR clinical trials.
We conducted surveys of two orthopaedic surgeon cohorts, which included (1) the leadership of international orthopaedic societies and surgeons (IOS; cohort 1), and (2) the members of the American Academy of Orthopaedic Surgeons’ Outcome Special Interest Group (AAOS-Outcome SIG), and/or the Outcome Research Interest Group of the Orthopaedic Research Society (ORS; cohort 2). Participants rated OMERACT-endorsed preliminary core area set for TJR clinical trials on a 1 to 9 scale, indicating 1–3 as domain of limited importance, 4–6 being important, but not critical, and 7–9 being critical.
Eighteen survey participants from the IOS group and 69 participants from the AAOS-Outcome SIG/ORS groups completed the survey questionnaire. The median (interquartile range [IQR]) scores were seven or higher for all six proposed preliminary core areas/domains across both groups, IOS and AAOS-Outcome SIG/ORS, respectively: pain, 8 [8, 9] and 8 [7, 9]; function, 8 [8, 8] and 8 [7, 9]; patient satisfaction, 8 [7, 9] and 8 [7, 8]; revision surgery, 7 [6, 9] and 8 [6, 8]; adverse events, 7 [5, 8] and 7 [6, 9]; and death, 7 [7, 9] and 8 [5, 9]. Respective median scores were lower for two additional optional domains: patient participation, 6.5 [5, 7] and 6 [5, 8]; and cost, 6 [5, 7] and 6 [5, 7].
This study showed that two independent surveys dervied from three groups of orthopaedic surgeons with international representation endorsed a preliminary/draft OMERACT core domain/area set for Joint Replacement clinical trials.
Total Joint Replacement (TJR) is one of the most common elective procedures performed in the US and worldwide. The rate of utilization of this highly cost effective and clinically successful procedure has, in part, been attributed to the increasing prevalence of osteoarthritis, obesity, and an aging population [1–3]. Despite the recognition that the measurement of pain and function after TJR is very important and the common practice is to assess these outcomes in clinical practice, there is no consensus as to which domains or outcome measures should be included in every TJR trial. Consistency in measurement is essential for enabling valid comparisons between TJR clinical trials and head-to-head studies, which currently is hampered by the heterogeneity of outcome measures , and the inability to pool data for meta-analysis. A TJR outcome Working Group (WG) first met at the OMERACT-9 meeting in 2008 where this critical issue was discussed in detail . Based on the strategy outlined, we have continued the work in this area within our WG for the last 8 years.
The method of development of TJR trial core domain set was a multi-step, data-driven process that required a consensus by key stakeholders, including but not limited to orthopaedic surgeons and patients; step-by-step details are available in a previous publication . Consensus for core domains was defined as the agreement of different stakeholders (and different groups of the same stakeholder type, i.e., two groups of patients or two groups of physicians) for a similar set, if not the same, core domains. In case of a discrepancy in ratings, we would discuss with various stakeholders to agree to a final common domain set, i.e., consensus. We performed two surveys among 1) orthopaedic surgeons and 2) patients, clinicians, orthopaedic surgeons, and methodologists  that identified seven potential domains for a draft TJR trial core domain set (including joint-specific quality of life). At the OMERACT-2014 meeting in Budapest, a multi-stakeholder group including orthopaedic surgeons, patients, academic researchers and industry scientists, discussed in detail and by consensus endorsed a preliminary core domain set for TJR clinical trials . The TJR clinical trial preliminary core domain set included six core domains, namely, pain, function, patient satisfaction, revision surgery, adverse events and death . Other domains were considered but not endorsed for inclusion in the core set were patient expectations, joint-specific quality of life and generic quality of life . Joint-specific quality of life was thought to be redundant with functional ability, pain and patient satisfaction and therefore not included in core domain set, details of the discussion are provided elsewhere . We previously determined that the next step in this consensus process was to obtain further input from more independent groups from two important stakeholder communities, the orthopaedic surgeons and the patients.
The objective of the current study was to assess whether a broader orthopaedic community will engage in further development of the draft TJR trial core domain set, endorse or propose modification. We performed two separate surveys with three groups of orthopaedic surgeons to obtain a consensus on the draft TJR trial core domain set. We surveyed: (1) the present/past leadership of international orthopaedic societies (IOS) (survey 1); and (2) the members of the American Academy of Orthopaedic Surgeons’ Outcome Special Interest Group (AAOS-Outcome SIG) and the Outcome Research Interest Group of the Orthopaedic Research Society (ORS) at the annual meetings (survey 2), using each groups’ email list.
We surveyed three groups of orthopaedic surgeons, key stakeholders for this consensus, by performing two separate surveys. These surgeons included not only clinicians, but also orthopaedic clinical trialists and leading total joint registry investigators. We met orthopaedic surgeons face to face at the AAOS meeting and subsequently obtained email lists for three groups of orthopaedic surgeons, categorized as follows for two independent surveys: (1) the present/past IOS leadership (survey 1); and (2) the members of the AAOS-Outcomes SIG and the Outcome Research Interest Group of the ORS (survey 2), who attended the respective meetings at the AAOS-2015.
Survey and analyses
We pre-tested a survey questionnaire that queried the importance of six core domains/areas (pain, function, patient satisfaction, revision surgery, adverse event and death) and two optional areas (patient participation in life/social activities, cost) for reporting in joint replacement clinical trials. Ratings were on a 1–9 scale, 1–3 indicating domain of limited importance, 4–6 being important domain, and 7–9 being critically important domain. If both groups of orthopaedic surgeons were to rate each core domain “7–9”, i.e., critical, it would indicate complete consensus, and the TJR core domain set would be considered endorsed by orthopaedic surgeons at this stage. If a majority of the six core domains, but not all, were rated as critical by both orthopaedic surgeon groups, it would indicate incomplete consensus and would lead to discussion with surgeons regarding whether we needed modification of the draft TJR trial core domain set. In case of lack of consensus, modification of core domain set would be done. A similar approach was planned for the future, in case of lack of complete consensus between surgeon and patient surveys for the draft TJR trial core domain set, leading either to its endorsement or modification (separate surveys of the same core domain set in patients) (Singh JA, et al. Patient Endorsement of the Outcome Measures in Rheumatology (OMERACT) Total Joint Replacement (TJR) Clinical Trial Draft Core Domain Set, submitted).
Summary statistics are calculated for the two cohorts of survey participants. We calculated the median (interquartile range) ratings for each of the preliminary core domain/area within each group, the orthopaedic surgeon/registry leadership group and the AAOS-Outcome SIG/ORS orthopaedic surgeon groups.
There were 18 survey participants from the IOS group and 69 participants from the AAOS-Outcomes SIG/ORS outcome groups who completed the questionnaire. Of the survey respondents, respectively, 100 and 78% were male, >90% in both were arthroplasty researcher/surgeons, and 67 and 64% were 55 years and older (Table 1).
The pattern of endorsement and ratings of both core domains and optional domains for measurement in arthroplasty RCTs were similar across the two diverse groups of orthopaedic surgeons (Table 2). All six proposed preliminary core domains of pain, function, patient satisfaction, revision surgery, adverse event and death achieved a median score of 7 or 8 on a 0–9 scale in both groups, i.e., were rated as critically important. The median score for two additional optional domains, patient participation and cost, achieved a median score of 6–6.5 in both groups (Table 2).
Several important comments were provided by survey respondents, which shed further insight into this issue (Table 3). Additional core domains brought up for consideration by 1 respondent each were: caregiver impact; venous thromboembolism occurrence; return to work; quality of life; and range of motion. Some participants commented that the domains of revision (at some specified time/s after TJR) and patient participation (social vs. work) need to be defined clearly.
TJR clinical trials often report outcomes that are heterogeneous and difficult to combine and compare across trials in a systematic review or comparative analysis [4, 5]. This heterogeneity is not only a major barrier to accurate comparison between trials but also an impediment to the development of consensus, which is critical for the collective analysis of the important findings that TJR trials may bring. The various international joint replacement registries have recognized the importance of harmonizing outcome collection and reporting based on expert consensus and have initiated a collaborative approach to establishing a framework . To date, such an effort has not been completed for a core measurement set in TJR clinical trials.
The OMERACT TJR Working Group, an International group with patient partnership, is pursuing this goal through International collaboration with groups such as the AAOS-Outcome SIG, the ORS, the International Society of Arthroplasty Registries (ISAR), and Functional Outcomes in Total Joint Replacement (FORCE-TJR). In the last few meetings, the OMERACT TJR WG has made significant progress in assembling a multi-stakeholder group and completed the literature reviews examining the frequency of reporting of various domains (pain, function, revision etc.) in TJR clinical trials (Richards B, et al. Outcome measures used in arthroplasty trials: Systematic review of the 2008 and 2013 literature, submitted; Wall P, et al. Do outcomes reported in randomised controlled trials of joint replacement surgery fulfil the OMERACT 2.0 Filter? A review of the 2008 and 2013 literature, submitted).
At OMERACT 2014 TJR WG, a preliminary core domain set for TJR clinical trials was developed based on systematic literature reviews, the OMERACT filter 2.0, and two separate surveys at orthopaedic surgery meetings and the OMERACT pre-meeting . This proposed draft TJR core domain set (pain, function, patient satisfaction, revision, adverse events, death, joint-specific quality of life) was discussed with a multi-stakeholder group at OMERACT-2014 TJR WG that included patients who had undergone arthroplasty, orthopaedic surgeons (in an academic setting or community-based practice), psychometricians, physical/occupational therapists, other clinicians and arthroplasty researchers, and finally endorsed with six core domains . This multi-stakeholder OMERACT group helped develop the draft TJR clinical trial core domain set including experts in the development of clinical trial core domain sets and following a well-described methodology based on a solid framework of the OMERACT filer 2.0 [9–11].
In the current study, both groups of orthopaedic surgeons achieved consensus on the six TJR clinical trial core domains (scores of 7 or above) and did not propose any new core domains. These are very important findings, since these Delphi involved two groups of orthopaedic surgeons independent of those who had participated in the development of the preliminary TJR clinical trial core domain set. This is not surprising since this TJR core domain set was developed over several years after multiple systematic reviews and with multi-stakeholder input consisting of orthopaedic surgeons, patients who had undergone arthroplasty and methodologists who have an extensive experience with developing and validating core domain and measurement sets. The OMERACT’s philosophy of a multi-stakeholder involvement from the beginning of the process, and a data driven approach, are key to successfully achieving a broad stakeholder consensus on this draft TJR clinical trial core domain set. This evidence indicates that preliminary TJR core domain set initially developed with multi-stakeholder input and endorsed at OMERACT 2014 meeting was endorsed independently by orthopaedic surgeons.
Core domain sets are those domains that should be reported in every clinical trial for the condition, regardless of the intervention, as they are critical to the conceptualization of the condition/disease. One must not confuse a core domain with the primary outcome for each clinical trial, which is always specific to each research question. In some studies, core domain measures will be primary outcome, since they are the focus of the research question; in other studies, they may be secondary study outcomes. Our work indicates that this draft TJR core domain set is now ready for the next phase, namely, the identification and validation of measures that accurately reflect each core domain, and further dissemination of this TJR core domain set to key stakeholders.
We received qualitative feedback from the surveys, which has helped us with further specification of two core domains. One suggestion was to consider any reoperation, not just revision, due to its relevance to patients- this indicates that the core domain would be revision/reoperation. Another suggestion was to specify whether death was procedure-related vs. not procedure-related. With further refinement of the core domains, these two proposals will be the subject of future WG discussions. It is interesting and reassuring that several of domains brought up one participant each (caregiver impact; venous thromboembolism occurrence; return to work; quality of life; and range of motion) were considered previously. Based on Delphi and detailed discussions with surgeons, patients and other stakeholders, they were recommended not for inclusion in the core domain set, since they were not considered to represent an independent core domain, above and beyond the six domains already included.
We noted that both groups independently rated the additional non-core domains (patient participation, cost) as important but not critical; the median scores were 6 to 6.5, appropriately lower than the core domains, since these are not core domains. Non-core domains are those domains that may be additionally reported in a TJR clinical trial, depending on the clinical question being asked.
Our study findings must be interpreted with caution, since they may not be generalizable to other groups. Our orthopaedic surgeon samples were 78 and 100% male, which may limit the generalizability of the findings to all surgeons. The gender distribution among orthopaedic surgeons is not balanced in general. We developed this draft TJR core domain set for clinical trials; it is possible that with validation and future work, this set or a similar set may be endorsed for observational studies as well.
Our study has several strengths, including use of the same survey in two groups of stakeholders, inclusion of both leaders of orthopaedic societies as well as the group of orthopaedic surgeons and researchers from community and academic practices.
In conclusion, the current study shows further independent endorsement of the draft TJR core domain set for reporting of TJR clinical trials. The next steps in this process, are a wider endorsement of these by OMERACT and/or orthopaedic organizations that we have been collaborating with. This will be followed by the development of a valid, data- and consensus-driven TJR clinical trial core outcome measurement set. Our group will aim to complete this step over the next 2 years. The process will be completed with multi-stakeholder input with a particular emphasis of involvement of patients and TJR surgeons.
American Academy of Orthopaedic Surgeons’ Outcome Special Interest Group
Numeric Rating Scale
Outcome measures in rheumatology
Outcome Research Interest Group of the Orthopaedic Research Society
Special Interest Group
Total Joint Replacement
Total knee arthroplasty
Western Ontario and McMaster Universities Osteoarthritis Index
Kurtz S, Ong K, Lau E, Mowat F, Halpern M. Projections of primary and revision hip and knee arthroplasty in the United States from 2005 to 2030. J Bone Joint Surg Am. 2007;89:780–5.
Healthcare Cost and Utilization Project (HCUP). HCUP Facts and Figures 2009- Section 3: Inpatient Hospital Stays by Procedure. Exhibit 3.1 Most Frequent All-listed Procedures. Rockville: Agency for Healthcare Research and Quality; 2012. Updated February 2012. http://hcup-us.ahrq.gov/reports/factsandfigures/2009/pdfs/FF_2009_section3.pdf. 10 Feb 2014.
Ethgen O, Bruyere O, Richy F, Dardennes C, Reginster JY. Health-related quality of life in total hip and total knee arthroplasty. A qualitative and systematic review of the literature. J Bone Joint Surg Am. 2004;86-A:963–74.
Riddle DL, Stratford PW, Bowman DH. Findings of extensive variation in the types of outcome measures used in hip and knee replacement clinical trials: a systematic review. Arthritis Rheum. 2008;59:876–83.
Riddle DL, Stratford PW, Singh JA, Strand CV. Variation in outcome measures in hip and knee arthroplasty clinical trials: a proposed approach to achieving consensus. J Rheumatol. 2009;36:2050–6.
Singh JA, Dohm M, Sprowson AP, Wall PD, Richards BL, Gossec L, et al. Outcome Domains and Measures in Total Joint Replacement Clinical Trials: Can We Harmonize Them? An OMERACT Collaborative Initiative. J Rheumatol. 2015;42(12):2496–502.
Singh JA, Dohm M. Towards Consensus on the Reporting of Core Outcome Domains in Total Joint Replacement Clinical Trials: The Derivation of the Preliminary Core Outcome Domain Set. J Arthritis. 2016;5:6.
Rolfson O, Wissig S, van Maasakkers L, Stowell C, Ackerman I, Ayers D, et al. Defining an International Standard Set of Outcome Measures for Patients with Hip or Knee Osteoarthritis: Consensus of the International Consortium for Health Outcomes Measurement Hip and Knee Osteoarthritis Working Group. Arthritis Care Res (Hoboken). 2016;68:1631–9.
Boers M, Kirwan JR, Wells G, Beaton D, Gossec L, d’Agostino MA, et al. Developing core outcome measurement sets for clinical trials: OMERACT filter 2.0. J Clin Epidemiol. 2014;67:745–53.
Boers M, Kirwan JR, Gossec L, Conaghan PG, D’Agostino MA, Bingham 3rd CO, et al. How to choose core outcome measurement sets for clinical trials: OMERACT 11 approves filter 2.0. J Rheumatol. 2014;41:1025–30.
Singh JA, Taylor WJ, Simon LS, Khanna PP, Stamp LK, McQueen FM, et al. Patient-reported outcomes in chronic gout: a report from OMERACT 10. J Rheumatol. 2011;38:1452–7.
We thank the orthopaedic surgeons for their time and participation in this survey study.
There is no direct funding for this study. This study was supported by the resources and use of facilities at Birmingham VA Medical Center at Birmingham, Alabama, USA.
Availability of data and materials
These data were collected as a part of this study and were available to us as study Investigators. These data are protected by the HIPAA and patient privacy laws in the United States; no public links are available to these protected health information datasets. These data will be made available to others after appropriate data privacy and human subjects approvals needed by our institution. Requests should be sent to email@example.com.
All authors have read and approved the manuscript. JS: development of study question and protocol. JS, MD: modification and finalization of study protocol. JS, MD, PC: Data collection. JS: Data programming, data analysis. JS, MD, PC: Interpretation of data analysis. JS: First Draft of manuscript. JS, MD, PC: Critical revisions and approval of the final version of the manuscript.
JAS is the co-chair of the OMERACT TJR Working group assessing the outcome measures in arthroplasty and has served as expert/lead on task forces for the specialty societies and the US FDA. Other authors have no other relevant information.
Each author certifies that his or her institution has approved the human protocol for this investigation and that all investigations were conducted in conformity with ethical principles of research.
“The views expressed in this article are those of the authors and do not necessarily reflect the position or policy of the Department of Veterans Affairs or the United States government.”
Names, Degrees and Academic title:
JA Singh, MBBS, MPH Professor of Medicine and Epidemiology.
M Dohm, MD, FAAOS, ABOS Assistant Professor of Orthopaedic Surgery, Assistant Professor of Medicine.
P F Choong, MBBS, MD, FRACS, FAOrthA Professor of Surgery.
JAS has received research grants from Takeda and Savient and consultant fees from Savient, Takeda, Regeneron, Merz, Bioiberica, Crealta and Allergan pharmaceuticals, WebMD, UBM LLC and the American College of Rheumatology. JAS serves as the principal investigator for an investigator-initiated study funded by Horizon pharmaceuticals through a grant to DINORA, Inc., a 501 (c)(3) entity. JAS is a member of the executive of OMERACT, an organization that develops outcome measures in rheumatology and receives arms-length funding from 36 companies; a member of the American College of Rheumatology’s (ACR) Annual Meeting Planning Committee (AMPC); Chair of the ACR Meet-the-Professor, Workshop and Study Group Subcommittee; and a member of the Veterans Affairs Rheumatology Field Advisory Committee. MD receives no direct funding and has no financial conflicts. He is a member of the American Joint Replacement Registry (AJRR) and Chairman of the Data Elements Committee as well as an associate member of the International Society of Arthroplasty Registers (ISAR), is a member of the evidence based practice committee for AAHKS and Chairman of the AAOS Outcome Special Interest Group. PC has received consultancy fees from DePuy, Zimmer and Stryker as part of surgeon design or research teams. PC’s institution has received research grant funding from Allergan, DePuy and Medacta. None of the authors have any non-financial disclosures. JAS is a Section Editor for BMC Musculoskeletal Disorders.
Consent for publication
Ethics approval and consent to participate
This was a study of orthopaedic surgeons participating in the AAOS-Outcomes SIG or the ORS meeting or the IOS group, and not patients. No ethical approval was needed for collecting opinions from orthopaedic surgeons attending special interest group meetings, regarding TJR trial core domains. The surgeons provided implicit consent by voluntarily completing the survey.
About this article
Cite this article
Singh, J.A., Dohm, M. & Choong, P.F. Consensus on draft OMERACT core domains for clinical trials of Total Joint Replacement outcome by orthopaedic surgeons: a report from the International consensus on outcome measures in TJR trials (I-COMiTT) group. BMC Musculoskelet Disord 18, 45 (2017). https://doi.org/10.1186/s12891-017-1409-4
- Core domain
- Clinical trails
- Total Joint Arthroplasty