- Open Access
Exploring the impact of Patient Reported Outcome Measures (PROMs) among orthopaedic surgeons in mainland China: systematic review and survey-based study on hip and knee instruments
BMC Musculoskeletal Disorders volume 22, Article number: 566 (2021)
Patient Reported Outcome Measures (PROMs) are widely used in Europe and North America in a variety of areas including research, clinical governance, clinical registries and insurance ascertainment. The aim of this study was to assess commonly used knee and hip PROMs among Chinese surgeons and to gain an insight into their impact on evaluation of clinical outcomes.
1. A systematic literature search of databases Medline, EMBASE, CINAHL and CNKI was performed from the earliest records to 22/07/2020 for knee instruments and 22/08/2020 for hip instruments, to retrieve Chinese Mandarin cross culturally adapted and validated knee and hip PROMs. 2. An 11-item electronic questionnaire was then designed under four domain categories. The survey was distributed via a ubiquitous online social media platform to orthopaedic surgeons. Responses were collected and analyzed. Output from 1. was used to populate parts of the survey questionnaire.
The systematic online search yielded a total of 41 evaluation instruments, (10 hip and 31 knee); all of which were incorporated as response options. 234 viable questionnaires were retrieved with the largest group representing attending surgeons. 59.0% were familiar with the concept of PROMs among which 78.4% reported to have used PROMs themselves. In order of frequency of use, PROMs were purposed for clinical assessment (55.6%), research (40.7%), health regulation policies (18.6%) and insurance service requirements (10.6%). Implementation was prompted by both departmental (43.4%) and institutional policy (34.5%). 89.4% of PROMs users reported difficulties in the use of PROMs, with major barriers including license fees, limited access, inadequate training and burden of fill-out time (all > 40%).
There is evidence of limited familiarity with knee and hip PROMs among orthopaedic surgeons. Barriers to their use are significant. Development of a Chinese language PROMs database would be helpful.
PROMs are subjective self-reported questionnaires, typically providing information on individual patients’ health status directly without clinician interpretation . The revolution in evidence-based medicine has placed the patient at the center of clinical decision-making, promoting a doctrine of the medical service which is geared at improving patient symptoms, function and quality of life . The advantages of PROMs and their applicability in comparing outcomes from different health providers, identifying strengths and deficiencies in health care delivery, boosting quality improvement, and promoting choice are increasingly evident .
Orthopedic arthroplasty registries have recognized the importance of PROMs and have incorporated general and specific PROMs such as the Oxford Hip Score (OHS), WOMAC (The Western Ontario and McMaster Universities Osteoarthritis Index) and 12-item Short Form Survey (SF-12) as part of patient function and quality of life assessment .
In the UK, where the use of certain locomotor PROMs is mandated by commissioning bodies, for example within the referral pathway for knee replacement , anecdotal evidence suggests PROMs use in primary care is fragmented and unsystematic; suggesting that studies to explore attitudes of clinicians towards PROMs use could be helpful . In China, development of the primary care system is embryonic and does not have a ‘gate-keeping’ function . The Chinese government is expanding the role of primary care for patients with common degenerative conditions. It is therefore to be hoped that general practitioners could become familiar with suitable Chinese-language validated PROMs in the future. However, their current impact is mainly in the secondary care sector. Although PROMs have been developed for use in Traditional Chinese Medicine , their utility in surgery may not be well understood since there is no literature exploring surgeon comprehension or use of PROMs in clinical practice or research in China. In 2019 it was estimated that nearly one million joint replacements were carried out in China with an annual growth rate of 38% .. The aim of this survey was to 1. explore knowledge of and attitudes towards commonly used knee and hip PROMs among Chinese orthopedic surgeons, 2. identify barriers to PROMs use, and 3. gain an insight into their impact on evaluation of clinical outcomes.
Identification of Chinese cross culturally adapted and original Chinese knee and hip PROMs
We performed separate online database searches for Hip and Knee assessment tools on the following databases PubMed/MEDLINE, EMBASE (OVID), CINAHL (EBSCO) and CNKI (mainland Chinese index database) with methodology compliant with previous recommendations . Retrieved reports dated from earliest records till end July 2020. Sensitive filters were designed using MeSH terms and keyword combinations based on previously described strategies for PROMs searches [11,12,13] (Additional file 1 filter); with inclusion criteria as follows: 1. Cross culturally adapted and original Chinese hip and knee PROMs. 2. Tested on the mainland Chinese population. 3. Chinese Mandarin as language of cross-cultural adaptation.
Screening was carried out as a 3-step process (Titles, Abstract and Full texts), each performed by two orthopedic surgeons fluent in the language. Following independent review consensus was reached on articles to be retrieved. After full-text screen and identification of suitable articles, references were manually reviewed to identify missing articles which were not part of the electronic search output.
We designed an 11-item questionnaire in 4 domains: Surgeon details (questions 1 and 2); Surgeon knowledge (questions 3–5); PROMs usage (questions 6–9); PROMs barriers (questions 10 and 11). Questions were modified based on adaptations from previous similar studies . The following items were included in the questionnaire:
Are you an Orthopedic Surgeon?
What level of training are you?
Do you know about PROMs?
Which Knee Scores are you familiar with?
Which Hip Scores are you familiar with?
Have you used Knee or Hip PROMs before?
Which Hip Scores have you used before?
Which Knee Scores have you used before?
For what purpose do/did you use PROMs?
Do you have any difficulties using PROMs?
What difficulties hinder you from using PROMs?
Response options were of mixed format, including binary, multiple choice and free-text options. Results of the systematic search for knee and hip tools were incorporated as response options for questions 4 and 5 respectively, with a capacity for respondents to add other PROMs not on the list. Answers to questions 7 and 8 were open-ended free-text. To increase accuracy of responses, the questions were encrypted with a logic coding whereby a negative answer to question 1,3,6 and 10 prevented display of further questions, automatically ending the questionnaire.
The questionnaire was designed using the online platform WenJuanXing® (Changsha RanXing Information Technology Co., Ltd), and was compatible for smart gadgets. Distribution of the electronic questionnaire was done online via the ubiquitous social media platform the WeChat®, targeting different orthopedic groups. The survey was sent to orthopedic surgeons over a 4-week period from 24 September 2020 to 24 October 2020. Sampling was purposive and intended to capture orthopaedic surgeons of all grades (residents, attending surgeons, vice directors and directors). An online network of Chinese Orthopedic Association conference attendees (2019), a provincial city-level orthopedic association group and university health-system affiliated hospital orthopedic department groups were engaged. Onward dissemination of the survey among fellow orthopedic colleagues was encouraged. Responses were automatically collected and indexed on the same online platform. Analysis and graphics were designed with Microsoft Excel® 2018 (Microsoft, Redmond, WA, USA). Statistical display and analysis was descriptive.
The systematic online search yielded a total of 41 instruments, (10 hip and 31 knee). The PRISMA chart of the review process is illustrated in Figs. 1 and 2. All of these were included in the surgeon questionnaire as multiple-choice options (Additional file 2 Table, Additional file 3 table references).
A total of 305 responses were retrieved and 234 were viable following exclusion of 71 respondents who were not orthopedic surgeons. Current training level was Resident 18.4%, Attending Surgeon 45.7%, Vice Director 29.1% and Director 6.8%.
The flowchart of responses is shown in Fig. 3 together with the list of PROMs which were identified in response to questions 4,5,7 and 8. The reasons for using PROMs and barriers to their introduction are also documented. 59.0% of respondents were familiar with the concept of PROMs, and the most familiar knee PROMs were: The Knee injury and Osteoarthritis Outcome Score (KOOS), International Knee Documentation Committee (IKDC), Oxford Knee Score (OKS),The Osteoarthritis of Knee and Hip Quality of Life (OAKHQOL), Hospital for Special Surgery Total Knee Replacement Expectations Survey (HSS-TKRES), Lower Extremity Function Scale (LEFS). Among those who knew about PROMs, the above six PROMs achieved recognition rates of over 35% (Fig. 3). Most familiar hip PROMs were: Hip disability and Osteoarthritis Outcome (HOOS), Oxford Hip Score (OHS), Copenhagen Hip and Groin Outcome Score (HAGOS), International Hip Outcome Tool (SC-iHOT-33), Osteoarthritis of Knee and Hip Quality of Life (OAKHQOL), Hospital for Special Surgery Hip Replacement Expectations Survey (HSS-THRES). Among those who knew about PROMs, the above six PROMs achieved recognition rates over 20% (Fig. 3). Among all surgeons, 46.2% reported to have used PROMs themselves. Named PROMs that had been used included Oxford Knee Score (OKS) and the Harris Hip Score (HHS). Surgeons use PROMs primarily for clinical assessment and research, as mandated by either departmental or hospital policy (all > 30%). Major difficulties using PROMs were high license fees, patient time burden, insufficient training and difficulties obtaining Chinese language versions (all > 40%).
The majority of Orthopedic surgeons in smaller cities in Mainland China work in general orthopedic wards addressing a wide variety of musculoskeletal disorders. Most doctors work at attending surgeon level, so respondent seniority profile is likely to be representative of Chinese orthopedic surgeons overall. Most surgeons (59.0%) were familiar with the concept of PROMs. The Knee injury and Osteoarthritis Outcome Score (KOOS) and the Hip disability and Osteoarthritis Outcome (HOOS) were the most familiar in each group. Hip PROMs were generally less well known than Knee PROMs. This might reflect the predominance of knee over hip osteoarthritis in east Asian populations . Although 46.2% of respondents reported use of PROMs before, very few were able to name those they had used. Possible reasons for lack of engagement with this question is that previous use of PROMs may have been unstructured and spasmodic. The VAS score is very commonly used as a pain assessment tool in orthopedic units; a similar study on common foot and ankle PROMs also reported that VAS was most commonly used . We had previously performed an online literature search on CNKI and Wangfangdata (Mainland China databases) to identify publications using knee PROMs for 2018 (unreported data). Results identified the Lysholm, VAS and the International Knee Documentation Committee (IKDC) scores as most commonly occurring in literature. The IKDC was also amongst the six most familiar knee articles in this survey.
The commonest response as to why surgeons choose to use PROMs was for clinical assessment and for research, however the difficulties encountered in describing PROMs used suggests that deep familiarity and frequent usage is unlikely. Responses suggest institutional encouragement for the use of PROMs, although their use in routine follow-up is uncertain. Responses as a whole indicate very limited national or provincial policy mandating the use of PROMs (this option checked by10.6% of respondents).
Arthroplasty Registries have existed since 1975 and have adopted PROMs as part of endpoint assessment highlighting patient quality of life and functionality . In several countries PROMs are incorporated into actual clinical practice with central funding of online databases such as FORCE-TJR for collection interpretation and analysis of data in the orthopedic community . No schemes exist yet in Mainland China as reflected in the current study’s surgeon responses.
The majority of PROMs users reported difficulties (89.4%). We did not allow respondents who did not use PROMs to answer this question, so it is possible that these individuals had also experienced difficulties which prevented their use at all. Prominent issues include: high license fees, difficulty finding Chinese versions of the PROMs, and long time spent administering PROMs. Although most PROMs are free-to-use others require fees; this may force clinicians to find alternative instruments. Most PROMs are originally designed in English and thus need to be rigorously translated before implementation in other language settings. Many translated versions are found on English language websites owned by the originator, adding to the difficulties of access and comprehension. Although a wide variety of tools have been cross culturally adapted and validated in mainland Chinese populations, some articles have excluded the Chinese language PROM from publication due to license fee requirements. This may have encouraged the development of multiple translated versions of original PROMs . Surgeons reported that time constraints in completing PROMs was one of the main difficulties. A previous study on general practitioners’ views on the use of PROMs also found this was the main barrier . Thus, the administrative burden is an important aspect that has to be considered to facilitate their clinical use.
Some linguistic regions have developed online PROMs libraries, for example in Spain via BibliPRo (https://www.bibliopro.org) which provides surgeons with search options and download portals. These ensure quality control via assessment of available PROMs using tools such as the Evaluation Measurement of Patient Reported Outcomes (EMPRO) and Consensus Based Standards for the Selection of Health Measurement Instruments (COSMIN) [18,19,20]. Similar initiatives should be undertaken in Mainland China to ascertain and assemble high quality PROMs for use by future arthroplasty registries, for research and in clinical management. Studies on modalities of PROMs collection will be useful to provide evidence on efficient time saving methods to improve PROMs returns and reduce patient and administrative burden through the advent of electronic PROMs delivered online. Surgeon and patient education will be vital to improve compliance and understanding of PROMs, setting the stage for wider clinical use.
We used purposive and snowball sampling so as to ascertain orthopaedic surgeons in different geographical settings and of differing seniority. However this valid sociological method to sample hard-to-reach groups is a non-random method and has inherent selection biases which are uncontrolled. The results of our survey on attitude may therefore not represent all views or even representative views of orthopaedic surgeons although we have no reason to believe they do not.
A wide variety of lower limb arthroplasty tools have been translated into Chinese, with the most familiar hip and knee scores being the Hip Disability and Osteoarthritis Outcome (HOOS) and the Knee injury and Osteoarthritis Outcome Score (KOOS). Use of PROMs is generally for research and clinical assessment. Most surgeons reported difficulties in accessing PROMs. Improvements in PROMs utility may rely on a Chinese language database to allow better access and quality control of available PROMs.
Availability of data and materials
The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.
Patient Reported Outcome Measures
Oxford Hip Score
The Western Ontario and McMaster Universities Osteoarthritis Index
12-item Short Form Survey
The Knee injury and Osteoarthritis Outcome Score
International Knee Documentation Committee
Oxford Knee Score
The Osteoarthritis of Knee and Hip Quality of Life
Hospital for Special Surgery Total Knee Replacement Expectations Survey
Lower Extremity Function Scale
Hip disability and Osteoarthritis Outcome
Copenhagen Hip and Groin Outcome Score
International Hip Outcome Tool
Osteoarthritis of Knee and Hip Quality of Life
Hospital for Special Surgery Hip Replacement Expectations Survey
Evaluation Measurement of Patient Reported Outcomes
Consensus Based Standards for the Selection of Health Measurement Instruments
Food and Drug Administration (FDA). Guidance for industry on patient-reported outcome measures: use in medical product development to support labeling claims. Fed Regist. 2009;74(235):65132–3.
International Alliance of Patients’ Organizations(IAPO): What is Patient-centered Health Care? A Review of Definitions and Principles. 2nd ed. London: IAPO; 2007. p. 1–34.
Black N. Patient reported outcome measures could help transform health care. BMJ. 2013;346(jan28 1):f167. https://doi.org/10.1136/bmj.f167.
Wilson I, Bohm E, Lübbeke A, Lyman S, Overgaard S, Rolfson O, et al. Orthopaedic registries with patient-reported outcome measures. EFORT Open Rev. 2019;4(6):357–67. https://doi.org/10.1302/2058-5241.4.180080 PMID: 31210973; PMCID: PMC6549110.
NHS Scarborough and Ryedale Clinical Commissioning Group (CCG) Knee replacement referral criteria 2015 [Available from: https://www.scarboroughryedaleccg.nhs.uk/data/uploads/rss2/orthopaedics/2015_10_13__update/knee-replacement-march-2015.pdf.
Turner GM, Litchfield I, Finnikin S, Aiyegbusi OL, Calvert M. General practitioners’ views on use of patient reported outcome measures in primary care: a cross-sectional survey and qualitative study. BMC Fam Pract. 2020;21(1):14. https://doi.org/10.1186/s12875-019-1077-6.
Xu J, Powell-Jackson T, Mills A. Effectiveness of primary care gatekeeping: difference-in-differences evaluation of a pilot scheme in China. BMJ Glob Health. 2020;5(8):e002792. https://doi.org/10.1136/bmjgh-2020-002792 PMID: 32792410; PMCID: PMC7430328.
Yan-yan L, Ru-nan C, Jing-jing Y, Chang-rong Y. Research process in patient-reported outcomes at home and abroad. Modern Prev Med. 2013;40(12):2268–79.
Yanyan B, Cheng K, Xiao C, Xisheng W. Reports and analysis of amount of hip and knee arthroplasty in China from 2011 to 2019. Chin J Orthop. 2020;40(21):1453–60. https://doi.org/10.3760/cma.j.cn121113-20200320-00177.
de Vet HCW, Terwee CB, Mokkink LB, Knol DL. Measurement in Medicine: A Practical Guide; Cambridge University Press; 2011.
Arias-de la Torre J, Puigdomenech E, Valderas JM, Evans JP, Martín V, Molina AJ, et al. Availability of specific tools to assess patient reported outcomes in hip arthroplasty in Spain. Identifying the best candidates to incorporate in an arthroplasty register. A systematic review and standardized assessment. PLoS One. 2019;14(4):e0214746. https://doi.org/10.1371/journal.pone.0214746 PMID: 30934024; PMCID: PMC6443164.
Harris K, Dawson J, Gibbons E, Lim CR, Beard DJ, Fitzpatrick R, et al. Systematic review of measurement properties of patient-reported outcome measures used in patients undergoing hip and knee arthroplasty. Patient Relat Outcome Meas. 2016;7:101–8. https://doi.org/10.2147/PROM.S97774 PMID: 27524925; PMCID: PMC4966645.
Terwee CB, Jansma EP, Riphagen II, de Vet HCW. Development of a methodological PubMed search filter for finding studies on measurement properties of measurement instruments. Qual Life Res. 2009; 18: 1115–1123. https://doi.org/10.1007/s11136-009-9528-5 PMID: 19711195.
Zwiers R, Weel H, Mallee WH, Kerkhoffs GMMJ, van Dijk CN. Ankle platform study collaborative – science of variation group. Large variation in use of patient-reported outcome measures: a survey of 188 foot and ankle surgeons. Foot Ankle Surg. 2018;24(3):246–51. https://doi.org/10.1016/j.fas.2017.02.013 Epub 2017 Mar 9.
Fransen, Marlene & Bridgett, Lisa & March, Lyn & Hoy, Damian & Penserga, Ester & Brooks, Peter. The epidemiology of osteoarthritis in Asia. Int J Rheumatic Dis. 2011;14:113–21. https://doi.org/10.1111/j.1756-185X.2011.01608.x.
15.Ayers, David C. MD; Implementation of Patient-reported Outcome Measures in Total Knee Arthroplasty. J Am Acad Orthop Surg. 2017;25:S48–50. https://doi.org/10.5435/JAAOS-D-16-00631.
Ngwayi JRM, Tan J, Liang N, Porter DE. Reliability and validity of 3 different Chinese versions of the Oxford knee score (OKS). Arthroplasty. 2020;2(1):31. https://doi.org/10.1186/s42836-020-00049-1.
Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, et al. The COSMIN checklist for assessing the methodo- logical quality of studies on measurement properties of health status measurement instruments: an international Delphi study. Qual Life Res. 2010;19(4):539–49. https://doi.org/10.1007/s11136-010-9606-8.
Terwee CB, Mokkink LB, Knol DL, Ostelo RW, Bouter LM, de Vet HC. Rating the methodological quality in systematic reviews of studies on measurement properties: a scoring system for the COSMIN checklist. Qual Life Res. 2012;21:651–7 doi: 10.1007/s11136–011–9960-1.
Valderas JM, Ferrer M, Mendivil J, et al. Development of EMPRO: a tool for the standardized assessment of patient-reported outcome measures. Value Health. 2008;11(4):700–8. https://doi.org/10.1111/j.1524-4733.2007.00309.x.
This study received no funding support.
Ethics approval and consent to participate
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Hip and Knee search Filters. PubMed/MEDLINE filter with mesh terms and keywords to search for scores adapted for the Chinese population.
Available Chinese adapted Instruments for Knee and Hip assessment. List of instrumeents and year published for the available Chinese adapted and designed Knee and Hip scores.
Chinese knee and hip score references. List of references for all the scores included in table.
About this article
Cite this article
Ngwayi, J.R.M., Tan, J., Liang, N. et al. Exploring the impact of Patient Reported Outcome Measures (PROMs) among orthopaedic surgeons in mainland China: systematic review and survey-based study on hip and knee instruments. BMC Musculoskelet Disord 22, 566 (2021). https://doi.org/10.1186/s12891-021-04459-3
- Hip PROMs
- Knee PROMs
- Systematic literature review