Enhancing the quality of international orthopedic medical mission trips using the blue distinction criteria for knee and hip replacement centers

Abstract

Background

Several organizations seek to address the growing burden of arthritis in developing countries by providing total joint replacements (TJR) to patients with advanced arthritis who otherwise would not have access to these procedures. Because these mission trips operate in resource poor environments, some of the features typically associated with high quality care may be difficult to implement. In the U.S., many hospitals that perform TJRs use the Blue Cross/Shield’s Blue Distinction criteria as benchmarks of high quality care. Although these criteria were designed for use in the U.S., we applied them to Operation Walk (Op-Walk) Boston’s medical mission trip to the Dominican Republic. Evaluating the program using these criteria illustrated that the program provides high quality care and, more importantly, helped the program to find areas of improvement.

Methods

We used the Blue Distinction criteria to determine if Op-Walk Boston achieves Blue Distinction. Each criterion was grouped according to the four categories included in the Blue Distinction criteria— “general and administrative”, “structure”, “process”, or “outcomes and volume”. Full points were given for criteria that the program replicates entirely and zero points were given for criteria that are not replicated entirely. Of the non-replicated criteria, Op-Walk Boston’s clinical and administrative teams were asked if they compensate for failure to meet the criterion, and they were also asked to identify barriers that prevent them from meeting the criterion.

Results

Out of 100 possible points, the program received 71, exceeding the 60-point threshold needed to qualify as a Blue Distinction center. The program met five out of eight “required” criteria and 11 out of 19 “informational” criteria. It scored 14/27 in the “general” category, 30/36 in the “structure” category, 17/20 in the “process” category, and 10/17 in the “outcomes and volume” category.

Conclusion

Op-Walk Boston qualified for Blue Distinction. Our analysis highlights areas of programmatic improvement and identifies targets for future quality improvement initiatives. Additionally, we note that many criteria can only be met by hospitals operating in the U.S. Future work should therefore focus on creating criteria that are applicable to TJR mission trips in the context of developing countries.

Background

Improved longevity in developing countries has led to the rise of chronic diseases, including arthritis [1, 2]. In developed countries, total joint replacement (TJR) is often used to address symptomatic advanced arthritis. Total hip and knee replacements have been shown to enhance quality of life by improving function [3] and decreasing pain [4]. Although TJRs are cost effective in developed countries [5, 6], the high cost of these procedures have made financing them difficult in developing countries. Despite the high costs of most surgical interventions, the WHO and other health organizations have called for renewed focus on building developing countries’ surgical capacities [7, 8]. To meet these needs, organizations such as Operation Walk (Op-Walk) Boston have annual mission trips to provide pro-bono total knee and total hip replacements to people from developing countries while concurrently building surgical capacity by educating local physicians and surgical teams about TJR [9].

Organizations that provide surgical care abroad seek to provide the highest quality care possible, although the effectiveness of these short-term medical mission programs has recently been debated [10]. Ideally, all medical mission trips would offer services with quality that is equal to the services offered in developed countries. Although some attempts have been made to standardize surgical procedures during medical mission trips [11, 12], quality criteria have not been established for TJR medical mission trips. One set of guidelines commonly used as a benchmark of TJR quality in the U.S. is the Blue Cross/Shield’s “Blue Distinction Center for Knee and Hip Replacement” criteria, which provide quality benchmarks for a range of program features including structural elements, processes, provider certifications, and reporting. Although the Blue Distinction criteria were developed to assess TJR program quality in developed countries, these criteria might also help medical mission groups to evaluate and improve the care they provide in developing countries.

This report provides the first attempt to evaluate the quality of care offered by medical missions based on TJR quality standards from developed countries. We also demonstrate that this kind of evaluation can help medical mission organizations identify areas of programmatic improvement.

Methods

Setting

The Dominican Republic is a small country (population 10,056,000) in the Caribbean Sea that shares the island of Espanola with Haiti. As of 2011, the country’s per capita income was approximately $9,300 [13]. Citizens have access to a free state health care plan, which provides basic primary care coverage, and private clinics provide most specialty care. The nation’s capital, Santo Domingo, is home to several private hospitals, including the Hospital General de la Plaza de la Salud. In 2008, when the Op-Walk Boston team made their first trip to the D.R., the hospital performed fewer than 20 TJRs annually, though this number has grown to more than 100 cases annually. Other private hospitals throughout the country also provide hip and knee replacements to patients who are able to pay for the procedures’ high costs, but information regarding the number of joint replacements and the outcomes of these other joint replacement programs is not available.

Operation walk Boston team

Op-Walk Boston is part of the national Operation Walk organization [14]. The Boston team has made annual service trips to the D.R. since 2008. The team consists of approximately 50 individuals, including orthopedic surgeons, anesthesiologists, internists, physical therapists, physician assistants, surgical and medical nurses, operating room personnel, medical students, and other staff. The Op-Walk Boston team works closely with their Dominican colleagues at La Hospital General de La Plaza de La Salud in Santo Domingo to identify low-income Dominican patients with severe joint disease; the team provides pro-bono knee and hip replacements for these patients during its annual trips.

Data collection and analysis

Using the Blue Cross/ Shield’s selection criteria for Blue Distinction Centers for Knee and Hip Replacements, we determined if Op-Walk Boston’s joint replacement program meets the Blue Distinction criteria (scores at least 60 out of 100 possible points) [15]. The study’s main author compiled the Blue Cross/Shield’s criteria and independently reviewed each criterion with two of Op-Walk Boston’s program directors during in-person interviews. The directors independently responded to each criterion with “meet”, “do not meet”, or “unsure”. For criteria that produced conflicting results, the criteria were sent to an independent person familiar with the specific criteria in question and this third party’s response served as a tie-breaker. Similarly, for all criteria that lacked responses from both directors, the study’s main author redirected the question to an independent team member who had the most knowledge of that criterion.

Full points were awarded for criteria that the program replicates exactly and zero points were given for criteria that are not replicated. If a criterion was not applicable outside of the United States, zero points were awarded and the criterion was labeled “not applicable”. For all criteria that are not replicated, we interviewed a leader from each clinical and administrative team to see if they compensate for failing to meet the criterion by introducing an alternative strategy or process to enhance quality. We also interviewed these key informants to see if there are barriers that prevent them from adopting certain criteria.

We classified the criteria as “General/Administrative”, “Process”, “Structure”, and “Outcomes and Volume” categories to follow widely-used frameworks for quality assessment [16] and to match the subcategories established in the Blue Distinction criteria. The scores from each category were graphed to represent visually the percentages of points that were met, not met, or somehow compensated for in each category. Points marked as “informational” represent criteria that are not formally scored in the Blue Cross/Shield’s criteria but are still important when evaluating joint replacement programs. Points marked as “required” are deemed essential for achieving the Blue Cross/Shield’s distinction.

Results

General criteria for all blue distinction centers

In the general criteria section, the program received 14 out of a possible 27 points (Table 1). The program lost four points because it lacks formal conflict of interest policies and it lost seven points because it does not collaborate with several U.S.-based quality improvement organizations. Two points were lost because it does not participate in the Surgical Care Improvement Project, although the program does compensate by following best practice surgical care guidelines.

Table 1 Blue distinction criteria, points awarded, accommodations made to meet the criteria, and barriers to criteria’s implementation for general and structure criteria

There are two “required” and three “informational” general criteria; the Op-Walk Boston program meets one of the required criteria and one of the informational criteria. The program did not meet one of the required criteria because the host hospital is not accredited by a CMS-deemed national accreditation organization. The two unmet informational criteria relate to using a Surgical Care Improvement Project (SCIP) database to produce procedure specific performance reports and to tracking FDA-recalled prostheses and contacting patients with these prostheses. The hospital does not have a SCIP database and it also has difficulty tracking patients, which makes it difficult to contact patients whose prosthesis are recalled.

Structure

In the structure category, the program was awarded 30 out of 36 possible points (Table 1). The only criterion that the program did not fully meet involved reporting to surgical quality improvement registries and databases. The program also lost three points because it lacks three out of eight required multi-disciplinary team members: psychiatrists and psychologists, pain management specialists, and dedicated case managers. Despite lacking psychiatrists and psychologists and formally trained pain management specialists, the program compensates by having doctors and nurses work directly with patients to address their mental health needs and by having well-trained anesthesiologists who commonly provide pain management services.

Aside from the scorable criteria, there were also two criteria that were listed as “required” and seven listed as “informational”. The program met both of the required criteria and six of the informational criteria. It did not meet one informational criterion because it lacks pain management specialists who have subspecialty certification in pain management.

Process

The program received 17 out of 20 possible points in the process category (Table 2). The program lost one point because it lacks standardized practices for case management and discharge planning. It lost an additional point because the program does not monitor care transitions for patients who are discharged to another care setting, although it does compensate by having an electronic medical record system that helps to track patients over time. It also lost a point because it lacks formal protocols that ensure patients’ op-notes and discharge summaries are made available to their PCPs upon discharge.

Table 2 Blue distinction criteria, points awarded, accommodations made to meet the criteria, and barriers to criteria’s implementation for process and outcomes and volume criteria

There are six informational criteria in the process category, and the program met three of them. One unmet criterion involved using shared decision making processes with patients. Another unmet criterion involved tracking op-notes and discharge forms to ensure the patient’s PCP receives the documents. The final unmet informational criterion involved using the Blue Cross/ Shield’s case management team to track transitions of care.

Outcomes and volume

In the outcomes and volume category, the program received 10 out of 17 possible points (Table 2). The program lost three points because the host facility does fewer than 250 TJRs annually, and it lost an additional two points because the host facility does fewer than 500 TJRs annually. It lost two additional points because the host hospital does fewer than 50 TJR revisions annually.

There are two required and two informational criteria in the outcomes and volume category. The program met both of the required criteria and did not meet either of the informational criteria. One informational criterion requests that the program publically report average lengths of stay for patients who need hip and knee revisions and the other criterion involves tracking selection, administration, and discontinuation of prophylactic antibiotics for knee replacements.

Overall evaluation

The Op-Walk Boston program met enough criteria to score 71 of 100 possible points, exceeding the 60-point threshold needed to qualify as a Blue Center of Distinction (Figure 1). The program met five of the possible eight “required” criteria and 11 out of the possible 19 “informational” criteria.

Figure 1

Weighted score breakdowns for criteria that are met or not met in each of the four quality categories.

Common barriers were identified that prevented some criteria from being met. First, many of the criteria require registration and participation in organizations that only exist in the United States or other developed countries. Second, many criteria were not met because they require frequent patient contact and PCP follow up. Third, some criteria require outcomes and patient tracking through online databases that are not currently available in many hospitals in the D.R.

Discussion

In this study, we evaluated the quality of the Op-Walk Boston’s TJR medical mission trip using the Blue Cross/ Shield’s Blue Distinction Criteria for Knee and Hip Replacement Centers. This analysis showed that the program scored 14/27 in the general/administrative scoring category, 30/36 in the structure category, 17/20 in the process category, and 10/17 in the outcomes and volume category. (Figure 1) Overall, the program scored 71/100, which exceeds the 60 point threshold for Blue Distinction designation. We spoke with the program’s respective clinical teams about accommodations they make to compensate for unmet criteria. If the program did not meet a criterion and does not compensate in some way, we inquired about the barriers that prevent the program from meeting the criteria.

Although there is a growing body of quality assessment literature for medical mission trips in other fields [12], our study is the first to evaluate programmatic quality in the context of an international TJR medical mission trip. Coupled with previously published outcomes data from the Op-Walk Boston program that showed patients’ pain and function greatly improved after their TJR surgeries [17], it seems that the Op-Walk Boston team provides high quality care to the patients it serves in the D.R. Similar international joint replacement programs should note that having a dedicated research team that monitors outcomes can provide data to improve clinical care and is therefore essential in ensuring continued quality improvement in short-term international medical missions [18]. Our research team’s operation accounted for eight out of the 71 points the program received, and it also helped to meet three of the informational criteria.

In addition to demonstrating quality, the process of evaluating the Op-Walk Boston program based on the Blue Distinction criteria helped generate ideas for quality improvement initiatives. For example, the program lost a point because it does not use patient navigators. Since Op-Walk Boston’s patients often have complex social needs and may have difficulties navigating the health care system, initiating a patient navigation program may help with the hospital experience and subsequent recovery. The program also lost a point because it does not evaluate patient’s discharge needs prior to their hospital admission. This is another area we have now identified for improvement. We plan to discuss with Op-Walk’s medical, nursing and rehabilitation colleagues in the Dominican Republic culturally acceptable ways of amplifying the patient's voice in decision making. Some of the barriers we identify could be overcome with greater funding. We have discussed our findings with administrative and clinical leaders in the Dominican Republic so that the observed deficiencies can be considered for funding in subsequent budgeting processes.

When weighted scoring breakdowns were graphed according to the Blue Cross/Shield’s four major quality categories—general/administrative, structure, process, and outcomes and volume—the program’s general/administrative and outcomes and volume categories emerged as the weakest. (Figure 1) These categories contained unique obstacles that are difficult or impossible for the program to overcome. In the general/administrative category, for example, the program lost five out of a potential 27 points because the criteria requested that the program report to agencies that are primarily based in the U.S. and do not currently operate in the D.R.; these criteria are therefore not applicable to Op-Walk Boston’s TJR program.

In the outcomes and volume category, the program lost 7 points mainly because its host hospital does fewer than 250 TJRs and 50 revisions procedures annually. When Op-Walk Boston first began working in the D.R., its host hospital performed fewer than 20 TJRs annually; volume has expanded to over 100 annually. Although it does not meet the 250 annual TJR threshold requested by the Blue Cross/Shield criteria, the host hospital has therefore made significant progress in terms of volume.

Although the Blue Distinction criteria for joint replacement centers were created as quality standards for U.S. hospitals, using these criteria as a benchmark for evaluating TJR medical mission trips can help demonstrate care quality and identify areas of quality improvement. Since some aspects of the Blue Distinction criteria require organizations to report to US-based quality improvement organizations, future work should alter the existing criteria so that these organizations can earn points for reporting to equivalent international quality improvement organizations or provide waivers for organizations that operate in countries without equivalent quality improvement organizations. Furthermore, some Blue Distinction criteria require investment in patient navigators or expensive health care infrastructure, which is difficult because cost is a common barrier for most international medical missions. The criteria should therefore be redesigned so that they can be implemented with a level of investment more congruent with the resource capacity in the country being evaluated. Having a revised set of criteria could help medical mission trips better evaluate their own programs and allow them to enhance the care they provide. In the meantime, other international TJR mission programs might consider using the Blue Distinction criteria—or other similar criteria—to evaluate and improve their own programs, as our study demonstrates that these analyses can facilitate programmatic improvement.

Limitations

This study’s data were potentially subject to observer bias, as data were collected by an investigator rather than a research assistant blinded to the study’s hypotheses and objectives. Anticipating this bias, we used data elements from the Blue Cross/Shield’s excellence criteria that were objective, binary, and subject to little interpretation. Furthermore, the data for this study was collected from a small number of respondents who provided key information to evaluate if the program meets or does not meet the criteria; it is therefore possible that responses would have been more heterogeneous if more people were surveyed but this variability should be limited by the objective and binary nature of the Blue Cross/Shield’s criteria.

Conclusion

Op-Walk Boston qualified for Blue Distinction. Our analysis highlights areas of programmatic improvement and identifies targets for future quality improvement initiatives. Additionally, we note that many criteria can only be met by hospitals operating in the U.S. Future work should therefore focus on creating criteria that are applicable to TJR mission trips in the context of developing countries.

Ethical approval

This study was approved by the IRBs at both the Brigham and Women’s Hospital in Boston and the Hospital General de la Plaza de la Salud in Santo Domingo.