Metaverse, virtual reality and augmented reality in total shoulder arthroplasty: a systematic review

Abstract

Purpose

This systematic review aims to provide an overview of the current knowledge on the role of the metaverse, augmented reality, and virtual reality in reverse shoulder arthroplasty.

Methods

A systematic review was performed using the PRISMA guidelines. A comprehensive review of the applications of the metaverse, augmented reality, and virtual reality in in-vivo intraoperative navigation, in the training of orthopedic residents, and in the latest innovations proposed in ex-vivo studies was conducted.

Results

A total of 22 articles were included in the review. Data on navigated shoulder arthroplasty was extracted from 14 articles: seven hundred ninety-three patients treated with intraoperative navigated rTSA or aTSA were included. Also, three randomized control trials (RCTs) reported outcomes on a total of fifty-three orthopedics surgical residents and doctors receiving VR-based training for rTSA, which were also included in the review. Three studies reporting the latest VR and AR-based rTSA applications and two proof of concept studies were also included in the review.

Conclusions

The metaverse, augmented reality, and virtual reality present immense potential for the future of orthopedic surgery. As these technologies advance, it is crucial to conduct additional research, foster development, and seamlessly integrate them into surgical education to fully harness their capabilities and transform the field. This evolution promises enhanced accuracy, expanded training opportunities, and improved surgical planning capabilities.

Introduction

The metaverse [1] is a virtual environment that merges physical and virtual realities, empowering users and avatars to interact within a technologically advanced ecosystem [2]. This setting can harness immersive technologies like augmented reality (AR), virtual reality (VR), and artificial intelligence (AI) to provide realistic experiences to individuals across the globe in several different contexts [3].

Computer-driven approaches have been used in many fields of surgery, such as ophthalmology, urology, and general surgery, to assist the surgeon in improving preoperative planning or perfecting surgical execution [4,5,6]. However, the role of Metaverse, AR, and VR in orthopedics is yet to be adequately elucidated, and their implementation in shoulder surgery is yet to be thoroughly investigated, particularly in the context of shoulder arthroplasty [7]. Several technological innovations are routinely implemented in orthopedic surgery [8], such as robotic surgery, 3D-printed patient-specific instrumentation, and navigation tools with tracking visualized on monitors [9].

The most recent advancement to improve intraoperative execution involves the utilization of computer-assisted navigation instruments. This navigation system offers real-time visual feedback during surgery, enabling precise alignment of the surgeon’s instruments with the preoperative plan. This alignment is achieved by integrating a line-of-sight camera and trackers attached to the surgical instruments and the scapula [10].

While traditional navigation techniques have been extensively utilized in orthopedic procedures, including shoulder arthroplasty, the emerging technologies of AR and VR represent a significant advancement in the field. Notably, there is currently a dearth of studies investigating the application of AR and VR specifically within the context of shoulder arthroplasty, highlighting an area ripe for exploration.

The increasing interest in AR and VR in orthopedics and trauma comes as no surprise, given that orthopedic surgical procedures frequently demand visual data from pre- and intra-operative medical imaging. These procedures involve mechanical actions like screw or implant placements, osteotomies, and deformity corrections, all of which can benefit from visualizing rigid relationships within AR environments. Advancements in haptic feedback, real-time imaging, and AI can further enhance surgical planning, precision, and patient outcomes. Collaborative virtual environments within the metaverse can foster interdisciplinary discussions and enable remote mentoring and guidance for orthopedic surgeons specializing in shoulder procedures [11]. Thus, such technical tasks appear to be predisposed to applications of AR and VR [12].

Also, revolutionary changes in medical education, surgical training, and interventional procedures occur within the metaverse [13]. In this domain, these technologies have the potential to significantly enhance the field of orthopedic surgery by providing a secure and readily accessible supplement to orthopedic surgical training, all without direct involvement of patients [14]. Surgical care and education are increasingly relying on VR, AR, and, ultimately, the newest metaverse applications. Nevertheless, the technologies themselves need further development in this direction, and, at present, it remains challenging to ascertain the extent to which these skills effectively translate into the clinical setting.

The aim of this systematic review is to provide an overview of the current knowledge on the role of the metaverse, AR, and VR in the context of total shoulder arthroplasty.

A comprehensive review of the applications of the metaverse, augmented reality, and virtual reality in in-vivo intraoperative navigation, in the training of orthopedic residents, and the latest innovations proposed in ex-vivo studies was conducted.

Materials and methods

Search strategy

The initial search strategy was organized according to the PICO (Population, Intervention, Comparison, Outcome) structure. Studies that reported outcomes of patients with indications (P) for reverse total shoulder arthroplasty (rTSA) or anatomical total shoulder arthroplasty (aTSA) (I) treated with a computer-assisted intraoperative navigation system were included. Also, studies reporting on orthopedics residents (P) who received VR or AR-based training (I) were included. Cadaver or Computer-based studies (P) reporting outcomes regarding the latest applications of AR or VR on total shoulder arthroplasty (I) were also considered.

Clinical and functional outcomes and questionnaires for each group were reported (C) to evaluate treatment outcomes after each intervention (O).

Two independent reviewers (A.N., A.L) performed article screening using the following research order: title and abstract followed by full article screening. The same reviewers then performed data extraction. In both cases, differences were reconciled by mutual agreement. In case of disagreement, a third reviewer (Longo UG) was consulted for consensus.

Literature search

A systematic review was performed using the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines. Medline, EMBASE, Scopus, and CINAHL bibliographic databases were searched using the following string: ((metaverse OR augmented reality OR virtual reality)) AND arthroplasty).

The search was performed by two authors (A.L., A.N.) from the inception of the database to August 2023. Additional studies were searched among reference lists of selected papers and systematic reviews.

Eligibility criteria

The outcomes assessed for patients treated with intraoperative computer-assisted rTSA or aTSA included: the mean number of screws and the mean screw length, the average surgical time, the number and type of augmented baseplates that were exploited, the mean glenoid version and inclination (in its preoperative, planned and postoperative values and the deviation from planned to postoperative glenoid version and inclination. Complications and revisions were also reported.

The following parameters were extracted from the studies that reported on orthopedics residents training with AR or VR and from in-vitro studies: the aim of the study, sample size, the instrumentation design, the study results, and conclusions.

To report these variables, peer-reviewed articles of each level of evidence according to the Oxford classification were included. Considering the authors’ proficiency in various languages, articles in English, Italian, French, and Spanish were screened.

Only studies utilizing either computer-assisted intraoperative navigation for rTSA or aTSA were considered. Patients undergoing revision surgery or concomitant procedures were excluded. No exclusion criteria were set regarding the surgical indication or follow-up. Technical notes, letters to editors, and instructional courses were excluded.

Also, only studies reporting outcomes regarding VR- or AR-based training in total shoulder arthroplasty of orthopedic surgical residents were included. Even though they included AR- or VR-based protocols, studies focusing on arthroscopic training were not considered.

Outcomes of interest

Data was extracted into predefined tables divided according to intervention.

Tables for intraoperative navigated aTSA and rTSA include a demographics table (Table 1), and two outcomes tables (Table 2 and 3.).

Table 1 Navigated total shoulder arthroplasty: demographics

Table 2 Navigated total shoulder arthroplasty: outcomes

Table 3 Navigated total shoulder arthroplasty: outcomes

Data from studies focusing on orthopedic surgical residents are reported in Table 4.

Table 4 Augmented reality-based training

Data from in-vitro studies reporting the most recent VR and AR applications in 3D models and cadaver specimens are summarized in Table 5.

Table 5 In vitro studies

General study characteristics extracted were Author, Year of Publication, Type of Study, Level of Evidence (LOE), Intervention, Sample Size, Instrumentation Design, Implant Design, and Last Follow-up.

Outcome measures were extracted from the final follow-up. Mean values and standard deviations were extracted. Depending on the availability of this data from each included study, a selection of these outcomes was included in the tables.

Methodological quality assessment

The Risk of Bias (RoB 2) tool for Randomized Trials, the Robins-I tool for case-control studies, and the Joanna Briggs Institute Critical Appraisal Tool for Case-Series were used to assess the quality of each study. Two reviewers independently evaluated selected articles (A.L, B.G.) and reviewed by a third in case of disagreement (Longo UG).

Results

Study selection

The literature search identified 359 articles from scientific databases and 27 from registers. Duplicate removal resulted in the exclusion of 114 studies, leaving 2 articles for screening.

At the final screening, 22 articles met the selection criteria and were included in the review. The PRISMA flowchart of the literature search is reported in Fig. 1.

Fig. 1

PRISMA flowchart

Fig. 2

Risk of bias assessment for randomized control trials

Fig. 3

Risk of bias assessment for case-control studies

Fig. 4

Risk of bias assessment for case series studies

Study characteristics

The LOE of each of the included studies was: 4 level I Randomized Control Trials [15,16,17,18], 9 level III Retrospective Case-Control Studies [19,20,21,22,23,24,25,26,27], 4 level IV Retrospective Case-Series Studies [28,29,30,31], 3 level V Basic Science cadaver studies [32,33,34] and 2 level V Proof of Concept studies [4, 35].

1701 patients treated with rTSA or aTSA from 14 studies [15, 19,20,21,22,23,24,25,26,27,28,29,30,31] were included in the review. 793 patients were treated with navigated rTSA or aTSA, while 908 were treated with standard, non-navigated rTSA or aTSA.

Indications for rTSA and aTSA, whether navigated or non-navigated, included rotator cuff arthropathy, osteoarthritis, massive rotator cuff tears, proximal humeral fractures, osteonecrosis, inflammatory arthropathy, dislocation arthropathy, rheumatoid arthritis, and post-traumatic arthritis. Only one patient in a single study [28] underwent rTSA as a two-stage revision procedure.

The arthroplasty implants included the Equinoxe implant (Exactech, USA), the Eclipse anatomical implant (Arthrex, USA), the Aequalis Reverse implant (Wright Medical Group, USA), and the Delta Extend reverse implant (DePuy Orthopedics, ENG).

Fifty-three orthopedics surgical residents and doctors receiving VR-based training for rTSA were also included in the review. 46 were orthopedics residents from junior to senior years, and 7 were expert orthopedic surgeons. Twenty-seven (23 residents and 4 experts) received VR-based training for rTSA, while 26 (23 residents and 3 experts) were allocated to the cadaver-based training control groups. Data was collected from 3 RCTs [16,17,18].

In the three studies [32,33,34] reporting the latest VR and AR-based rTSA applications, 48 fresh-frozen human cadaver shoulders were implanted with the glenoid baseplate via intraoperative navigation integrated with head-mounted displays. Two studies focused on navigated rTSA coupled with a head-mounted display, while a third study exploited a novel robotic platform for glenoid guidewire placement.

Two proof of concept studies [4, 35] reported outcomes following navigated rTSA coupled with the Microsoft Hololens 1 and Hololens 2 devices, used in 19 3D phantom scapulae. The Wright Medical Group Aequalis Reversed Implant (Wright Medical Group, USA) and the BF Glenoid Trabecular Metal System (Zimmer Biomet, USA) were implanted, respectively.

Demographics of patients undergoing navigated and non-navigated rTSA and aTSA are reported in Table 1. Demographics for trainees receiving VR-based training and for in-vitro studies are reported in Tables 4 and 5, respectively.

Quality of evidence

All the included RCTs were judged as “low risk of bias.” Four RCCs were also identified as “low risk of bias,” the remaining three were judged as having a “moderate risk of bias.” CS studies were overall of good quality [36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,51,52]. The Proof of Concept and Basic Science studies were also of adequate quality, even though it was not possible to comment directly on their methodological quality due to the absence of an objective quality assessment measure.

The risk of bias assessments for RCTs, CCs, and CSs are reported in Figs. 2 and 3, and 4, respectively.

Surgical outcomes

Five studies [19, 20, 23, 24, 30] reported the mean number of screws used in their cohorts, while four studies [20, 21, 23, 28] reported the mean length of the screws used. The mean surgical time was reported by seven studies [15, 23, 25, 26, 28, 30, 31]: the longest time for the navigated and non-navigated cohorts was found by Sasaki et al. [19,20,21,22,23,24,25,26, 29].

Nine articles also reported the number of augmented baseplates used [28].

Complications and revisions were also reported by six articles [19, 27,28,29,30,31]. Common complications included glenoid loosening, persistent pain unexplained by mechanical causes, infection, and intraoperative fractures.

Their preoperative, planned, and postoperative values also reported mean glenoid version and inclination. The mean deviation from the planned and executed glenoid version and inclination were also reported when present in the included articles.

Surgical outcomes for patients undergoing navigated and non-navigated rTSA and aTSA are reported in Table 2and 3.

VR-based training

One study [16] compared training outcomes for rTSA procedures with iVR platform (PrecisionOS, Canada) as compared with cadaver laboratories among junior orthopedics residents. The VR platform was comprised of a 3D visual tool, auditory cues and handheld controllers for haptic feedback and position tracking. Six residents received the VR-based training and six were enrolled in the control cadaver-based training group. They found no statistically significant differences in written knowledge score, Global Rating Scale (GRS) score, time to completion of assessment, or post-training written knowledge score after implantation of the Reverse Shoulder Augmented Baseplate System (Zimmer Biomet, USA).

A second RCT [18] aimed at determining whether VR training would lead to improved surgical skills in performing rTSA compared to an instructional video in orthopedic surgery residents. Nine residents received the VR-based training and nine were enrolled in the control cadaver-based training group. They found that the VR-trained group had significantly improved Objective structured assessment of technical skill (OSATS) scores as well as higher verbal questioning scores after a single training session.

A third study [17] involved 12 VR-trained residents and surgical experts and 11 residents and experts as controls. They utilized the Glenoid Exposure Model (PrecisionOS, Canada) coupled with a head-mounted display and with haptics tools and found that the immersive VR group completed the cadaveric glenoid exposure task faster as well as demonstrating superior OSATS instrument handling scores compared with the control group.

The outcomes from studies focusing on VR-based training are reported in Table 4.

In-vitro studies

Two proof of concept studies [4, 35] involving phantom 3D scapular models were included. They aimed to demonstrate a proof-of-concept solution for delivering AR guidance during the placement of k-wires to position the glenoid component in reversed shoulder arthroplasty, employing the Microsoft HoloLens 1 and HoloLens 2 systems. The first one [4] reported that the average standard deviation (SD) ± error between the planned and achieved entry point was 2.4 ± 0.7 mm. The average SD ± error between the planned k-wire orientation was 3.9° ± 2.4°. The other study [35] showed that the mean 3D deviation angle of the ten placed wires measured 2.7° ± 1.3° and that the mean deviation to the entry point of the ten placed target wires measured 2.3 mm ± 1.1 mm.

Three cadaver studies [32,33,34] were included. They involved twelve, twelve, and twenty-four fresh-frozen shoulders, respectively. They showed that AR-based systems demonstrate accuracy levels consistent with the technology platforms currently employed in shoulder arthroplasty when evaluated in a simulated cadaveric trial.

Outcomes from in-vitro cadaveric and proof of concept studies are reported in Table 5.

Discussion

The main finding of this systematic review is that intraoperative computer-assisted navigation can attain accuracy levels consistent with the standard technology platforms employed in shoulder arthroplasty. Furthermore, this review shows that VR-based training in rTSA results in comparable if not improved outcomes in surgical skill acquisition in orthopedics residents compared to traditional training protocols. Also, the included cadaveric and proof of concept studies demonstrated that utilizing a navigated AR system through a head-mounted display results in minimal deviation between planned and postoperative values. Furthermore, this system offers precise data regarding the variance between intraoperative and postoperative values.

The integration of emerging technologies such as virtual reality, augmented reality, and the metaverse has ushered in a transformative era in the field of orthopedic surgery [53]. These innovative approaches are shaping the landscape of surgical education and hold substantial clinical relevance within orthopedics, particularly in shoulder surgery [54].

At present, VR is widely recognized for its capacity to develop surgical training simulators and aid in preoperative planning, while AR appears to be a more promising tool for intraoperative purposes [55].

AR use was described as early as 2007 when Ortega et al., who assessed the effects and potential advantages of a heads-up device in spine surgery [56]. Since then, it has been demonstrated that AR could be applied to a wide spectrum of orthopedic procedures, such as tumor resection, fracture fixation, and components alignment in total joint arthroplasty [57].

By projecting 3D models of anatomical structures onto the surgeon’s field of vision, AR can aid in preoperative planning, implant positioning, and intraoperative navigation. Surgeons can visualize patient-specific anatomical landmarks and instrumental paths, ensuring precise alignment during joint replacements and spinal surgeries [58]. AR also enables real-time feedback and guidance, reducing the risk of errors and improving surgical outcomes. Furthermore, AR-based remote collaboration allows experienced surgeons to guide and support less experienced colleagues, enhancing surgical training and fostering interdisciplinary collaboration.

From a technical perspective, the main challenge that must be tackled to make AR a practical instrument for surgery is ensuring the precision of calibration between the virtual content shown by the headset and the actual surroundings. In the context of shoulder replacement, the accurate positioning of the glenoid component has been revealed to be one of the most relevant causes of early revision surgery [15, 21, 23, 30, 59]. To decrease the risk of postoperative aseptic glenoid loosening, understanding the morphology and orientation of the glenoid is a key issue that surgeons must face. Numerous factors have been considered when assessing glenoid stability, including bone density, glenoid morphology, baseplate position, screw length, quantity of peripheral screws, screw angular orientation, and central peg length [60, 61].

The introduction of CT-based preoperative planning software has arguably transformed the mindset of surgeons. Numerous authors have demonstrated that such software enhances a surgeon’s ability to achieve the desired positioning of the glenoid component [28]. However, relying on preoperative 2D analyses has been questioned in terms of accuracy [62,63,64].

With navigation, the central component of computer-assisted orthopedic surgery systems empowers orthopedic surgeons to precisely monitor and intuitively visualize surgical instruments in real-time within the context of anatomical structures. The human-machine interface, an essential element of image-guided orthopedic navigation systems, is a platform for merging preoperative and intraoperative images from various modalities and three-dimensional models, streamlining operative planning and navigation. The surgeon’s control over the baseplate’s position in terms of version, inclination, rotational alignment, and height is key to enhancing baseplate stability on the native glenoid. Nevertheless, aside from baseplate orientation and bone factors, the number and length of peripheral screws used for primary fixation also play a crucial role in long-term stability [65,66,67,68].

A recent systematic review showed that the navigation system increased efficiency in reducing the number of screws necessary for fixation per patient. However, the system’s ultimate clinical and economic impact could not be determined in their study [60].

It has been demonstrated that computer-assisted navigation reduces the deviation of the postoperative component position from the preoperative blueprint in cadaveric studies and in the clinical setting [15, 22, 69,70,71,72].

However, while intraoperative navigation has demonstrated enhanced accuracy and precision in glenoid baseplate implantation, there is currently no evidence in the literature to confirm whether these improvements have resulted in better clinical outcomes and reduced complication rates [29, 73]. A recent study showed lower rates of complications and revisions in the navigation group compared to the standard non-navigated procedures. However, it failed to identify increased improvement in range of motion and functional outcome scores compared to the navigated cohort [19].

Another significant factor in glenoid fixation is the number and length of baseplate screws. Before the advent of computer navigation, the capacity to accurately position longer screws was hindered by the difficulty of visualizing the screw’s trajectory due to the absence of clear visual bony reference points. Studies have indicated that increasing the number of screws reduces the likelihood of baseplate displacement, while extending the length of screws may also serve as an effective alternative [20, 65]. A retrospective case-control study showed that computer navigation results in the use of fewer and longer baseplate screws, suggesting that these results may decrease scapular spine stresses and allow for maintained bone stock [20].

While traditional navigation methods have been foundational in guiding surgical procedures, there is a growing recognition of the potential of AR and VR to further enhance surgical precision and improve patient outcomes. Indeed, AR and VR may represent the next evolutionary step beyond traditional navigation techniques. However, it is important to acknowledge that the transition from navigation to AR/VR is not linear, and each technology offers unique advantages and challenges.

AR can combine the advantages of preoperative planning and intraoperative navigation at a low-cost [54]. Following preoperative planning and data transfer to the head-mounted device, the only required intraoperative step is the registration using an optical tracking marker. This surface tracking method eliminates the need for intraoperative imaging, thereby reducing radiation exposure. Kriechling et al. were the first to assess the accuracy and feasibility of guidewire positioning for the placement of glenoid components using AR [35]. The initial outcomes following AR implementation to shoulder replacement surgery were also confirmed by Ponce et al. [74]. Recently, it has been shown that guidewire positioning navigation for placing glenoid components using AR is viable and precise in both cadaver specimens and 3D phantom models [4, 32, 33, 75].

One question is whether AR can replace or improve computer-assisted navigation or robotic-assisted total joint arthroplasty in everyday clinical settings [34]. According to the authors, these novel processes have great potential for transferability to other orthopedic applications in arthroplasty and beyond. As of now, there are no documented clinical applications of AR specifically in shoulder arthroplasty. This underscores the pioneering nature of research in this area and the need for further investigation to explore the potential benefits of AR and VR technologies in improving surgical outcomes in shoulder arthroplasty.

Orthopedic surgical training is also undergoing a paradigm shift [76]. In orthopedic surgical training, the metaverse can provide a collaborative and immersive environment where surgeons, residents, and experts worldwide can interact and learn together [77]. Trainees can participate in virtual surgical conferences, attend live-streamed surgeries, and engage in multidisciplinary discussions. The metaverse offers opportunities for networking, sharing knowledge, and accessing a vast repository of surgical resources. Additionally, the metaverse can facilitate the development of AI-driven surgical assistants, allowing trainees to practice complex procedures with virtual colleagues or receive real-time guidance from virtual mentors [18]. The next logical step would be to systematically employ metaverse, AR, and VR in a training setting. By enabling precise hand-eye coordination, VR fosters the development of surgical skills and has been shown to improve performance in orthopedic procedures such as joint replacements, fracture fixations, and arthroscopic surgeries.

Results have shown that VR-based training significantly reduces surgical errors and enhances surgical proficiency among trainees [78]. Additionally, VR-based simulators offer objective performance metrics, enabling trainees to track their progress and identify areas for improvement.

In a recent investigation, the utilization of AR was assessed for instructing medical students in the placement of acetabular cups for total hip arthroplasty, using a phantom pelvis as the training model [79]. The study revealed that participants exhibited comparable levels of accuracy in their training, whether instructed by an expert surgeon or through AR. Consequently, the authors concluded that the AR approach could be a valuable educational tool, highlighting that certain arthroplasty skills can be acquired without direct supervision [80]. . A recent systematic review [81] has shown that VR-trained residents performed surgery faster and with fewer errors than those trained traditionally. Nonetheless, it has also been shown that VR training significantly improves surgical performance and reduces errors [78].

While it has been demonstrated that AR could offer advantages in training orthopedic residents, it would be intriguing to explore the extent to which AR could truly enhance the learning experience for orthopedic trainees. Furthermore, investigating the learning curve in this context appears to be a promising avenue that warrants further research [82]. However, the training-based application of VR is yet to be fully validated.

The strengths of the present systematic review lie in its novelty: to the authors’ knowledge, this is the first study that provides a comprehensive review of the literature focusing on the applications of AR and VR, as intraoperative computer-assisted navigation, and on the future endeavors that lie in the educational field and technological advancements such as head-mounted displays. Additionally, as per the intraoperative navigation, only primary rTSA or aTSA were included to provide homogeneity of the cohort and improve outcome validation. This review also benefits from using numerous RCTs and including studies with low or moderate risk of bias.

However, there are also limitations associated with the work, including the lack of a meta-analysis, which was not performed given data heterogeneity. Furthermore, indications for total shoulder arthroplasty were not set as exclusion criteria, nor was a minimum follow-up. These limit the validity of the results, particularly on the long-term assessment. Also, the sample size of the VR-based training and cadaveric studies is limited, leaving room for future validation.

Conclusions

Virtual reality, augmented reality, and the metaverse are transforming the landscape of orthopedic surgery. These technologies provide immersive and interactive platforms that enhance surgical training, improve precision, and advance patient care. By offering realistic simulations, objective feedback, and remote collaboration, virtual reality, augmented reality, and the metaverse hold great promise for the future of orthopedic surgery. As these technologies evolve, further research, development, and integration into surgical education are essential to maximize their potential and revolutionize the field.