Future role of virtual and augmented reality in thoracic surgery: a narrative review

Introduction

Virtual reality (VR) and augmented reality (AR) are rapidly evolving technologies that allow users to interact with dynamic, computer-generated environments within a three-dimensional (3D) space. VR produces a computer-generated, fully virtual environment in which the user is completely immersed, often through a head-mounted display to allow interaction with objects and spaces as though they were real. AR, in contrast, superimposes computer-generated images, data, or other digital elements onto the real-world view, thereby enhancing rather than replacing the surrounding environment. Both technologies rely on binocular displays to create the perception of depth and spatial relationships, which are essential to an immersive 3D experience.

As these technologies have become more accessible, their integration within surgical practice has expanded, both within thoracic surgery (1) and also other subspecialties, including—but not limited to—orthopedics, spinal surgery, and interventional radiology (2-4). In thoracic surgery specifically, there is a growing body of literature exploring these technologies and their potential, particularly in relation to preoperative planning and simulation (1,5-7). However, their intraoperative applications remain limited and largely unexplored despite growing interest, improving technologies, and decreasing costs. Thoracoscopic surgery, whether robotic-assisted (RATS) or video-assisted (VATS) presents unique spatial and technical challenges all while employing a display monitor for visualization, making it a compelling candidate for VR/AR applications.

In this review, we highlight future potential roles for these technologies within thoracoscopic surgery, particularly in the intraoperative setting, where continued innovation has the potential to improve safety, efficiency, and outcomes. A thorough systematic review by Arjomandi Rad et al. summarized many of the current applications of AR/VR in thoracic surgery to date (1); herein, we focus more on potential future roles that, to our knowledge, have not yet been described in the thoracic surgery literature. We present this article in accordance with the Narrative Review reporting checklist (available at https://vats.amegroups.com/article/view/10.21037/vats-25-19/rc).

Methods

The search strategy summary is shown in Table 1. The authors conducted literature searches using the PubMed and Google Scholar databases, using combinations of the following search terms: “virtual reality”, “augmented reality”, “thoracic surgery”, “thoracoscopic surgery”, “VATS”, and “robotic surgery”, and “situs inversus”. All resultant articles in English language were considered. The authors conducted literature searches between 2/15/2025–4/21/2025. The earliest article that was appropriately related to the applicability of VR/AR in the surgical treatment of thoracic disease was published in 2007 (8); we therefore considered articles published between 2007–4/21/2025 (ending on the date of the authors’ final search). We initially identified 31 full-text, peer-reviewed articles that appeared to be relevant to the present topic; after full-text assessment, 14/31 articles were deemed not relevant to the scope of this narrative review. Disagreements regarding study inclusion were resolved by discussion between authors until consensus was reached.

Discussion

Current applications of VR/AR in thoracic surgery

Presently, VR and AR in the context of thoracic surgery are more widely adopted outside of the operating room, most notably in the settings of surgical training, simulation, and preoperative planning. VR-based simulation is an increasingly popular educational tool, with multiple commercial simulators available (5), and studies demonstrating the value among trainees in thoracic surgery (9-11). In the preoperative setting, VR allows surgeons to visualize patient-specific anatomy based on cross-sectional imaging: three-dimensional (3D) reconstructions from computed tomography (CT) imaging can be rendered to enable viewing of structures and spatial relationships from any angle, which is especially valuable for cases involving anatomical aberrancy or cases where the feasibility of a complete resection is questioned (5,8,12). While this form of preoperative planning has been deemed by many surgeons as useful, its adoption has been limited, and it has obvious limitations in that images and reconstructions are based on the inflated lung, which is different than the deflated lung encountered in most thoracoscopic procedures.

Within the operating room itself, there are a limited number of reports describing the application of VR and/or AR during thoracoscopic surgery (1). For example, Ujiie et al. (2024) integrated VR-generated 3D reconstructions of the pulmonary artery, vein, and bronchial anatomy within the DaVinci console display for assistance in a robotic-assisted lingular segmentectomy, reporting that their utilization of VR in preoperative planning, with the immediate availability of the virtual reconstruction, was instrumental in identifying unexpected anatomical variations (5). Similarly, Sardari Nia et al. (2019) utilized a double-monitor setup with 3D vision to display VR-generated, 3D reconstructions of patient-specific anatomy during 25 VATS cases, reporting good correspondence between virtual intraoperative anatomy, and detection of anatomical variations in 15.4% of cases (13). Nevertheless, we did not find any reports of the intraoperative use of AR for thoracoscopic surgery, wherein a virtual reconstruction would be superimposed onto the real-world environment displayed by robotic or VATS platforms.

Outside of thoracoscopy, AR has been utilized in the operating room as an adjunct for localization of ribs for chest wall procedures in pediatric patients: one case report by Spijkerboer and colleagues (2022) describes AR-localization of a non-palpable chest wall tumor to guide resection in real time (14), while a more recent series expanded this approach across 8 chest wall resections, combining landmark-based registration techniques with a holographic overlay to enable accurate tumor localization (6).

Tumor localization in the deflated lung

A desirable application for VR/AR in thoracoscopic surgery would be the ability to indicate the location of a tumor directly onto the video feed, thereby guiding dissection and parenchymal division with greater precision. Perhaps the most significant challenge that needs to be surmounted in its potential development is that preoperative CT scans are obtained with the lung fully expanded, whereas intraoperatively the lung is deflated to varying degrees. This discrepancy causes the intraoperative location of a tumor to vary from its location based on preoperative imaging. Consequently, current approaches for lung nodule localization rely on palpation or lung marking techniques with indocyanine green, methylene blue, or radio-opaque markers, often using navigational or robotic-assisted bronchoscopy or CT-guided percutaneous techniques. In the future, VR/AR systems might overcome this challenge by creating 3D reconstructions of a collapsed lung using data from the expanded lung on CT. A model that proposes an expected tumor location based on relative locations to other structures such as hilar structures, distance from the fissures, or relative location to different parts of the chest wall could potentially be leveraged to help surgeons localize tumors more effectively. If this technology was created and optimized, its incorporation into routine practice could foreseeably increase the efficiency of operations, possibly obviating the need for separate bronchoscopic marking procedures and, in select patients, reducing the number of invasive procedures required to achieve resection.

Localization of important structures

The concept of using VR/AR for intraoperative tumor localization extends beyond the lung parenchyma, and could be used to highlight other anatomy such as vasculature, airways, lymph nodes, and other critical structures. For example, an AR overlay in video-assisted or robot-assisted thoracoscopic surgery indicating the precise location of critical vascular structures could increase the efficiency and safety of certain operations. This type of overlay seems to be gaining some traction in spinal and neurosurgical procedures through the use of head-mounted devices, guiding interventions such as pedicle screw placement and percutaneous vertebroplasty (3). Percutaneous procedural guidance using head-mounted devices is also being explored by interventional radiologists, most notably for percutaneous thermal ablation of tumors (4,15). Within thoracic surgery, there is active research in the realm of using AR and visual tools to assist surgeons in appreciating chest wall anatomy (7), which we can likely expect to see becoming explored in chest wall cases such as with Nuss procedures for pectus excavatum, chest wall reconstruction cases, and the like.

Operating in a mirrored setting & alternating handedness

An interesting clinical scenario that the authors came upon was an esophagectomy performed on an individual with situs inversus totalis. This is a rare, but well-recognized congenital condition in which a patient’s anatomy is mirrored across the sagittal plane. This condition may go undiagnosed into adulthood, and often presents incidentally (16). Operating in a mirrored setting is inherently awkward and unintuitive, as surgeons are confronted not only with mirrored laterality and an unfamiliar visual field, but also with the challenge of adapting their handedness to the orientation of the anatomy and dissection. Case reports of operations on patients with situs inversus have often commented on the operative challenges posed by the mirrored laterality, and the potential increase in operative risks, though these have not been definitively demonstrated in the literature (16,17). VR and AR, in the era of robotic surgery, are uniquely poised to transform this experience by mirroring laterality in both the console display and its handedness, in order to simulate an anatomically “normal” operation (Figure 1). If the imaging shown to the operating surgeon and the motions performed by the surgeon were both mirrored, an operation with this pathology would be made significantly more facile.

Figure 1 Robotic esophagectomy performed on a patient with situs inversus totalis. Images on the left show the thoracoscopic view of the dissection performed in the actual patient; images on the right are horizontally mirrored to illustrate the potential application of VR in robotic thoracoscopic surgery in a case where laterality matters. From top to bottom, these images show the dissection of the azygos vein, preparation of the gastric conduit along the greater curvature of the stomach, and the preoperative CT scan. Images courtesy of Dr. Benjamin Lee (Weill Cornell Medicine, New York, NY, USA). CT, computed tomography; VR, virtual reality.

Although situs inversus is rare, the broader concept of mirroring the operative field and instrument control could apply to a variety of surgical scenarios where anatomic orientation or surgeon ergonomics are challenging, such as unusual tumor positions, complex reoperations, or constrained thoracoscopic access. For example, the concept of alternating handedness can be extended to scenarios in which anatomical constraints prevent a surgeon from using their dominant hand for precise dissection. In such cases, temporarily mirroring the visual field and instrument control might enhance comfort, efficiency, and precision.

Strengths and limitations

This narrative review offers a speculative, forward-looking perspective on the future roles of VR and AR in thoracic surgery, a topic that remains in its early investigative phase. Being relatively novel technologies within the field of surgery overall, the existing literature is sparse and predominantly exploratory, with studies focusing on proof-of-concept or pilot applications, rather than large-scale clinical implementation. Furthermore, the technological capabilities of VR/AR in general appear to be evolving rapidly, limiting generalizability of current literature. Our narrative aims to highlight potential avenues for future application while acknowledging that substantial research and validation are required before these tools can be widely and meaningfully adopted.

Conclusions

Virtual and augmented reality are emerging technologies with the potential to meaningfully improve upon existing strategies within thoracoscopic surgery. While current applications have mostly remained within the domains of preoperative planning and surgical simulation, the role of VR/AR during actual operations is enticing and worthy of exploration. Future roles within thoracic surgery could aid in real-time intraoperative localization and has the potential to improve ergonomics and safety. Continued innovation and research in these fields will be important to advance the potential of these technologies in thoracic surgery.

Acknowledgments

We would like to thank Dr. Benjamin Lee (Weill Cornell Medicine, New York, NY, USA) for providing the original operative photos shown in Figure 1.

Footnote

Reporting Checklist: The authors have completed the Narrative Review reporting checklist. Available at https://vats.amegroups.com/article/view/10.21037/vats-25-19/rc

Peer Review File: Available at https://vats.amegroups.com/article/view/10.21037/vats-25-19/prf

Funding: None.

Conflicts of Interest: Both authors have completed the ICMJE uniform disclosure form (available at https://vats.amegroups.com/article/view/10.21037/vats-25-19/coif). The authors have no conflicts of interest to declare.

Ethical Statement: The authors are accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved. Patient consent was not obtained as that there are no identifiable attributes in the image or text.

Open Access Statement: This is an Open Access article distributed in accordance with the Creative Commons Attribution-NonCommercial-NoDerivs 4.0 International License (CC BY-NC-ND 4.0), which permits the non-commercial replication and distribution of the article with the strict proviso that no changes or edits are made and the original work is properly cited (including links to both the formal publication through the relevant DOI and the license). See: https://creativecommons.org/licenses/by-nc-nd/4.0/.

References

1. Arjomandi Rad A, Vardanyan R, Thavarajasingam SG, et al. Extended, virtual and augmented reality in thoracic surgery: a systematic review. Interact Cardiovasc Thorac Surg 2022;34:201-11. [Crossref] [PubMed]
2. Sun P, Zhao Y, Men J, et al. Application of Virtual and Augmented Reality Technology in Hip Surgery: Systematic Review. J Med Internet Res 2023;25:e37599. [Crossref] [PubMed]
3. Ghaednia H, Fourman MS, Lans A, et al. Augmented and virtual reality in spine surgery, current applications and future potentials. Spine J 2021;21:1617-25. [Crossref] [PubMed]
4. Finos K, Datta S, Sedrakyan A, et al. Mixed reality in interventional radiology: a focus on first clinical use of XR90 augmented reality-based visualization and navigation platform. Expert Rev Med Devices 2024;21:679-88. [Crossref] [PubMed]
5. Ujiie H, Chiba R, Yamaguchi A, et al. Developing a Virtual Reality Simulation System for Preoperative Planning of Robotic-Assisted Thoracic Surgery. J Clin Med 2024;13:611. [Crossref] [PubMed]
6. van der Woude R, Fitski M, van der Zee JM, et al. Clinical Application and Further Development of Augmented Reality Guidance for the Surgical Localization of Pediatric Chest Wall Tumors. J Pediatr Surg 2024;59:1549-55. [Crossref] [PubMed]
7. Arensmeyer J, Bedetti B, Schnorr P, et al. A System for Mixed-Reality Holographic Overlays of Real-Time Rendered 3D-Reconstructed Imaging Using a Video Pass-through Head-Mounted Display-A Pathway to Future Navigation in Chest Wall Surgery. J Clin Med 2024;13:2080. [Crossref] [PubMed]
8. Hu Y, Malthaner RA. The feasibility of three-dimensional displays of the thorax for preoperative planning in the surgical treatment of lung cancer. Eur J Cardiothorac Surg 2007;31:506-11. [Crossref] [PubMed]
9. Solomon B, Bizekis C, Dellis SL, et al. Simulating video-assisted thoracoscopic lobectomy: a virtual reality cognitive task simulation. J Thorac Cardiovasc Surg 2011;141:249-55. [Crossref] [PubMed]
10. Jensen K, Hansen HJ, Petersen RH, et al. Evaluating competency in video-assisted thoracoscopic surgery (VATS) lobectomy performance using a novel assessment tool and virtual reality simulation. Surg Endosc 2019;33:1465-73. [Crossref] [PubMed]
11. Whittaker G, Aydin A, Raveendran S, et al. Validity assessment of a simulation module for robot-assisted thoracic lobectomy. Asian Cardiovasc Thorac Ann 2019;27:23-9. [Crossref] [PubMed]
12. Ujiie H, Yamaguchi A, Gregor A, et al. Developing a virtual reality simulation system for preoperative planning of thoracoscopic thoracic surgery. J Thorac Dis 2021;13:778-83. [Crossref] [PubMed]
13. Sardari Nia P, Olsthoorn JR, Heuts S, et al. Interactive 3D Reconstruction of Pulmonary Anatomy for Preoperative Planning, Virtual Simulation, and Intraoperative Guiding in Video-Assisted Thoracoscopic Lung Surgery. Innovations (Phila) 2019;14:17-26. [Crossref] [PubMed]
14. Spijkerboer KGP, Fitski M, Siepel FJ, et al. Augmented reality-guided localization of a chest wall tumor in a pediatric patient. Eur J Cancer 2022;170:103-5. [Crossref] [PubMed]
15. Gadodia G, Yanof J, Hanlon A, et al. Early Clinical Feasibility Evaluation of an Augmented Reality Platform for Guidance and Navigation during Percutaneous Tumor Ablation. J Vasc Interv Radiol 2022;33:333-8. [Crossref] [PubMed]
16. Sakran N, Stier C, Parmar C, et al. Bariatric and Metabolic Surgery in Patients with Situs Inversus: a Systematic Review. Obes Surg 2023;33:3628-35. [Crossref] [PubMed]
17. Chaouch MA, Jerraya H, Dougaz MW, et al. A Systematic Review of Laparoscopic Cholecystectomy in Situs Inversus. J Invest Surg 2021;34:324-33. [Crossref] [PubMed]