How AR Remote Assistance Can Aid Surgery

Augmented reality for telehealth

Augmented reality (AR)—a set of technologies that integrates digital 3D content into a person’s visual field and allows them to interact with that content—is being used for a wide range of health applications, from training and education to neurodiagnostics and surgical navigation.

There is excitement around enhancing telehealth with AR. A recent market report covering AR in healthcare estimates that there will be more than 2 million patients and healthcare workers utilizing this technology by 2025, and there are a growing number of peer-reviewed publications that describe AR enabled telehealth for purposes such as home-based telerehabilitation. However, these applications are limited and generally still in the early design and pilot testing phases.

AR enabled remote assistance for surgical procedures

One segment of the telehealth market in which AR technology could make a significant impact is remote assistance (remote assist) for surgery. With AR enabled remote assist, surgeons can receive real-time intraoperative guidance from other physicians, surgical specialists, or medical device/technology experts. Such guidance can be critical when a surgeon is presented with a complex procedure and may benefit from additional technical skills or expertise or during procedures that involve devices or implants that necessitate input from an industry representative. If a procedure is unplanned and time-sensitive, it may not be feasible for a remote expert to travel to the site of the operation or to transport the patient to another facility. Remote assist capabilities can alleviate these constraints, particularly in rural settings, where surgical resources might be limited.

How does AR enabled remote assist work?

A surgeon operating on a patient can use an AR enabled headset like the one developed by Magic Leap, combined with live-streamed video that is supported by high-definition cameras. This allows a remote expert to see what is happening from the surgeon’s point of view (i.e., “see what I see”). With an AR enabled headset, the remote expert can virtually label, pin and draw directly onto the patient’s anatomy to better guide the surgeon in real time. While the remote expert views the surgery on a monitor, the surgeon’s use of the AR enabled headset allows the remote expert to see 3D renderings of a patient’s anatomy and to apply markings onto those 3D models. Even if the surgeon moves around the operating room, the markings for details such as an incision point or device placement remain anchored in place for the surgeon.

Benefits of AR enabled remote assist and current use

The result of this fusion of the physical and virtual worlds, and subsequent enhanced understanding of the position of the user within that world can potentially optimize surgical decision-making, improve precision, and reduce errors. AR enabled remote assist can also reduce costs, as a remote expert does not need to physically travel to the operating room and coordinate scheduling with the surgeon. Cost savings can also be achieved through improved precision and fewer complications for the patient.

There are already some examples of this technology being used. For example, spine surgeons have been advised remotely on screw entry point and trajectory, lateral positioning and cage placement.

Magic Leap 2 can enable remote assist for surgery

A central aim of Magic Leap Health is to better equip healthcare providers with AR enabled tools with the aim of bringing safe and effective care to patients, wherever the patients or applicable experts are located. The upcoming Magic Leap 2* platform is poised to have several capabilities conducive to remote assist for surgery, including the ability to use patient data to generate 3D renderings of anatomical features that can be shared remotely before or during surgery, segmented dimming, marker tracking integration, remote annotation, and spatialized audio. Compared with the previous version of the AR headset, Magic Leap 2 will have an extended battery life and increased field of view, both critical for remote assist use cases in the clinical setting.

AR enabled remote assist beyond surgery

Demand for surgical procedures is increasing, driven in large part by an aging population. Projected gaps between predicted surgeons entering practice and needed surgeons are growing. By 2030, there will be an estimated shortfall of 19,184 general surgeons.

Technology that helps maximize surgical efficiency and case volumes without compromising safety and outcomes, and enables the scaling of expertise of highly knowledgeable surgeons, can play an important role in meeting this growing demand for surgical services.

Beyond intraoperative use, there are several applications for which this technology could be deployed across the healthcare sector, from installing or repairing complex equipment in the operating room, to supporting nurses or home health workers in hospital-at-home programs, to guiding emergency medical services and first responders in disasters or accidents.

As attention continues to turn toward leveraging technology to improve access and equitable care to all patients, AR enabled systems and headsets such as Magic Leap’s have great potential to help healthcare stakeholders achieve these critical goals.

*Magic Leap 2 is not yet available for sale. Regulatory approval pending. Specifications subject to change.

For more healthcare-related information from Magic Leap:

• AR: A Game-Changer for Sports-Related Concussions
• The Magic Leap/SentiAR partnership
• How AR is Revolutionizing Rehabilitation for Neurological Disorders
• How AR in Healthcare is Transforming Patient Outcomes