AI and the Future of Surgery

Artificial Intelligence in the OR: Separating the Hype From the Reality

By VICTORIA STERN

Artificial intelligence in medicine is here. But it’s still in its infancy.

The FDA has approved about three dozen AI algorithms in medicine over the past few years, many of which aim to help physicians diagnose disease: diabetic retinopathy from retinal images, cancerous liver and lung lesions from CT and MRI scans, and breast cancer from mammograms.

Entrepreneurs and health care leaders also have jumped on board. Last year, health AI startups brought in $4 billion in funding, and a recent survey found that about half of hospital executives intend to invest in AI over the next year or two.

Although some experts predict that AI will transform medicine, others worry the buzz may be overblown. A recent headline in The Los Angeles Times asked, “Can AI live up to the hype?” and an article from The Hill explored the “Dangers of Artificial Intelligence in Medicine.”

“At the moment, the term AI is very hyped, but a lot of it is real and already translating into something tangible,” said Sandip Panesar, MD, a postdoctoral research fellow in neurosurgery at Stanford University, in California. But, he cautioned, “robots and devices that can take over human dexterity: That’s in the future. Humans will have to hold AI’s hand for a long time.”

But will AI someday replace physicians? Judith Pins, RN, MBA, the president of the health care education company Pfiedler Education, a subsidiary of the Association of periOperative Registered Nurses, does not think so. Ms. Pins, who recently spoke at a Nurse Executive Leadership Seminar on the OR of the future, sees AI as a tool that can assist providers.

“AI won’t replace doctors and nurses, but it can help make good providers great ones in terms of validating diagnoses and patient care,” she said.

Dan Hashimoto, MD, MS, a general surgery resident at Massachusetts General Hospital, in Boston, has some concerns about the hype surrounding AI in medicine. Dr. Hashimoto believes that unrealistic expectations about AI can “lead to significant disappointment and disillusionment” (Ann Surg 2018;268[1]:70-76).

Artificial intelligence experts have seen it happen before. In the early 1990s, when advances in expert systems—AI software that can mimic the decision-making ability of a human—did not live up to the media hype, the research fizzled.

“Problems arose because we did not have enough computational power several decades ago to develop robust algorithms that could deal with large image data sets,” said Sharmila Majumdar, PhD, a professor and the vice chair of research in the Department of Radiology at the University of California, San Francisco, who launched the Artificial Intelligence Center to Advance Medical Imaging last year.

But Dr. Hashimoto—who co-directs the Surgical Artificial Intelligence and Innovation Laboratory at MGH with Ozanan Meireles, MD, an assistant professor of surgery at Harvard Medical School, in Boston—says things are different now.

“I think it’s the right time to pursue AI because the hardware is catching up to the concepts,” he said.

An AI Algorithm to Guide Surgeons

Despite the growing number of medical AI designs, not many products being developed are specific to surgery.

Dr. Hashimoto is one of the few researchers working on AI for surgeons, specifically, an algorithm that can predict the risk for complications and readmissions for patients during surgery and warn surgeons in real time.

The idea came to Dr. Hashimoto in 2012 after he reread Michael Lewis’s book “Moneyball: The Art of Winning an Unfair Game,” which delved into the statistical world of baseball. The book described, among other things, how a player’s likelihood of getting on base contributed to a team’s chances of winning or losing.

Dr. Hashimoto wondered whether he could apply this idea to surgery: Develop an AI algorithm to identify surgical events linked to outcomes and use that information to predict outcomes, complications, even who lives or dies. But unlike hits and walks, what if these are measures humans can’t easily count, such as how a surgeon holds a needle?

“We had no way of discovering or tracking that kind of data,” Dr. Hashimoto said.

In 2014, the duo teamed up with Guy Rosman, PhD, and Daniela Rus, PhD, from the Computer Science and Artificial Intelligence Laboratory at Massachusetts Institute of Technology, in Cambridge. Drs. Rosman and Rus had already built an algorithm to analyze large-scale streaming data that could segment continuous data. The algorithm, for instance, could identify scene breaks in the 2014 movie “Birdman,” which appeared to be filmed as one continuous shot.

The researchers expanded on this premise and developed an algorithm that could pinpoint the individual steps in a laparoscopic procedure from a video recording about 90% of the time (Ann Surg 2019;270[3]:414-421).

The next step: Determine whether the algorithm can do this in real time.

“When we tweaked the algorithm, we still got about 90% accuracy in real time,” Dr. Hashimoto said.

The final hurdle, Dr. Hashimoto noted, is still ahead. Can AI use realtime data, along with patient records, to predict what will happen during an operation?

“Essentially we want to know, will a surgeon’s next move come with a high probability of making an error or resulting in a poor outcome for the patient, and can we develop an AI that predicts this risk and warns the surgeon?” Dr. Hashimoto explained.

That phase represents the “holy grail” of what he’s trying to do. The biggest obstacle is the lack of data. He said he will likely need to incorporate data from thousands of surgical videos to build a robust algorithm, but “we currently have no centralized, uniform way to collect and store video data and no standard way to annotate that video,” he said.

Dr. Hashimoto sees the technology largely as a support tool that can make medicine and surgery safer. “My hope is that AI can help physicians make better decisions, enhance performance, and eventually augment some more routine aspects of a case,” he said. “With AI, we’re not building ‘The Terminator’; we’re building ‘Ironman.’” ■

‘My hope is that AI can help physicians make better decisions, enhance performance, and eventually augment some more routine aspects of a case. With AI, we’re not building “The Terminator”; we’re building “Ironman.”’ —Dan Hashimoto, MD, MS

Here are a handful of other surgery-specific artificial intelligence designs that show potential to improve patient care:

Robotics STAR (Smart Tissue Autonomous Robot), developed by researchers at Children’s National Health System and Johns Hopkins University, can suture two parts of a pig’s intestine together with minimal to no help from humans.

XAware The Computational Modeling and Analysis of Medical Activities group at Strasbourg University Hospital, in France, led by Nicolas Padoy, PhD, is designing an AI tool that monitors radiation during image-guided interventions, using augmented reality to detect and visualize radiation scatter patterns and ultimately reduce OR staff’s exposure to the emissions.

Context-aware SensorOR Researchers at the National Center for Tumor Diseases in Dresden, Germany, led by Stefanie Speidel, PhD, are developing machine learning technology that can analyze sensor data from the laparoscopic images, surgical devices and other activities in the OR. The goal is to understand, guide and optimize the flow of surgery.

Brain Surgery A new technique that combines AI and imaging can help neurosurgeons diagnose tumors in a few minutes without removing brain tissue, which is the current standard approach for diagnosing these tumors. A recent Nature Medicine paper outlined the technique, which uses a deep convolutional neural network algorithm to learn brain tumor characteristics and predict the diagnosis from MRI scans (2020;26:52-58).