潘金莲传媒映画

In an arena where the smallest bit of data can change the course of an operation鈥攁nd ultimately have a huge impact on patient outcomes鈥攕urgeons are taking a cue from medical imaging鈥檚 advancements in artificial intelligence (AI) to glean all the information they can get.

Keri A. Seymour, DO, MHS, F潘金莲传媒映画, FASMBS, a general and bariatric surgeon at Duke University Medical Center in Durham, North Carolina, saw an opportunity to optimize her patients鈥� success when she teamed up with a Duke radiologist who鈥檚 studying body composition and analysis from abdominal computed tomography (CT) scans.

Dr. Seymour, an associate professor of surgery at Duke University and director of research in the Division of Minimally Invasive Surgery, has conducted multiple studies on how metabolic factors affect patient outcomes, examining variables that influence the success of an operation and the patient鈥檚 postoperative progress.

鈥淭reatments for obesity tend to focus on body mass index (BMI) as a way to standardize our evaluation of patients,鈥� said Dr. Seymour, who also is President of the North Carolina Chapter of the 潘金莲传媒映画. 鈥淏ut that doesn鈥檛 really describe their body composition, and distribution of adipose tissue and muscle as well. Patients can have a significant amount of muscle and still have increased weight and a higher BMI.鈥�

Bioelectrical impedance testing provides more comprehensive information, especially for measuring changes in body composition over time. 鈥淭here is an interplay in patients鈥� metabolism and their pre- and post-op states,鈥� she explained. 鈥淰isualizing that relationship is key to understanding their progress. What I鈥檝e come to appreciate is that we can use medical imaging to evaluate their fat mass鈥攖o see if patients are losing not just fat but also muscle.鈥�

Enter Kirti Magudia, MD, PhD, an assistant professor of radiology at Duke University investigating high-level applications of machine learning in radiology.

Drs. Magudia and Seymour are currently working on a study of how CT-based body composition analysis could help optimize the selection and management of bariatric surgery patients. Preliminary results suggest that bariatric surgery patients with low or very low food security have less skeletal muscle and higher subcutaneous fat compared with those who have food security. 鈥淒espite these differences, bariatric surgery outcomes were similar across both groups, suggesting its effectiveness in improving the health of patients with obesity, including those facing food insecurity,鈥� they observed.

The two physicians soon learned that their individual collections of data, including routine CT scans, could be combined and mined for important insights on an individual patient, even beyond Dr. Magudia鈥檚 passion for CT-based body composition. 鈥淔or example, hepatic arterial anatomy can have many vascular variants,鈥� Dr. Magudia said. 鈥淚 keep drilling into our radiology trainees that they need to report it. You never know when it鈥檚 going to be needed, even for routine surgeries, like cholecystectomy.鈥�

AI tools could also aid in the identification of patients in the emergency department who need the most urgent imaging and surgical intervention, Dr. Magudia said. She further noted that she and Dr. Seymour had both been on call during the prior weekend shift. 鈥淥ur goal was to find those CT scans that Dr. Seymour needs to know about, so that they could be acted upon鈥攁nd were not buried under all the other radiology exams for patients with less urgent issues. That way, they could get to the OR as quickly as possible.鈥�

That also means making sure patients get the right kind of imaging, giving the surgeon the most useful information. Deep learning models can help prefill recommendations for appropriate imaging tests, giving providers both a heads-up and a head start.

An Israeli study presented at the 2023 annual meeting of the Radiological Society of North America (RSNA), for example, found that ChatGPT can deliver recommendations¹ for appropriate imaging tests that might be as reliable as the recommendations of the European Society of Radiology (ESR) iGuide. In their presentation, Mor Saban, PhD, and Shani Rosen, MSc, demonstrated that when ChatGPT is presented with clinical data about patient symptoms, it can generate suggestions to help clinicians select the imaging modality鈥擷-ray, CT, ultrasound, magnetic resonance imaging, and beyond鈥攖hat an experienced radiologist might recommend.

In that study, human experts evaluated the ChatGPT suggestions and found that up to 87% of them were medically accurate, when compared with those compiled in the ESR iGuide. And, as the authors noted, ChatGPT isn鈥檛 even specifically designed for medical tasks.

Fear Not鈥擨t鈥檚 Just an Algorithm

To the uninitiated, statements like 鈥淎I can recommend medical imaging tests鈥� might seem like the unsettling prelude to a scenario where physicians could be replaced by machines that lack the nuance of human insight. But understanding how tools like ChatGPT are trained鈥攐n the collective knowledge of humans鈥攃an shine light on the possibilities for maximizing human potential.

For example, ChatGPT is fed chunks of text called 鈥渢okens鈥� that come from websites, books, articles, and other publicly available sources. By building a dataset from these tokens, the model learns to predict the words or phrases human experts would be likely to use given a particular context.¹

In a clinical setting, having auto-filled suggestions could take some of the legwork out of initial evaluation鈥攁nd even encourage more thorough documentation. In a scenario such as Dr. Magudia鈥檚 example, in which being aware of unusual hepatic arterial anatomy could be vital to perioperative planning, an AI tool could help ensure that information is documented, whether the radiologist in the previous clinical case thought it relevant to note or not.

Elizabeth Burnside, MD, MPH, a professor in the Department of Radiology at the University of Wisconsin-Madison, explained during an RSNA 2023 plenary session the differences between discriminative AI models and generative AI models, offering digestible analogies for what each can accomplish. While discriminative models are primarily used to classify existing data into predetermined outcomes of interest, generative models use algorithms to craft content, incorporating text and images based on the data that trained them.

As an example, a discriminative model could be trained on millions of images of cats and dogs to learn their differences and, when presented with a new image, accurately label it as a cat or dog, Dr. Burnside said. Generative models train on similar data, but in this context, they would then be tasked with generating an image of a new cat or dog.

In a radiology setting, discriminative AI tasks could include identifying cancer on a mammogram or finding a bleed on a neuroimaging study鈥攐r determining whether pneumonia seen on a chest X-ray is related to COVID-19 infection. A generative model might be employed to create a radiology report based on the images it receives, simulate disease progression in a body system, or create summaries for patients in lay language.

The accuracy and the generalizability of an algorithm is dependent not only on the amount of information it鈥檚 given, but also on the composition and diversity鈥攊ncluding patient and surgeon characteristics鈥攐f the training data, said Jennifer A. Eckhoff, MD, from Massachusetts General Hospital in Boston.

Dr. Eckhoff, a senior resident at University Hospital Cologne in Germany, interrupted her residency in 2021 to start a postdoctoral fellowship at Mass General鈥檚 Surgical Artificial Intelligence and Innovation Laboratory (SAIIL). She鈥檚 now harnessing AI鈥檚 predictive qualities to assess risk from interoperative events.

鈥淢y research focus is on computer vision-based analysis of surgical video data鈥攕pecifically intra-abdominal minimally invasive surgical data,鈥� Dr. Eckhoff explained. 鈥淲e work on predicting what is happening in the next couple of seconds, or the next phase of an operation, in order to anticipate surgical risk.鈥�

Using video analysis, Dr. Eckhoff鈥檚 team examines the spatial and temporal relationships of the actions and tools that compose surgical workflow, using them to predict a surgeon鈥檚 next move. They train AI models to identify procedural steps on a granular level, down to tissue-to-tool interaction.

The next step is to integrate quantitative data from these video analyses alongside perioperative data to help predict patient-specific complications, readmissions, and oncological outcomes. One of SAIIL鈥檚 current projects, coincidentally, focuses on patients undergoing laparoscopic cholecystectomy.

Most AI applications in surgery are currently based on supervised machine learning models, which involve training an algorithm on labeled data, Dr. Eckhoff explained. 鈥淪o an algorithm is provided with a certain video dataset, which might be labeled with respect to the critical view of safety and its three subcomponents,鈥� she said, referring to visual criteria in a laparoscopic image鈥攁lso known as Strasberg鈥檚 criteria鈥攖hat let a surgeon know it鈥檚 safe to proceed with removing the gallbladder.

A challenge for AI-augmented surgery is building models that adequately integrate human knowledge and understanding. Dr. Eckhoff and her colleagues have proposed a novel approach to training the networks: incorporating a knowledge graph into the video analysis, to identify an algorithm鈥檚 鈥渦nderstanding鈥� of surgical notions and its ability to acquire conceptual knowledge as it applies to the data.

Their research demonstrated that AI models are able to learn tasks such as verification of the critical view of safety, apply the Parkland grading scale, and recognize instrument-action-tissue triplets.²

Meanwhile, Back at Duke

The principal investigator on the SAIIL project, Ozanan R. Meireles, MD, F潘金莲传媒映画, has assumed a new role as the Duke University Department of Surgery鈥檚 inaugural vice-chair for innovation. Dr. Meireles joined Duke in January, bringing with him the collaborative efforts of SAIIL and the Massachusetts Institute of Technology Computer Science and Artificial Intelligence Lab.

鈥淏y using the interaction between the surgeon and the machine to improve operational efficiency, the machines will get better over time,鈥� Dr. Meireles said. 鈥淲e鈥檙e going to be building our shared knowledge to create what we call a shared surgical consciousness, one that holds more knowledge than any single surgeon can acquire. That collective surgical consciousness can guide us away from complications and truly improve patient care.鈥�

Drs. Meireles and Eckhoff both expressed their excitement about the Critical View of Safety (CVS) Challenge, endorsed by the Society of American Gastrointestinal and Endoscopic Surgeons. It鈥檚 a global initiative to generate a large, diverse, annotated dataset for assessing the CVS, and it encourages researchers to compete in developing AI algorithms for real-time CVS detection, enhancing surgical safety and potentially easing surgeons鈥� workloads.

鈥淚n our work, we very much focus on governance of surgical video data and AI as it is applied to surgery,鈥� Dr. Eckhoff added. 鈥淲e鈥檙e composing a framework for interdisciplinary and international collaboration, which is essential for assembling large datasets, with respect to internationally varying privacy and data management regulations.鈥�

As Dr. Meireles explained, the CVS Challenge platform is designed to automatically de-identify all videos that contributors submit. 鈥淲hen you upload a video, you do it through a secure account, and there鈥檚 a data-sharing agreement explaining that the video will be de-identified. The platform strips all the metadata, and, if the camera comes out of the abdomen and there are images taken outside the body, it blurs them.鈥�

He adds that, while privacy regulations vary in different parts of the world and there are special considerations for certain rare cases, this process for anonymizing data has been well received by participants across the globe who recognize that the ability to share surgical knowledge is essential for actionable research.

More Like GPS than a Self-Driving Car

An analogy that鈥檚 often used in comparing AI versus human decision-making is that it鈥檚 akin to a self-driving car versus a human driver鈥攖he former hasn鈥檛 quite mastered complicated driving that benefits from the nuances of human experience. Dr. Meireles likens AI-assisted surgery to a human driver using GPS. He noted that drivers are more likely to follow a suggestion from a GPS鈥攚hich mines collective data to predict the most efficient route鈥攖han they are from a human passenger.

Still, 鈥渋f you鈥檙e using a navigation tool and it tells you to turn right or left, you could ignore it and just keep driving,鈥� he said.

Which raises questions about accountability and communication: 鈥淎s we鈥檙e going through this cultural transformation era through artificial intelligence, patients should understand that AI agents might be helping their physician make a decision鈥攐r even that their physician could be disagreeing with the AI. How are we going to be explaining that, and what鈥檚 the patient鈥檚 role in this?鈥� Dr. Meireles asked.

Best Outcomes Take a Village

If incorporating these steps into surgical workflow seems daunting, Dr. Magudia has a reminder for clinicians. 鈥淎round 30 years ago, most radiology exams were on physical film, and it took a lot of work and effort among vendors and clinical radiologists to get to where we are today with PACS(picture archiving and communication systems) and the DICOM (digital imaging and communications in medicine) standard imaging format. This has revolutionized the way radiology is practiced and allowed us to advance further.鈥�

Dr. Seymour has begun conversations with other clinicians in her role as chief quality officer about using accessible data to reveal additional factors that contribute to a surgery鈥檚 success. 鈥淲e鈥檝e talked about surgical site infection management, looking at the information we already have in the operating room鈥攁nesthesia, patient temperature, the timing of antibiotics鈥攁ll the things we can record and review to see if they鈥檒l be predictive of patient outcomes.鈥�

And incorporating those factors into an automated system can help surgeons better anticipate the course of their workflow. One of Dr. Meireles鈥檚 recent projects鈥攁gain in laparoscopic cholecystectomies鈥攊nvolved an AI model that was trained to grade intraoperative difficulty via the Parkland grading scale from an initial view of the gallbladder.

The AI鈥檚 performance was comparable to that of a human surgeon in identifying the degree of gallbladder inflammation, which is predictive of intraoperative course.³ By quickly predicting how difficult a cholecystectomy will be鈥攁nd how long it will take a surgeon to complete鈥攖his automated assessment could be useful for optimizing workflow in the operating room, the researchers stated.

The model also could help develop personalized feedback for surgeons and trainees, offering opportunities for them to perfect their technique.

Harnessing the potential of AI will naturally come with regulatory and data management responsibilities, Dr. Eckhoff noted. 鈥淭hat also entails involving different stakeholders, including patients, other operating room staff, computer scientists, industry representatives, and other medical specialties.鈥�

Medical specialties like radiology and pathology have embraced AI at a particularly impressive pace, explained Dr. Eckhoff. Indeed, the RSNA annual meeting in November boasted nearly 400 sessions covering AI topics alone, and not just for clinical decision support. Presenters explored applications from opportunistic screening to patient-centered practice to creating a more egalitarian process for leadership selection.

鈥淭he impact that clinical societies have is unmatched, especially in the United States,鈥� she said. 鈥淎nd we have a great opportunity to shape the perception of AI among clinicians in the future, demonstrating that we can use it as a tool, and how the umbrella term 鈥楢I鈥� can be divided into many different subsections and subdisciplines.鈥�

Dr. Eckhoff said she is excited to see how AI will impact outcomes. 鈥淓ach tool needs to be tested for clinical validity, but we鈥檙e not far from seeing how AI can really change the concept of surgical safety.鈥�

Evonne Acevedo is a freelance writer.

References

1. Rosen S, Saban M. Evaluating the reliability of ChatGPT as a tool for imaging test referral: A comparative study with a clinical decision support system. Eur Radiol. 2023 Oct 13. Epub ahead of print.
2. Ban Y, Eckhoff JA, Ward TM, Hashimoto DA, et al. Concept graph neural networks for surgical video understanding. IEEE Trans Med Imaging. 2024;43(1):264-274.
3. Ward TM, Hashimoto DA, Ban Y, Rosman G, et al. Artificial intelligence prediction of cholecystectomy operative course from automated identification of gallbladder inflammation. Surg Endosc. 2022;36(9):6832-6840.