Emerging role of AI in endoscopy

In the near future, AI will become more and more relevant in clinical practice. In addition, it might also be expected that AI will become relevant not only for endoscopic detection but also for more accurate AI-guided endoscopic resection.

Since the first published use of endoscope in 1805 by Bozzini, termed the Lichtleiter (light conductor), rapid developments in endoscopic imaging have ensued with the foundation of major endoscopy systems. The realm of optical biopsy has been widely studied, using advanced imaging modalities, including magnification, narrow-band imaging (NBI), dye-based chromoendoscopy (DBCE), virtual chromoendoscopy (VCE) provided by various imaging systems including optical enhancement, Fujinon intelligent color enhancement (FICE), blue-light imaging (BLI), and linked color imaging (LCI) to optimize luminal endoscopic diagnosis. The latest technological advances offer an opportunity to further push the boundaries of the current advanced endoscopic imaging.

The latest state-of-the-art technological innovations have led to a palpable progression in endoscopic imaging, and may facilitate standardization of practice. One of the most rapidly evolving modalities is artificial intelligence (AI), with recent studies providing real-time diagnoses and encouraging results in the first randomized trials to conventional endoscopic imaging.

The introduction of AI using deep-learning methods revolutionized the use of algorithms in the medical field. Deep learning neural networks are currently trained, using a large amount of manually annotated datasets. These datasets can be images like the ones that are taken during routine endoscopy or videos. However, not only images can be used to train AI, sounds and even laboratory test results from medical health records can also be used to generate an AI that intelligently guides decision-making in endoscopy. Training is performed, using powerful computers with multiple graphic cards specialized on the creation of neural networks. These are usually located in a server facility. Training and optimizing a neural network takes time. A fully trained unmodifiable neuronal network is then usually applied, using a computer with less computational resources, similar to the one that is located in the endoscopy room. One advantage of these AIs is that their application can be performed in real time due to the fact that images are mostly processed at a lower resolution than the image seen on the examination screen, and thus require less computational power. Efforts are made not only to develop neural networks within the server facilities but also to apply the real-time application, using cloud-based solutions.

AI in colonoscopy. Prevention of colorectal cancer is one of the main aims of endoscopy. Technological advances like high-definition images and virtual chromoendoscopy help identify and characterize lesions. Adenoma detection rate (ADR) is one of the key performance measures for screening colonoscopy and is correlated to death due to colorectal cancer. Guidelines recommend a clean bowel and taking enough time for withdrawal to improve ADR. Due to the fact that the diagnostic process depends only on the examiner looking for polyps on the endoscopic image, AI as a second pair of eyes like the ones of an observing nurse is suited to support the examiner. Algorithms to detect polyps in screening colonoscopy (computer-assisted detection, CADe) have been developed and tested in multiple prospectively randomized clinical trials with ADR as the primary endpoint.

Another important field of AI in colonoscopy is the computer-aided characterization (CADx) of polyps. High-definition endoscopy and virtual chromoendoscopy with magnification of mucosal areas are opening up new perspectives on surface details for examiners. CADx might help interpret these details and establish a diagnosis. Still, studies analyzing the effect of CAD systems do not have the same quality including a randomized prospective design as the ones examining the effect of CADe systems. Additionally, vendors are grading their devices up by adding CADx to the already-existing CADe modes.

AI in gastroscopy. In contrast to colonoscopy, the use of AI in gastroscopy is currently rather limited. So far, there is no AI-based product commercially available. Nevertheless, promising data have been reported, and it can be expected that several products developed for upper GI may be available soon.

One of the greatest challenges in gastrointestinal endoscopy is to detect and delineate Barrett’s neoplasia. Apart from detection and differentiation of neoplasia, further prototypic applications of AI in gastroscopy were to establish deep-learning models for endoscopic classification of gastroesophageal reflux disease and detection of Helicobacter pylori-associated gastritis.

Last but not the least, similar to colonoscopy, measurement of key performance measures for gastroscopy is a potential target for AI. The quality of gastroscopy is directly related to the documented examination of certain anatomical landmarks, starting from the hypopharynx to the descending duodenum. Since AI may control whether landmarks have been adequately visualized during an ongoing endoscopy, it is an ideal tool to assist the examiner.

Using AI in other fields of endoscopic imaging. In general, all imaging technologies may be used for AI. In endoscopy, these include endoscopic ultrasound (EUS) images, fluoroscopical images acquired during ERCP, and images from capsule endoscopy. Clinical indications that have been observed with respect to EUS include assessment of quality measures for EUS imaging of the pancreas, diagnosis, and differentiation of sub-epithelial tumors, and detection of pancreatic ductal adenocarcinoma. For the latter two indications, an accuracy of 90 percent has been reported in these studies.

For ERCP, data are sparse. Here, apart from the diagnosis of diseases affecting the biliary system, an interesting approach of AI can be regarded as the adoption of a scoring system for ERCP difficulty, thereby guiding treatment. This means that based on the ERCP image (and potentially other anamnestic or clinical data), AI may suggest whether a procedure is likely to bear an increased risk and experts should perform the examination rather than novices. Capsule endoscopy can be regarded as an ideal imaging platform for AI. Taking under consideration the time that is needed to evaluate capsule videos, adoption of AI for diagnostic assistance would be very attractive. It has been demonstrated that AI-guided detection of blood is almost 100-percent accurate. Based on these excellent results and the time-consuming reading of images by capsule endoscopy, it can be expected that AI will soon be incorporated into clinical practice.

Nonetheless, there are also several challenges yet to be addressed. It is almost impossible to eliminate the risks of selective bias during data collection. There are significant variations among the study designs in published evidence. They still identically demonstrated the favorable diagnostic performances, equivalent to previously reported human performances. The terminology to describe the diagnostic performance of a CAD model, such as sensitivity and specificity, may have different values, as they are widely applied for binary classification analyses in other medical fields also. For example, most of the research for CADe models is designed to evaluate the diagnostic performances in imaging datasets of cases and controls prepared in advance. Therefore, the values of the sensitivity and specificity are not very meaningful in the real world. The specificity can be analyzed by counting the false-positive results in control images that do not present lesions. Furthermore, diagnostic performances may vary even for the same model and same validation dataset, when they are calculated per lesion, per image, and per patient. The negative effects of false-positive cases also need to be evaluated carefully in prospective clinical trials considering risks to overlook true lesions by misleading the endoscopist’s attention toward false positives. Prospective randomized controlled trials, applying a reliable clinical quality indicator of examination, such as the ADR or adenoma miss rate (AMR) for colonoscopy, as a primary outcome, would be more ideal to fairly evaluate the value of a model in clinical settings.

However, reasonable indicators to assess the quality of endoscopic diagnoses are almost nonexistent outside the colon and rectum. Moreover, RCTs may be too cumbersome to efficiently validate the rapidly evolving technology. Even after completing the study and peer-review process, the published data can be outdated when published, as updated models are generated one after the other. The reproducibility of outcomes is rarely reassessed in a replication test by other researchers. Thus, it would be difficult to establish regulatory standards to secure the clinical use of such models until academic standards are established to ensure their clinical value and safety. It is difficult to structurally explain the decision-making process of AI to humans, but the opaqueness of the relevant information is essentially demanded in a medical law suit. Prior to their clinical introduction, one must place importance on mitigating the risks of malpractices, caused by misjudgments of AI or miscommunication between AI and humans. Academic and legal safety considerations for AI may critically delay their introduction into daily clinical practice.

Finally, AI should be regarded much more rather than solely image analysis. Other relevant data, such as anamnestic data or laboratory values, should be integrated in such deep learning systems, thereby helping the examiner diagnose and treat individual patients with maximum accuracy and minimal invasiveness.

There are several categories of AI systems to be potentially used in endoscopy. The spectrum is large with many potentially useful clinical applications, such as improved detection of lesions, differentiation of lesions based on their mucosal or vascular pattern, risk stratification before/during therapy, and assessment of key performance measures. Overall, the main advantages of AI-assisted endoscopy should be regarded to reduce the workload and lead to greater accuracy. Hence, in the near future, AI will become more and more relevant in clinical practice. In addition, it might also be expected that AI will become relevant not only for endoscopic detection but also for more accurate AI-guided endoscopic resection.