Machine learning and hyper spectral imaging: multi spectral endoscopy in the gastro intestinal tract towards hyper spectral endoscopy

Worldwide, carcinomas are the second leading cause of death after cardiovascular diseases. Of these, more than 25 % are caused by cancer in the gastrointestinal tract. However, screening still misses about 20 % of carcinomas. An optimal solution would be a red flag technology to locate the suspicious area. Despite the use of many optical technologies and advancements in traditional endoscopic technologies, most methods have failed to significantly improve the detectability of carcinomas in the GI. Therefore, it has been suggested in the literature to use spectroscopic quantitative measurements, which is achieved by hyperspectral imaging. However, in this work, a multispectral endoscope system for in vivo use is used for human (stomach) and mouse (colon) studies. In the human study, the tumours could only be found to a limited extent with the standard methods (MCC = 0. 32; ACC2=0. 68). Nevertheless, this is a significant improvement compared to previous results. The introduction of spectral spatial variation as a spatial-spectral feature improves the results significantly again. Compared to the human study, the results from the mouse model show better classification results: ACC2 = 0. 73 and MCC = 0. 47. Thus, for the development of an endoscopic red flag technique, a point technique for correct pre-labelling of the data is needed. Moreover, the results of this MSI in vivo study with spatial features are similar to the recent HSI ex vivo studies. Therefore, it is likely that MSI will remain part of future research.