Journal of Pathology Informatics

ORIGINAL ARTICLE
Year
: 2019  |  Volume : 10  |  Issue : 1  |  Page : 19-

Deep learning-based retrieval system for gigapixel histopathology cases and the open access literature


Roger Schaer1, Sebastian Otálora2, Oscar Jimenez-del-Toro2, Manfredo Atzori1, Henning Müller2 
1 Institute of Information Systems, HES-SO (University of Applied Sciences of Western Switzerland), Sierre, Switzerland
2 Institute of Information Systems, HES-SO (University of Applied Sciences of Western Switzerland), Sierre; Department of Computer Science, University of Geneva (UNIGE), Geneva, Switzerland

Correspondence Address:
Mr. Roger Schaer
HES-SO Valais, Institute of Information Systems, Techno-Pôle 3, 3960 Sierre
Switzerland

Background: The introduction of digital pathology into clinical practice has led to the development of clinical workflows with digital images, in connection with pathology reports. Still, most of the current work is time-consuming manual analysis of image areas at different scales. Links with data in the biomedical literature are rare, and a need for search based on visual similarity within whole slide images (WSIs) exists. Objectives: The main objective of the work presented is to integrate content-based visual retrieval with a WSI viewer in a prototype. Another objective is to connect cases analyzed in the viewer with cases or images from the biomedical literature, including the search through visual similarity and text. Methods: An innovative retrieval system for digital pathology is integrated with a WSI viewer, allowing to define regions of interest (ROIs) in images as queries for finding visually similar areas in the same or other images and to zoom in/out to find structures at varying magnification levels. The algorithms are based on a multimodal approach, exploiting both text information and content-based image features. Results: The retrieval system allows viewing WSIs and searching for regions that are visually similar to manually defined ROIs in various data sources (proprietary and public datasets, e.g., scientific literature). The system was tested by pathologists, highlighting its capabilities and suggesting ways to improve it and make it more usable in clinical practice. Conclusions: The developed system can enhance the practice of pathologists by enabling them to use their experience and knowledge to control artificial intelligence tools for navigating repositories of images for clinical decision support and teaching, where the comparison with visually similar cases can help to avoid misinterpretations. The system is available as open source, allowing the scientific community to test, ideate and develop similar systems for research and clinical practice.


How to cite this article:
Schaer R, Otálora S, Jimenez-del-Toro O, Atzori M, Müller H. Deep learning-based retrieval system for gigapixel histopathology cases and the open access literature.J Pathol Inform 2019;10:19-19


How to cite this URL:
Schaer R, Otálora S, Jimenez-del-Toro O, Atzori M, Müller H. Deep learning-based retrieval system for gigapixel histopathology cases and the open access literature. J Pathol Inform [serial online] 2019 [cited 2019 Nov 12 ];10:19-19
Available from: http://www.jpathinformatics.org/article.asp?issn=2153-3539;year=2019;volume=10;issue=1;spage=19;epage=19;aulast=Schaer;type=0