Background: Deep learning diagnostic algorithms are proving comparable results with human experts in a wide variety of tasks, and they still require a huge amount of well-annotated data for training, which is often non affordable. Metric learning techniques have allowed a reduction in the required annotated data allowing few-shot learning over deep learning architectures. Aims and Objectives: In this work, we analyze the state-of-the-art loss functions such as triplet loss, contrastive loss, and multi-class N-pair loss for the visual embedding extraction of hematoxylin and eosin (H&E) microscopy images and we propose a novel constellation loss function that takes advantage of the visual distances of the embeddings of the negative samples and thus, performing a regularization that increases the quality of the extracted embeddings. Materials and Methods: To this end, we employed the public H&E imaging dataset from the University Medical Center Mannheim (Germany) that contains tissue samples from low-grade and high-grade primary tumors of digitalized colorectal cancer tissue slides. These samples are divided into eight different textures (1. tumour epithelium, 2. simple stroma, 3. complex stroma, 4. immune cells, 5. debris and mucus, 6. mucosal glands, 7. adipose tissue and 8. background,). The dataset was divided randomly into train and test splits and the training split was used to train a classifier to distinguish among the different textures with just 20 training images. The process was repeated 10 times for each loss function. Performance was compared both for cluster compactness and for classification accuracy on separating the aforementioned textures. Results: Our results show that the proposed loss function outperforms the other methods by obtaining more compact clusters (Davis-Boulding: 1.41 ± 0.08, Silhouette: 0.37 ± 0.02) and better classification capabilities (accuracy: 85.0 ± 0.6) over H and E microscopy images. We demonstrate that the proposed constellation loss can be successfully used in the medical domain in situations of data scarcity.

A whole-slide imaging (WSI) system is a digital color imaging system used in digital pathology with the potential to substitute the conventional light microscope. A WSI system digitalizes a glass slide by converting the optical image to digital data with a scanner and then converting the digital data back to the optical image with a display. During the digital-to-optical or optical-to-digital conversion, a color space is required to define the mapping between the digital domain and the optical domain so that the numerical data of each color pixel can be interpreted meaningfully. Unfortunately, many current WSI products do not specify the designated color space clearly, which leaves the user using the universally default color space, sRGB. sRGB is a legacy color space that has a limited color gamut, which is known to be unable to reproduce all color shades present in histology slides. In this work, experiments were conducted to quantitatively investigate the limitation of the sRGB color space used in WSI systems. Eight hematoxylin and eosin (H and E)-stained tissue samples, including human bladder, brain, breast, colon, kidney, liver, lung, and uterus, were measured with a multispectral imaging system to obtain the true colors at the pixel level. The measured color truth of each pixel was converted into the standard CIELAB color space to test whether it was within the color gamut of the sRGB color space. Experiment results show that all the eight images have a portion of pixels outside the sRGB color gamut. In the worst-case scenario, the bladder sample, about 35% of the image exceeded the sRGB color gamut. The results suggest that the sRGB color space is inadequate for WSI scanners to encode H and E-stained whole-slide images, and an sRGB display may have insufficient color gamut for displaying H and E-stained histology images.

Background: Clinicopathological scores are used to predict the likelihood of recurrence-free survival for patients with clear cell renal cell carcinoma (ccRCC) after surgery. These are fallible, particularly in the middle range. This inevitably means that a significant proportion of ccRCC patients who will not develop recurrent disease enroll into clinical trials. As an exemplar of using digital pathology, we sought to improve the predictive power of “recurrence free” designation in localized ccRCC patients, by precise measurement of ccRCC nuclear morphological features using computational image analysis, thereby replacing manual nuclear grade assessment. Materials and Methods: TNM 8 UICC pathological stage pT1-pT3 ccRCC cases were recruited in Scotland and in Singapore. A Leibovich score (LS) was calculated. Definiens Tissue studio® (Definiens GmbH, Munich) image analysis platform was used to measure tumor nuclear morphological features in digitized hematoxylin and eosin (H&E) images. Results: Replacing human-defined nuclear grade with computer-defined mean perimeter generated a modified Leibovich algorithm, improved overall specificity 0.86 from 0.76 in the training cohort. The greatest increase in specificity was seen in LS 5 and 6, which went from 0 to 0.57 and 0.40, respectively. The modified Leibovich algorithm increased the specificity from 0.84 to 0.94 in the validation cohort. Conclusions: CcRCC nuclear mean perimeter, measured by computational image analysis, together with tumor stage and size, node status and necrosis improved the accuracy of predicting recurrence-free in the localized ccRCC patients. This finding was validated in an ethnically different Singaporean cohort, despite the different H and E staining protocol and scanner used. This may be a useful patient selection tool for recruitment to multicenter studies, preventing some patients from receiving unnecessary additional treatment while reducing the number of patients required to achieve adequate power within neoadjuvant and adjuvant clinical studies.

Background: Recent advancements in machine learning (ML) bring great possibilities for the development of tools to assist with diagnostic tasks within histopathology. However, these approaches typically require a large amount of ground truth training data in the form of image annotations made by human experts. As such annotation work is a very time-consuming task, there is a great need for tools that can assist in this process, saving time while not sacrificing annotation quality. Methods: In an iterative design process, we developed TissueWand – an interactive tool designed for efficient annotation of gigapixel-sized histopathological images, not being constrained to a predefined annotation task. Results: Several findings regarding appropriate interaction concepts were made, where a key design component was semi-automation based on rapid interaction feedback in a local region. In a user study, the resulting tool was shown to cause substantial speed-up compared to manual work while maintaining quality. Conclusions: The TissueWand tool shows promise to replace manual methods for early stages of dataset curation where no task-specific ML model yet exists to aid the effort.

Context: Several therapeutically important mutations in cancers are economically detected using immunohistochemistry (IHC), which highlights the overexpression of specific antigens associated with the mutation. However, IHC panels can be imprecise and relatively expensive in low-income settings. On the other hand, although hematoxylin and eosin (H&E) staining used to visualize the general tissue morphology is a routine and low cost, it does not highlight any specific antigen or mutation. Aims: Using the human epidermal growth factor receptor 2 (HER2) mutation in breast cancer as an example, we strengthen the case for cost-effective detection and screening of overexpression of HER2 protein in H&E-stained tissue. Settings and Design: We use computational methods that reliably detect subtle morphological changes associated with the over-expression of mutation-specific proteins directly from H&E images. Subjects and Methods: We trained a classification pipeline to determine HER2 overexpression status of H&E stained whole slide images. Our training dataset was derived from a single hospital containing 26 (11 HER2+ and 15 HER2–) cases. We tested the classification pipeline on 26 (8 HER2+ and 18 HER2–) held-out cases from the same hospital and 45 independent cases (23 HER2+ and 22 HER2–) from the TCGA-BRCA cohort. The pipeline was composed of a stain separation module and three deep neural network modules in tandem for robustness and interpretability. Statistical Analysis Used: We evaluate our trained model through area under the curve (AUC)-receiver operating characteristic. Results: Our pipeline achieved an AUC of 0.82 (confidence interval [CI]: 0. 65–0. 98) on held-out cases and an AUC of 0.76 (CI: 0. 61–0. 89) on the independent dataset from TCGA. We also demonstrate the region-level correspondence of HER2 overexpression between a patient's IHC and H&E serial sections. Conclusions: Our work strengthens the case for automatically quantifying the overexpression of mutation-specific proteins in H&E-stained digital pathology, and it highlights the importance of multi-stage machine learning pipelines for added robustness and interpretability.

Introduction: Biomedical and translational research often relies on the evaluation of patients or specimens that meet specific clinical or laboratory criteria. The typical approach used to identify biospecimens is a manual, retrospective process that exists outside the clinical workflow. This often makes biospecimen collection cost prohibitive and prevents the collection of analytes with short stability times. Emerging data architectures offer novel approaches to enhance specimen-identification practices. To this end, we present a new tool that can be deployed in a real-time environment to automate the identification and notification of available biospecimens for biomedical research. Methods: Real-time clinical and laboratory data from Cloverleaf (Infor, NY, NY) were acquired within our computational health platform, which is built on open-source applications. Study-specific filters were developed in NiFi (Apache Software Foundation, Wakefield, MA, USA) to identify the study-appropriate specimens in real time. Specimen metadata were stored in Elasticsearch (Elastic N. V., Mountain View, CA, USA) for visualization and automated alerting. Results: Between June 2018 and December 2018, we identified 2992 unique specimens belonging to 2815 unique patients, split between two different use cases. Based on laboratory policy for specimen retention and study-specific stability requirements, secure E-mail notifications were sent to investigators to automatically notify of availability. The assessment of throughput on commodity hardware demonstrates the ability to scale to approximately 2000 results per second. Conclusion: This work demonstrates that real-world clinical data can be analyzed in real time to increase the efficiency of biospecimen identification with minimal overhead for the clinical laboratory. Future work will integrate additional data types, including the analysis of unstructured data, to enable more complex cases and biospecimen identification.

Background: The Visiopharm human epidermal growth factor receptor 2 (HER2) digital imaging analysis (DIA) algorithm assesses digitized HER2 immunohistochemistry (IHC) by measuring cell membrane connectivity. We aimed to validate this algorithm for clinical use by comparing with pathologists' scoring and correlating with HER2 fluorescence in situ hybridization (FISH) results. Materials and Methods: The study cohort consisted of 612 consecutive invasive breast carcinoma specimens including 395 biopsies and 217 resections. HER2 IHC slides were scanned using Philips IntelliSite Scanners, and the digital images were analyzed using Visiopharm HER2-CONNECT App to obtain the connectivity values (0–1) and scores (0, 1+, 2+, and 3+). HER2 DIA scores were compared with Pathologists' manual scores, and HER2 connectivity values were correlated with HER2 FISH results. Results: The concordance between HER2 DIA scores and pathologists' scores was 87.3% (534/612). All discordant cases (n = 78) were only one-step discordant (negative to equivocal, equivocal to positive, or vice versa). Five cases (0.8%) showed discordant HER2 IHC DIA and HER2 FISH results, but all these cases had relatively low HER2 copy numbers (between 4 and 6). HER2 IHC connectivity showed significantly better correlation with HER2 copy number than HER2/CEP17 ratio. Conclusions: HER2 IHC DIA demonstrates excellent concordance with pathologists' scores and accurately discriminates between HER2 FISH positive and negative cases. HER2 IHC connectivity has better correlation with HER2 copy number than HER2/CEP17 ratio, suggesting HER2 copy number may be more important in predicting HER2 protein expression, and response to anti-HER2-targeted therapy.