High-quality medical data is critical to the development and implementation of machine learning (ML) algorithms in healthcare; however, security, and privacy concerns continue to limit access. We sought to determine the utility of “synthetic data” in training ML algorithms for the detection of tuberculosis (TB) from inflammatory biomarker profiles. A retrospective dataset (A) comprised of 278 patients was used to generate synthetic datasets (B, C, and D) for training models prior to secondary validation on a generalization dataset. ML models trained and validated on the Dataset A (real) demonstrated an accuracy of 90%, a sensitivity of 89% (95% CI, 83–94%), and a specificity of 100% (95% CI, 81–100%). Models trained using the optimal synthetic dataset B showed an accuracy of 91%, a sensitivity of 93% (95% CI, 87–96%), and a specificity of 77% (95% CI, 50–93%). Synthetic datasets C and D displayed diminished performance measures (respective accuracies of 71% and 54%). This pilot study highlights the promise of synthetic data as an expedited means for ML algorithm development.

Background: Prediction of clinical outcomes for individual cancer patients is an important step in the disease diagnosis and subsequently guides the treatment and patient counseling. In this work, we develop and evaluate a joint outcome and biomarker supervised (estrogen receptor expression and ERBB2 expression and gene amplification) multitask deep learning model for prediction of outcome in breast cancer patients in two nation-wide multicenter studies in Finland (the FinProg and FinHer studies). Our approach combines deep learning with expert knowledge to provide more accurate, robust, and integrated prediction of breast cancer outcomes. Materials and Methods: Using deep learning, we trained convolutional neural networks (CNNs) with digitized tissue microarray (TMA) samples of primary hematoxylin-eosin-stained breast cancer specimens from 693 patients in the FinProg series as input and breast cancer-specific survival as the endpoint. The trained algorithms were tested on 354 TMA patient samples in the same series. An independent set of whole-slide (WS) tumor samples from 674 patients in another multicenter study (FinHer) was used to validate and verify the generalization of the outcome prediction based on CNN models by Cox survival regression and concordance index (c-index). Visual cancer tissue characterization, i.e., number of mitoses, tubules, nuclear pleomorphism, tumor-infiltrating lymphocytes, and necrosis was performed on TMA samples in the FinProg test set by a pathologist and combined with deep learning-based outcome prediction in a multitask algorithm. Results: The multitask algorithm achieved a hazard ratio (HR) of 2.0 (95% confidence interval [CI] 1.30–3.00), P < 0.001, c-index of 0.59 on the 354 test set of FinProg patients, and an HR of 1.7 (95% CI 1.2–2.6), P = 0.003, c-index 0.57 on the WS tumor samples from 674 patients in the independent FinHer series. The multitask CNN remained a statistically independent predictor of survival in both test sets when adjusted for histological grade, tumor size, and axillary lymph node status in a multivariate Cox analyses. An improved accuracy (c-index 0.66) was achieved when deep learning was combined with the tissue characteristics assessed visually by a pathologist. Conclusions: A multitask deep learning algorithm supervised by both patient outcome and biomarker status learned features in basic tissue morphology predictive of survival in a nationwide, multicenter series of patients with breast cancer. The algorithms generalized to another independent multicenter patient series and whole-slide breast cancer samples and provide prognostic information complementary to that of a comprehensive series of established prognostic factors.

Background: Digital pathology operations that precede viewing by a pathologist have a substantial impact on costs and fidelity of the digital image. Scan time and file size determine throughput and storage costs, whereas tissue omission during digital capture (“dropouts”) compromises downstream interpretation. We compared how these variables differ across scanners. Methods: A 212 slide set randomly selected from a gynecologic-gestational pathology practice was used to benchmark scan time, file size, and image completeness. Workflows included the Hamamatsu S210 scanner (operated under default and optimized profiles) and the Leica GT450. Digital tissue dropouts were detected by the aligned overlay of macroscopic glass slide camera images (reference) with images created by the slide scanners whole slide images. Results: File size and scan time were highly correlated within each platform. Differences in GT450, default S210, and optimized S210 performance were seen in average file size (1.4 vs. 2.5 vs. 3.4 GB) and scan time (93 vs. 376 vs. 721 s). Dropouts were seen in 29.5% (186/631) of successful scans overall: from a low of 13.7% (29/212) for the optimized S210 profile, followed by 34.6% (73/211) for the GT450 and 40.4% (84/208) for the default profile S210 profile. Small dislodged fragments, “shards,” were dropped in 22.2% (140/631) of slides, followed by tissue marginalized at the glass slide edges, 6.2% (39/631). “Unique dropouts,” those for which no equivalent appeared elsewhere in the scan, occurred in only three slides. Of these, 67% (2/3) were “floaters” or contaminants from other cases. Conclusions: Scanning speed and resultant file size vary greatly by scanner type, scanner operation settings, and clinical specimen mix (tissue type, tissue area). Digital image fidelity as measured by tissue dropout frequency and dropout type also varies according to the tissue type and scanner. Dropped tissues very rarely (1/631) represent actual specimen tissues that are not represented elsewhere in the scan, so in most cases cannot alter the diagnosis. Digital pathology platforms vary in their output efficiency and image fidelity to the glass original and should be matched to the intended application.

Background: Training convolutional neural networks using pathology whole slide images (WSIs) is traditionally prefaced by the extraction of a training dataset of image patches. While effective, for large datasets of WSIs, this dataset preparation is inefficient. Methods: We created a custom pipeline (histo-fetch) to efficiently extract random patches and labels from pathology WSIs for input to a neural network on-the-fly. We prefetch these patches as needed during network training, avoiding the need for WSI preparation such as chopping/tiling. Results & Conclusions: We demonstrate the utility of this pipeline to perform artificial stain transfer and image generation using the popular networks CycleGAN and ProGAN, respectively. For a large WSI dataset, histo-fetch is 98.6% faster to start training and used 7535x less disk space.

Background: The fast acquisition process of frozen sections allows surgeons to wait for histological findings during the interventions to base intrasurgical decisions on the outcome of the histology. Compared with paraffin sections, however, the quality of frozen sections is often strongly reduced, leading to a lower diagnostic accuracy. Deep neural networks are capable of modifying specific characteristics of digital histological images. Particularly, generative adversarial networks proved to be effective tools to learn about translation between two modalities, based on two unconnected data sets only. The positive effects of such deep learning-based image optimization on computer-aided diagnosis have already been shown. However, since fully automated diagnosis is controversial, the application of enhanced images for visual clinical assessment is currently probably of even higher relevance. Methods: Three different deep learning-based generative adversarial networks were investigated. The methods were used to translate frozen sections into virtual paraffin sections. Overall, 40 frozen sections were processed. For training, 40 further paraffin sections were available. We investigated how pathologists assess the quality of the different image translation approaches and whether experts are able to distinguish between virtual and real digital pathology. Results: Pathologists’ detection accuracy of virtual paraffin sections (from pairs consisting of a frozen and a paraffin section) was between 0.62 and 0.97. Overall, in 59% of images, the virtual section was assessed as more appropriate for a diagnosis. In 53% of images, the deep learning approach was preferred to conventional stain normalization (SN). Conclusion: Overall, expert assessment indicated slightly improved visual properties of converted images and a high similarity to real paraffin sections. The observed high variability showed clear differences in personal preferences.

Background: Population-based state cancer registries are an authoritative source for cancer statistics in the United States. They routinely collect a variety of data, including patient demographics, primary tumor site, stage at diagnosis, first course of treatment, and survival, on every cancer case that is reported across all U.S. states and territories. The goal of our project is to enrich NCI’s Surveillance, Epidemiology, and End Results (SEER) registry data with high-quality population-based biospecimen data in the form of digital pathology, machine-learning-based classifications, and quantitative histopathology imaging feature sets (referred to here as Pathomics features). Materials and Methods: As part of the project, the underlying informatics infrastructure was designed, tested, and implemented through close collaboration with several participating SEER registries to ensure consistency with registry processes, computational scalability, and ability to support creation of population cohorts that span multiple sites. Utilizing computational imaging algorithms and methods to both generate indices and search for matches makes it possible to reduce inter- and intra-observer inconsistencies and to improve the objectivity with which large image repositories are interrogated. Results: Our team has created and continues to expand a well-curated repository of high-quality digitized pathology images corresponding to subjects whose data are routinely collected by the collaborating registries. Our team has systematically deployed and tested key, visual analytic methods to facilitate automated creation of population cohorts for epidemiological studies and tools to support visualization of feature clusters and evaluation of whole-slide images. As part of these efforts, we are developing and optimizing advanced search and matching algorithms to facilitate automated, content-based retrieval of digitized specimens based on their underlying image features and staining characteristics. Conclusion: To meet the challenges of this project, we established the analytic pipelines, methods, and workflows to support the expansion and management of a growing repository of high-quality digitized pathology and information-rich, population cohorts containing objective imaging and clinical attributes to facilitate studies that seek to discriminate among different subtypes of disease, stratify patient populations, and perform comparisons of tumor characteristics within and across patient cohorts. We have also successfully developed a suite of tools based on a deep-learning method to perform quantitative characterizations of tumor regions, assess infiltrating lymphocyte distributions, and generate objective nuclear feature measurements. As part of these efforts, our team has implemented reliable methods that enable investigators to systematically search through large repositories to automatically retrieve digitized pathology specimens and correlated clinical data based on their computational signatures.

Background: The objective of the project was to evaluate the feasibility of introducing a single-networked digital histopathology reporting platform in the Southwest Peninsula region of England by allowing pathologists to experience the technology and recording their perceptions. This information was then used in planning future service development. The project was funded by the National Health Service (NHS) Peninsula Cancer Alliance and took place in 2020 during the COVID-19 pandemic. Materials and Methods: Digital slides of 500 cases from Taunton were reported remotely in Truro, Plymouth, Exeter, Bristol, or Bath by using a single remote reporting platform located on the secure Health and Social Care Network (HSCN) that links NHS sites. These were mainly small gastrointestinal, skin, and gynecological specimens. The digital diagnoses were compared with the diagnoses issued on reporting the glass slides. At the end of the project, the pathologists completed a Google Forms questionnaire of their perceptions of digital pathology. The results were presented at a meeting with the funder and discussed. Results: From the 500 cases there were nine cases of significant diagnostic discrepancy, seven of which involved the misrecognition of Helicobacter pylori in gastric biopsies. The questionnaire at the end of the project showed that there was a general agreement that the platform was easy to use, and the image quality was acceptable. It was agreed that extra work, such as deeper levels, was easy to request on the software platform. Most pathologists did not agree that digital reporting was quicker than glass slide reporting. Some were less confident in their digital diagnoses than glass diagnoses. They agreed that some types of specimens cannot easily be reported digitally. All users indicated that they would like to report at least half of their work digitally in the future if they could, and all strongly agreed that digital pathology would improve access to expert opinions, teaching, and multidisciplinary meetings. It was difficult to find pathologists with time to undertake remote digital reporting, in addition to their existing commitments. Conclusions: Overall, the pathologists developed a positive perception of digital pathology and wished to continue using it.

Background: Pathology reports serve as an auditable trial of a patient’s clinical narrative, containing text pertaining to diagnosis, prognosis, and specimen processing. Recent works have utilized natural language processing (NLP) pipelines, which include rule-based or machine-learning analytics, to uncover textual patterns that inform clinical endpoints and biomarker information. Although deep learning methods have come to the forefront of NLP, there have been limited comparisons with the performance of other machine-learning methods in extracting key insights for the prediction of medical procedure information, which is used to inform reimbursement for pathology departments. In addition, the utility of combining and ranking information from multiple report subfields as compared with exclusively using the diagnostic field for the prediction of Current Procedural Terminology (CPT) codes and signing pathologists remains unclear. Methods: After preprocessing pathology reports, we utilized advanced topic modeling to identify topics that characterize a cohort of 93,039 pathology reports at the Dartmouth-Hitchcock Department of Pathology and Laboratory Medicine (DPLM). We separately compared XGBoost, SVM, and BERT (Bidirectional Encoder Representation from Transformers) methodologies for the prediction of primary CPT codes (CPT 88302, 88304, 88305, 88307, 88309) as well as 38 ancillary CPT codes, using both the diagnostic text alone and text from all subfields. We performed similar analyses for characterizing text from a group of the 20 pathologists with the most pathology report sign-outs. Finally, we uncovered important report subcomponents by using model explanation techniques. Results: We identified 20 topics that pertained to diagnostic and procedural information. Operating on diagnostic text alone, BERT outperformed XGBoost for the prediction of primary CPT codes. When utilizing all report subfields, XGBoost outperformed BERT for the prediction of primary CPT codes. Utilizing additional subfields of the pathology report increased prediction accuracy across ancillary CPT codes, and performance gains for using additional report subfields were high for the XGBoost model for primary CPT codes. Misclassifications of CPT codes were between codes of a similar complexity, and misclassifications between pathologists were subspecialty related. Conclusions: Our approach generated CPT code predictions with an accuracy that was higher than previously reported. Although diagnostic text is an important source of information, additional insights may be extracted from other report subfields. Although BERT approaches performed comparably to the XGBoost approaches, they may lend valuable information to pipelines that combine image, text, and -omics information. Future resource-saving opportunities exist to help hospitals detect mis-billing, standardize report text, and estimate productivity metrics that pertain to pathologist compensation (RVUs).

Background: Pandemics are unpredictable and can rapidly spread. Proper planning and preparation for managing the impact of outbreaks is only achievable through continuous and systematic collection and analysis of health-related data. We describe our experience on how to comply with required reporting and develop a robust platform for surveillance data during an outbreak. Materials and Methods: At Mount Sinai Health System, New York City, we applied Visiun, a laboratory analytics dashboard, to support main response activities. Epic System Inc.’s SlicerDicer application was used to develop clinical and research reports. We followed World Health Organization (WHO); federal and state guidelines; departmental policies; and expert consultation to create the framework. Results: The developed dashboard integrated data from scattered sources are used to seamlessly distribute reports to key stakeholders. The main report categories included federal, state, laboratory, clinical, and research. The first two groups were created to meet government and state reporting requirements. The laboratory group was the most comprehensive category and included operational reports such as performance metrics, technician performance assessment, and analyzer metrics. The close monitoring of testing volumes and lab operational efficiency was essential to manage increasing demands and provide timely and accurate results. The clinical data reports were valuable for proper managing of medical surge requirements, such as healthcare workforce and medical supplies. The reports included in the research category were highly variable and depended on healthcare setting, research priorities, and available funding. We share a few examples of queries that were included in the designed framework for research projects. Conclusion: We reviewed here the key components of a conceptual surveillance framework required for a robust response to COVID-19 pandemics. We demonstrated leveraging a lab analytics dashboard, Visiun, combined with Epic reporting tools to function as a surveillance system. The framework could be used as a generic template for possible future outbreak events.

Bioinformatics analysis is a key element in the development of in-house next-generation sequencing assays for tumor genetic profiling that can include both tumor DNA and RNA with comparisons to matched-normal DNA in select cases. Bioinformatics analysis encompasses a computationally heavy component that requires a high-performance computing component and an assay-dependent quality assessment, aggregation, and data cleaning component. Although there are free, open-source solutions and fee-for-use commercial services for the computationally heavy component, these solutions and services can lack the options commonly utilized in increasingly complex genomic assays. Additionally, the cost to purchase commercial solutions or implement and maintain open-source solutions can be out of reach for many small clinical laboratories. Here, we present Software for Clinical Health in Oncology for Omics Laboratories (SCHOOL), a collection of genomics analysis workflows that (i) can be easily installed on any platform; (ii) run on the cloud with a user-friendly interface; and (iii) include the detection of single nucleotide variants, insertions/deletions, copy number variants (CNVs), and translocations from RNA and DNA sequencing. These workflows contain elements for customization based on target panel and assay design, including somatic mutational analysis with a matched-normal, microsatellite stability analysis, and CNV analysis with a single nucleotide polymorphism backbone. All of the features of SCHOOL have been designed to run on any computer system, where software dependencies have been containerized. SCHOOL has been built into apps with workflows that can be run on a cloud platform such as DNANexus using their point-and-click graphical interface, which could be automated for high-throughput laboratories.