Context : Content-based image retrieval (CBIR) systems allow for retrieval of images from within a database that are similar in visual content to a query image. This is useful for digital pathology, where text-based descriptors alone might be inadequate to accurately describe image content. By representing images via a set of quantitative image descriptors, the similarity between a query image with respect to archived, annotated images in a database can be computed and the most similar images retrieved. Recently, non-linear dimensionality reduction methods have become popular for embedding high-dimensional data into a reduced-dimensional space while preserving local object adjacencies, thereby allowing for object similarity to be determined more accurately in the reduced-dimensional space. However, most dimensionality reduction methods implicitly assume, in computing the reduced-dimensional representation, that all features are equally important. Aims : In this paper we present boosted spectral embedding (BoSE), which utilizes a boosted distance metric to selectively weight individual features (based on training data) to subsequently map the data into a reduced-dimensional space. Settings and Design : BoSE is evaluated against spectral embedding (SE) (which employs equal feature weighting) in the context of CBIR of digitized prostate and breast cancer histopathology images. Materials and Methods : The following datasets, which were comprised of a total of 154 hematoxylin and eosin stained histopathology images, were used: (1) Prostate cancer histopathology (benign vs. malignant), (2) estrogen receptor (ER) + breast cancer histopathology (low vs. high grade), and (3) HER2+ breast cancer histopathology (low vs. high levels of lymphocytic infiltration). Statistical Analysis Used : We plotted and calculated the area under precision-recall curves (AUPRC) and calculated classification accuracy using the Random Forest classifier. Results : BoSE outperformed SE both in terms of CBIR-based (area under the precision-recall curve) and classifier-based (classification accuracy) on average across all of the dimensions tested for all three datasets: (1) Prostate cancer histopathology (AUPRC: BoSE = 0.79, SE = 0.63; Accuracy: BoSE = 0.93, SE = 0.80), (2) ER + breast cancer histopathology (AUPRC: BoSE = 0.79, SE = 0.68; Accuracy: BoSE = 0.96, SE = 0.96), and (3) HER2+ breast cancer histopathology (AUPRC: BoSE = 0.54, SE = 0.44; Accuracy: BoSE = 0.93, SE = 0.91). Conclusion : Our results suggest that BoSE could serve as an important tool for CBIR and classification of high-dimensional biomedical data.

Background: In many surgical pathology laboratories, operating room schedules are prospectively reviewed to determine specimen distribution to different subspecialty services and to predict the number and nature of potential intraoperative consultations for which prior medical records and slides require review. At our institution, such schedules were manually converted into easily interpretable, surgical pathology-friendly reports to facilitate these activities. This conversion, however, was time-consuming and arguably a non-value-added activity. Objective: Our goal was to develop a semi-automated method of generating these reports that improved their readability while taking less time to perform than the manual method. Materials and Methods: A dynamic Microsoft Excel workbook was developed to automatically convert published operating room schedules into different tabular formats. Based on the surgical procedure descriptions in the schedule, a list of linked keywords and phrases was utilized to sort cases by subspecialty and to predict potential intraoperative consultations. After two trial-and-optimization cycles, the method was incorporated into standard practice. Results: The workbook distributed cases to appropriate subspecialties and accurately predicted intraoperative requests. Users indicated that they spent 1-2 h fewer per day on this activity than before, and team members preferred the formatting of the newer reports. Comparison of the manual and semi-automatic predictions showed that the mean daily difference in predicted versus actual intraoperative consultations underwent no statistically significant changes before and after implementation for most subspecialties. Conclusions: A well-designed, lean, and simple information technology solution to determine subspecialty case distribution and prediction of intraoperative consultations in surgical pathology is approximately as accurate as the gold standard manual method and requires less time and effort to generate.

Introduction: Nucleolar changes in cancer cells are one of the cytologic features important to the tumor pathologist in cancer assessments of tissue biopsies. However, inter-observer variability and the manual approach to this work hamper the accuracy of the assessment by pathologists. In this paper, we propose a computational method for prominent nucleoli pattern detection. Materials and Methods: Thirty-five hematoxylin and eosin stained images were acquired from prostate cancer, breast cancer, renal clear cell cancer and renal papillary cell cancer tissues. Prostate cancer images were used for the development of a computer-based automated prominent nucleoli pattern detector built on a cascade farm. An ensemble of approximately 1000 cascades was constructed by permuting different combinations of classifiers such as support vector machines, eXclusive component analysis, boosting, and logistic regression. The output of cascades was then combined using the RankBoost algorithm. The output of our prominent nucleoli pattern detector is a ranked set of detected image patches of patterns of prominent nucleoli. Results: The mean number of detected prominent nucleoli patterns in the top 100 ranked detected objects was 58 in the prostate cancer dataset, 68 in the breast cancer dataset, 86 in the renal clear cell cancer dataset, and 76 in the renal papillary cell cancer dataset. The proposed cascade farm performs twice as good as the use of a single cascade proposed in the seminal paper by Viola and Jones. For comparison, a naive algorithm that randomly chooses a pixel as a nucleoli pattern would detect five correct patterns in the first 100 ranked objects. Conclusions: Detection of sparse nucleoli patterns in a large background of highly variable tissue patterns is a difficult challenge our method has overcome. This study developed an accurate prominent nucleoli pattern detector with the potential to be used in the clinical settings.

Background: Pathology reports typically require manual review to abstract research data. We developed a natural language processing (NLP) system to automatically interpret free-text breast pathology reports with limited assistance from manual abstraction. Methods: We used an iterative approach of machine learning algorithms and constructed groups of related findings to identify breast-related procedures and results from free-text pathology reports. We evaluated the NLP system using an all-or-nothing approach to determine which reports could be processed entirely using NLP and which reports needed manual review beyond NLP. We divided 3234 reports for development (2910, 90%), and evaluation (324, 10%) purposes using manually reviewed pathology data as our gold standard. Results: NLP correctly coded 12.7% of the evaluation set, flagged 49.1% of reports for manual review, incorrectly coded 30.8%, and correctly omitted 7.4% from the evaluation set due to irrelevancy (i.e. not breast-related). Common procedures and results were identified correctly (e.g. invasive ductal with 95.5% precision and 94.0% sensitivity), but entire reports were flagged for manual review because of rare findings and substantial variation in pathology report text. Conclusions: The NLP system we developed did not perform sufficiently for abstracting entire breast pathology reports. The all-or-nothing approach resulted in too broad of a scope of work and limited our flexibility to identify breast pathology procedures and results. Our NLP system was also limited by the lack of the gold standard data on rare findings and wide variation in pathology text. Focusing on individual, common elements and improving pathology text report standardization may improve performance.

Background: Ontology is one strategy for promoting interoperability of heterogeneous data through consistent tagging. An ontology is a controlled structured vocabulary consisting of general terms (such as "cell" or "image" or "tissue" or "microscope") that form the basis for such tagging. These terms are designed to represent the types of entities in the domain of reality that the ontology has been devised to capture; the terms are provided with logical defi nitions thereby also supporting reasoning over the tagged data. Aim: This paper provides a survey of the biomedical imaging ontologies that have been developed thus far. It outlines the challenges, particularly faced by ontologies in the fields of histopathological imaging and image analysis, and suggests a strategy for addressing these challenges in the example domain of quantitative histopathology imaging. Results and Conclusions: The ultimate goal is to support the multiscale understanding of disease that comes from using interoperable ontologies to integrate imaging data with clinical and genomics data.

Background: The human protein atlas (HPA) is a powerful proteomic tool for visualizing the distribution of protein expression across most human tissues and many common malignancies. The HPA includes immunohistochemically-stained images from tissue microarrays (TMAs) that cover 48 tissue types and 20 common malignancies. The TMA data are used to provide expression information at the tissue, cellular, and occasionally, subcellular level. The HPA also provides subcellular data from confocal immunofluorescence data on three cell lines. Despite the availability of localization data, many unique patterns of cellular and subcellular expression are not documented. Materials and Methods: To get at this more granular data, we have developed a suite of Python scripts, HPASubC, to aid in subcellular, and cell-type specific classification of HPA images. This method allows the user to download and optimize specific HPA TMA images for review. Then, using a playstation-style video game controller, a trained observer can rapidly step through 10's of 1000's of images to identify patterns of interest. Results: We have successfully used this method to identify 703 endothelial cell (EC) and/or smooth muscle cell (SMCs) specific proteins discovered within 49,200 heart TMA images. This list will assist us in subdividing cardiac gene or protein array data into expression by one of the predominant cell types of the myocardium: Myocytes, SMCs or ECs. Conclusions: The opportunity to further characterize unique staining patterns across a range of human tissues and malignancies will accelerate our understanding of disease processes and point to novel markers for tissue evaluation in surgical pathology.

Background: Mobile networks and smartphones are growing in developing countries. Expert telemedicine consultation will become more convenient and feasible. We wanted to report on our experience in using a smartphone and a 3-D printed adapter for capturing microscopic images. Methods: Images and videos from a gastrointestinal biopsy teaching set of referred cases from the AFIP were captured with an iPhone 5 smartphone fitted with a 3-D printed adapter. Nine pathologists worldwide evaluated the images for quality, adequacy for telepathology consultation, and confidence rendering a diagnosis based on the images viewed on the web. Results: Average Likert scales (ordinal data) for image quality (1=poor, 5=diagnostic) and adequacy for diagnosis (1=No, 5=Yes) had modes of 3 and 4, respectively. Adding a video overview of the specimen improved diagnostic confidence. The mode of confidence in diagnosis based on the images reviewed was four. In 31 instances, reviewers' diagnoses completely agreed with AFIP diagnosis, with partial agreement in 9 and major disagreement in 5. There was strong correlation between image quality and confidence (r = 0.78), image quality and adequacy of image (r = 0.73) and whether images were found adequate when reviewers were confident (r = 0.72). Intraclass Correlation for measuring reliability among the four reviewers who finished a majority of cases was high (quality=0.83, adequacy= 0.76 and confidence=0.92). Conclusions: Smartphones allow pathologists and other image dependent disciplines in low resource areas to transmit consultations to experts anywhere in the world. Improvements in camera resolution and training may mitigate some limitations found in this study.

Aims: In Japan, cancer is the most prevalent cause of death; the number of patients suffering from cancer is increasing. Hence, there is an increased burden on pathologists to make diagnoses. To reduce pathologists' burden, researchers have developed methods of auto-pathological diagnosis. However, virtual slides, which are created when glass slides are digitally scanned, saved in a unique format, and it is difficult for researchers to work on the virtual slides for developing their own image processing method. This paper presents the support system for pathologists and researchers who use auto-pathological diagnosis (P-SSD). Main purpose of P-SSD was to support both of pathologists and researchers. P-SSD consists of several sub-functions that make it easy not only for pathologists to screen pathological images, double-check their diagnoses, and reduce unimportant image data but also for researchers to develop and apply their original image-processing techniques to pathological images. Methods: We originally developed P-SSD to support both pathologists and researchers developing auto-pathological diagnoses systems. Current version of P-SSD consists of five main functions as follows: (i) Loading virtual slides, (ii) making a supervised database, (iii) learning image features, (iv) detecting cancerous areas, (v) displaying results of detection. Results: P-SSD reduces computer memory size random access memory utilization and the processing time required to divide the virtual slides into the smaller-size images compared with other similar software. The maximum observed reduction in computer memory size and reduction in processing time is 97% and 99.94%, respectively. Conclusions: Unlike other vendor-developed software, P-SSD has interoperability and is capable of handling virtual slides in several formats. Therefore, P-SSD can support both of pathologists and researchers, and has many potential applications in both pathological diagnosis and research area.

Hematoxylin and eosin (H&E) staining is ubiquitous in pathology practice and research. As digital pathology has evolved, the reliance of quantitative methods that make use of H&E images has similarly expanded. For example, cell counting and nuclear morphometry rely on the accurate demarcation of nuclei from other structures and each other. One of the major obstacles to quantitative analysis of H&E images is the high degree of variability observed between different samples and different laboratories. In an effort to characterize this variability, as well as to provide a substrate that can potentially mitigate this factor in quantitative image analysis, we developed a technique to project H&E images into an optimized space more appropriate for many image analysis procedures. We used a decision tree-based support vector machine learning algorithm to classify 44 H&E stained whole slide images of resected breast tumors according to the histological structures that are present. This procedure takes an H&E image as an input and produces a classification map of the image that predicts the likelihood of a pixel belonging to any one of a set of user-defined structures (e.g., cytoplasm, stroma). By reducing these maps into their constituent pixels in color space, an optimal reference vector is obtained for each structure, which identifies the color attributes that maximally distinguish one structure from other elements in the image. We show that tissue structures can be identified using this semi-automated technique. By comparing structure centroids across different images, we obtained a quantitative depiction of H&E variability for each structure. This measurement can potentially be utilized in the laboratory to help calibrate daily staining or identify troublesome slides. Moreover, by aligning reference vectors derived from this technique, images can be transformed in a way that standardizes their color properties and makes them more amenable to image processing.

As a result of their practice of medicine, forensic pathologists create a wealth of data regarding the causes of and reasons for sudden, unexpected or violent deaths. This data have been effectively used to protect the health and safety of the general public in a variety of ways despite current and historical limitations. These limitations include the lack of data standards between the thousands of death investigation (DI) systems in the United States, rudimentary electronic information systems for DI, and the lack of effective communications and interfaces between these systems. Collaboration between forensic pathology and clinical informatics is required to address these shortcomings and a path forward has been proposed that will enable forensic pathology to maximize its effectiveness by providing timely and actionable information to public health and public safety agencies.

Background: Optimization of workflow for breast cancer samples with equivocal human epidermal growth factor receptors 2 (HER2)/neu score 2 ⁺ results in routine practice, remains to be a central focus of the on-going efforts to assess HER2 status. According to the College of American Pathologists/American Society of Clinical Oncology guidelines equivocal HER2/neu score 2 ⁺ cases are subject for further testing, usually by fluorescence in situ hybridization (FISH) investigations. It still remains on open question, whether quantitative digital image analysis of HER2 immunohistochemistry (IHC) stained slides can assist in further refining the HER2 score 2 ⁺ . Aim of this Work: To assess utility of quantitative digital analysis of IHC stained slides and compare its performance to FISH in cases of breast cancer with equivocal HER2 score 2 ⁺ . Materials and Methods: Fifteen specimens (previously diagnosed as breast cancer and was evaluated as HER 2^– score 2 ⁺) represented the study population. Contemporary new cuts were prepared for re-evaluation of HER2 immunohistochemical studies and FISH examination. All the cases were digitally scanned by iScan (Produced by BioImagene [Now Roche-Ventana]). The IHC signals of HER2 were measured using an automated image analyzing system (MECES, www.Diagnomx.eu/meces). Finally, a comparative study was done between the results of the FISH and the quantitative analysis of the virtual slides. Results: Three out of the 15 cases with equivocal HER2 score 2 ⁺ , turned out to be positive (3 ⁺) by quantitative digital analysis, and 12 were found to be negative in FISH too. Two of these three positive cases proved to be positive with FISH, and only one was negative. Conclusions: Quantitative digital analysis is highly sensitive and relatively specific when compared to FISH in detecting HER2/neu overexpression. Therefore, it represents a potential reliable substitute for FISH in breast cancer cases, which desire further refinement of equivocal IHC results.

Background: The pathology laboratory course at the New York College of Podiatric Medicine involves the use of Virtual Microscopy. The students can scan the whole slide, section by section, and zoom in or out. Methods: Using the advantages of digital pathology, the students can, in addition, access the slide collections from other medical schools and put up normal histology (control) slides side-by-side with the pathology. They can cut and paste and preserve the region of interest that they find. They can edit and annotate their slides. A wiki was created (http://pathlab2014.wikifoundry.com) for the Class of 2014. The students saved, edited and uploaded their slides. In the wiki format, other students could comment, further edit, and even delete the slides. Results: The students studied Basic Mechanisms and System Pathology. During this time, they saved, edited, shared, and uploaded their slides to the wiki. These were available in one full presentation and were also grouped into 16 albums. They were available to all. Student access was followed by Google analytics. At the end of the course, a questionnaire was distributed, assessing their impression of the wiki format and soliciting strengths and weaknesses. Conclusions: The use of a wiki has a number of important advantages in pathology education. It trains the students in the more sophisticated skills that they will use as professional pathologists or as clinicians: (1) Telepathology-it enables them to share slides and discuss observations. (2) Archiving and retrieval - It models the challenge faced by hospitals, diagnostic labs and physicians in maintaining a collection of slides in a form that is easily accessible. (3) Image analysis-familiarity with the wiki format allows them to jump easily to capturing and storing images found in the literature or in a pathologist's report. Experience with the use of a wiki in pathology education has been quite satisfactory from both the faculty and the student's point of view.

The digitization of images has not only led to increasingly sophisticated methods of quantitating information from those images themselves, but also to the development of new physics-based techniques for extracting information from the original specimen and presenting this as visual data in both two and three-dimensional (3D) forms. This evolution of an image-based discipline has reached maturity in Radiology, but it is only just beginning in Pathology. An historical perspective is provided both on the current state of computational imaging in pathology and of the factors that are impeding further progress in the development and application of these approaches. Emphasis is placed on barriers to the dissemination of information in this area. The value of computational imaging in basic and translational research is clear. However, while there are many examples of "virtual diagnostics" in Radiology, there are only relatively few in Pathology. Nevertheless, we can do cellular level analysis of lesions accessible by endoscopic or catheterization procedures, and a number of steps have been taken toward real-time imaging as adjuncts to traditional biopsies. Progress in computational imaging will greatly expand the role of pathologists in clinical medicine as well as research.

The way of viewing whole slide images (WSI) can be tracked and analyzed. In particular, it can be useful to learn how medical students view WSIs during exams and how their viewing behavior is correlated with correctness of the answers they give. We used software-based view path tracking method that enabled gathering data about viewing behavior of multiple simultaneous WSI users. This approach was implemented and applied during two practical exams in oral pathology in 2012 (88 students) and 2013 (91 students), which were based on questions with attached WSIs. Gathered data were visualized and analyzed in multiple ways. As a part of extended analysis, we tried to use machine learning approaches to predict correctness of students' answers based on how they viewed WSIs. We compared the results of analyses for years 2012 and 2013 - done for a single question, for student groups, and for a set of questions. The overall patterns were generally consistent across these 3 years. Moreover, viewing behavior data appeared to have certain potential for predicting answers' correctness and some outcomes of machine learning approaches were in the right direction. However, general prediction results were not satisfactory in terms of precision and recall. Our work confirmed that the view path tracking method is useful for discovering viewing behavior of students analyzing WSIs. It provided multiple useful insights in this area, and general results of our analyses were consistent across two exams. On the other hand, predicting answers' correctness appeared to be a difficult task - students' answers seem to be often unpredictable.

Visualizing tissue structures in three-dimensions (3D) is crucial to understanding normal and pathological phenomena. However, staining and imaging of thick sections and whole mount samples can be challenging. For decades, researchers have serially sectioned large tissues and painstakingly reconstructed the 3D volume. Advances in automation, from sectioning to alignment, now greatly accelerate the process. In addition, immunofluorescent staining methods allow multiple antigens to be simultaneously detected and analyzed volumetrically. The objective was to incorporate multi-channel immunofluorescent staining and automation in 3D reconstruction of serial sections for volumetric analysis. Paraffin-embedded samples were sectioned manually but were processed, stained, imaged and aligned in an automated fashion. Reconstructed stacks were quantitatively analyzed in 3D. By combining automated immunofluorescent staining and tried-and-true methods of reconstructing adjacent sections, we were able to visualize, in detail, not only the geometric structures of the sample but also the presence and interactions of multiple proteins and molecules of interest within their 3D environment. Advances in technology and software algorithms have significantly expedited the 3D reconstruction of serial sections. Automated, multi-antigen immunofluorescent staining will significantly broaden the range and complexity of scientific questions that can be answered with this methodology.

Background: Recent breakthroughs in computer vision and digital microscopy have prompted the application of such technologies in cancer diagnosis, especially in histopathological image analysis. Earlier, an attempt to classify hepatocellular carcinoma images based on nuclear and structural features has been carried out on a set of surgical resected samples. Here, we proposed methods to enhance the process and improve the classification performance. Methods: First, we segmented the histological components of the liver tissues and generated several masked images. By utilizing the masked images, some set of new features were introduced, producing three sets of features consisting nuclei, trabecular and tissue changes features. Furthermore, we extended the classification process by using biopsy resected samples in addition to the surgical samples. Results: Experiments by using support vector machine (SVM) classifier with combinations of features and sample types showed that the proposed methods improve the classification rate in HCC detection for about 1-3%. Moreover, detection rate of low-grades cancer increased when the new features were appended in the classification process, although the rate was worsen in the case of undifferentiated tumors. Conclusions: The masking process increased the reliability of extracted nuclei features. The additional of new features improved the system especially for early HCC detection. Likewise, the combination of surgical and biopsy samples as training data could also improve the classification rates. Therefore, the methods will extend the support for pathologists in the HCC diagnosis.

Background: Whole-slide imaging (WSI), while technologically mature, remains in the early adopter phase of the technology adoption lifecycle. One reason for this current situation is that current methods of visualizing and using WSI closely follow long-existing workflows for glass slides. We set out to "reimagine" the digital microscope in the era of cloud computing by combining WSI with the rich collaborative environment of the Scalable Adaptive Graphics Environment (SAGE). SAGE is a cross-platform, open-source visualization and collaboration tool that enables users to access, display and share a variety of data-intensive information, in a variety of resolutions and formats, from multiple sources, on display walls of arbitrary size. Methods: A prototype of a WSI viewer app in the SAGE environment was created. While not full featured, it enabled the testing of our hypothesis that these technologies could be blended together to change the essential nature of how microscopic images are utilized for patient care, medical education, and research. Results: Using the newly created WSI viewer app, demonstration scenarios were created in the patient care and medical education scenarios. This included a live demonstration of a pathology consultation at the International Academy of Digital Pathology meeting in Boston in November 2014. Conclusions: SAGE is well suited to display, manipulate and collaborate using WSIs, along with other images and data, for a variety of purposes. It goes beyond how glass slides and current WSI viewers are being used today, changing the nature of digital pathology in the process. A fully developed WSI viewer app within SAGE has the potential to encourage the wider adoption of WSI throughout pathology.