|J Pathol Inform 2014,
Validation of a whole slide imaging system for primary diagnosis in surgical pathology: A community hospital experience
Thomas P Buck, Rebecca Dilorio, Lauren Havrilla, Dennis G O'Neill
Department of Pathology Eastern Connecticut Health Network, Manchester Memorial Hospital, 71 Haynes Street, Manchester, Connecticut, USA
|Date of Submission||12-Jun-2014|
|Date of Acceptance||07-Sep-2014|
|Date of Web Publication||28-Nov-2014|
Thomas P Buck
Department of Pathology Eastern Connecticut Health Network, Manchester Memorial Hospital, 71 Haynes Street, Manchester, Connecticut
Source of Support: None, Conflict of Interest: None
| Abstract|| |
Guidelines for validating whole slide imaging (WSI) for primary diagnosis in surgical pathology have been recommended by an expert panel commissioned by the College of American Pathologists. The implementation of such a system using these validation guidelines has not been reported from the community hospital setting. The objective was to implement a WSI system, validate each pathologist using the system and run the system in parallel with routine glass slide interpretation. Six pathologists re-reviewed approximately 300 previously diagnosed specimens each, divided equally between glass slides and digital images (scanned at Χ20). Baseline intraobserver discordance rates (glass to glass) were calculated and compared to discordance rates between the original glass slide interpretation and the reviewed digital slide interpretation. A minimum of 3 months was used as the washout period. After validation, a subset of daily cases was diagnosed in parallel using traditional microscopy (TM) and WSI over an 8-month period. The TM and WSI discordance rates ranged from 3.3% to 13.3% and 2.1% to 10.1%, respectively. There was no statistically significant difference among the pathologists. The parallel study yielded similar rates of discordances. In our laboratory, after appropriate implementation and training, there was no difference between the WSI and TM methods.
Keywords: Digital pathology, telepathology, whole slide imaging
|How to cite this article:|
Buck TP, Dilorio R, Havrilla L, O'Neill DG. Validation of a whole slide imaging system for primary diagnosis in surgical pathology: A community hospital experience. J Pathol Inform 2014;5:43
|How to cite this URL:|
Buck TP, Dilorio R, Havrilla L, O'Neill DG. Validation of a whole slide imaging system for primary diagnosis in surgical pathology: A community hospital experience. J Pathol Inform [serial online] 2014 [cited 2017 May 23];5:43. Available from: http://www.jpathinformatics.org/text.asp?2014/5/1/43/145731
| Introduction|| |
Whole slide imaging (WSI) for use in the practice of surgical pathology has been selectively investigated and used over the past decade in the areas of frozen section diagnosis, immunohistochemistry interpretation, consultation and routine diagnosis, principally in academic settings. ,,,,,,, Building on that experience, the College of American Pathologists (CAP) commissioned an expert panel to develop guidelines for the validation of WSI for primary diagnosis in pathology.  In the United States, this technology has been used sparingly in the clinical setting. Much of the practical clinical experience with this technology comes from the European Union and Canada where acceptance and utilization have outpaced that in the U.S. With the newly recommended guidelines, the path is clearer for the validation and implementation of this technology for primary diagnosis. Herein, we report our experience validating this technology for primary diagnosis in a community hospital setting.
| Materials and Methods|| |
Our laboratory utilizes an Aperio AT Turbo scanner, Spectrum Version 18.104.22.1680 slide manager software, ImageScope Version 22.214.171.1240 digital slide viewer and a Microsoft Windows 7 platform. Our intent was to validate the system using the recently recommended guidelines and, at the same time, validate each pathologist (not required in the guidelines) who would be using the system. The digital slide scanner was installed and set up according to the manufacturer's specifications. For validation purposes, we scanned the entire slide, regardless of tissue size, at ×20 magnification; re-scanning at ×40 was available on request. A snapshot was taken of each slide which included the slide label, the patient name and the surgical pathology accession number. The unit of comparison used in all phases of the study was a specimen part (or "part"). An individual case could have one or more parts (e.g. an upper endoscopy case could have several parts from different anatomic locations).
Our study initially contained three separate phases: An educational phase during which the six members of our pathology group learned how to use digital imaging technology; the WSI validation phase using the recommended guidelines for primary diagnosis; and the parallel study phase during which we gained additional experience in WSI interpretation. After we had started the study, we added a digital cropping validation procedure because several of our pathologists felt uncomfortable with the possibility of "losing" tissue during the transition from the glass slide to digital image. Each of these phases in the study had its own procedural steps as outlined below.
Prior to validation, each pathologist completed a 43-part study set, which included glass slides stained with hematoxylin and eosin (H & E) and a variety of immunohistochemical (IHC) stains. These slides were selected and scanned by laboratory technical staff and included parts of nonconsecutive benign and malignant cases representing a variety of specimen sizes (needle biopsies, small diagnostic biopsies and resections) and organ sites (breast, gastrointestinal [GI], prostate, genitourinary and lung). Each pathologist was required to review the digital images and compare his/her diagnoses with the surgical pathology report. The results of these comparisons were not included in the subsequent validation phase.
To validate routine H & E-stained slides, in accordance with the recent recommendations of the expert panel,  approximately 60 nonconsecutive cases/150 parts for traditional microscopy ("TM" or glass to glass) and a similar number of cases and parts for whole slide imaging ("WSI" or glass to digital), totaling approximately 120 cases/300 parts for each pathologist, were selected. The total number of cases and parts varied slightly by pathologist because we wanted to ensure an appropriate case distribution and did not want to disrupt the completeness of any given case. Different parts were used for both glass slide and digital slide review. Each pathologist reviewed only his/her own cases. As with the educational study set, the parts were selected to reflect a representative cross-section of benign and malignant specimens from our laboratory and included the following: Routine diagnostic biopsies (breast, GI, prostate, genitourinary and gynecologic); routine small surgical specimens (skin, appendix, gallbladder); and routine resections (breast, colon, lung, prostate, thyroid). The washout period between initial diagnosis and re-review was 3-6 months. All tissue was formalin-fixed and paraffin-embedded. All slides were reviewed, including levels on individual paraffin blocks and special histochemical and IHC stains. A working draft (including the clinical history, specimen site and gross description) was provided for each case together with a spreadsheet listing the individual part types with a column for the review diagnosis. The initial diagnosis/diagnoses were later entered by a second person and the results for TM and WSI were compared. If multiple diagnoses were appropriate and only a portion of the diagnosis was discordant, the entire part was marked as discordant. Discordances were categorized as minor (no change in patient management) or major (change in patient management). An example of a minor discordance was the presence or absence of active inflammation in the upper GI tract with concurrent chronic inflammation. In our experience, the additional finding of active inflammation in the upper tract does not substantively change clinical management; an isolated finding of active inflammation, however, in the upper or lower tract does typically change clinical management and, if missed, was considered a major discordance. Other examples of a major discordance included a hyperplastic polyp versus a tubular adenoma in the colon or a benign versus a malignant diagnosis. The TM arm variance rate served to inform us of each pathologist's degree of intraobserver variability. The WSI variance rate was then compared to each pathologist's baseline variance rate. Our null hypothesis was that there would be no difference between the two methods. Statistical P values (Fisher-exact test) were determined utilizing 2014 GraphPad software (http://graphpad.com).
To validate special histochemical and IHC stains, slides from 38 parts representing 20 cases in accordance with the recommended guidelines were scanned and evaluated by each pathologist. While not all-inclusive, the stains selected were representative of our laboratory's typical specimen mix. Similar to the H & E procedure described above, the results of the reviews were tabulated in a spreadsheet and compared to the initial interpretations.
Although for the validation phase we scanned the entire slide, we also validated a cropping procedure in anticipation of using this feature during the parallel phase of the study.
We scanned 100 H & E slides of varying specimen types and sizes using the Aperio "whole slide" setting and re-scanned the slides after cropping by a member of the laboratory technical staff. The test sample included needle biopsies, small diagnostic biopsies and major resection specimens. Five different laboratory technicians were evaluated. The cropped and uncropped images of each slide were compared by a pathologist.
Parallel Study Phase
After we had completed the validation phase of the study, we felt that we still needed practice on the system before its real-time integration into the workflow of the department. We likened this process to a new driver needing practice behind the wheel even after receiving a license to drive a motor vehicle. Good drivers are, usually, not created at the moment they receive a license but only after logging many hours driving in a variety of weather conditions and traffic patterns. Hence, we decided to review cases in parallel for another 8 months. This phase involved one or more rotating pathologists reviewing 10-20 digitally scanned slides each morning before reviewing the corresponding glass slides and signing out the cases. The material done in parallel reflected nonconsecutive cases and consisted only of diagnostic biopsies of the GI tract. The results of the digital and glass slide interpretations were compared in real-time, and any discordance was recorded on a spreadsheet.
| Results|| |
All pathologists reported basic facility with the WSI system after completing the 43-part study set.
After the pathologist validation process, we found that the intraobserver discordance rates were similar between the TM and WSI methods [Table 1]. There was no trend as to when in the validation process the discordances occurred; that is, no learning curve was identified. Among the six pathologists, the TM discordance rates varied from 3.3% to 13.3% (average 7.5%). The WSI discordance rates varied from 2.1% to 10.1% (average 6.9%). There was one major discordance which involved a missed piece of tissue on the digital image of a section of endocervix containing a small focus of endometrial adenocarcinoma; sections of the endometrium from the same case contained clearly identified endometrial adenocarcinoma. The missed tissue fragment was minute and separate from the larger fragment of benign endocervix. This discordance was attributed to pathologist error.
The validation of the special histochemical stains involved four pathologists reviewing 38 parts prepared from a variety of stains, including acid-fast, alcian blue/PAS, Brown-Brenn, Congo red, methenamine silver, iron, Masson trichrome, mucicarmine, reticulin and Warthin-Starry stains. Only one acid-fast stain was incorrectly interpreted as negative; there were no false positive interpretations.
Two croppers inadequately cropped two specimens out of 100, one cropper inadequately cropped one specimen out of 100 and the remaining two croppers made no mistakes. The inadequately cropped specimens consisted of breast tissue with a rim of nearly transparent adipose tissue on the slide, the edge of which was missed by the cropper or, in one case, a tissue edge that went beyond the coverslip. We concluded that all croppers were competent.
Parallel Study Phase
During this phase, there were 1545 additional digitized parts reviewed and dictated prior to reconciling the interpretation with the corresponding glass slides. Spreadsheets were completed by the pathologists, and any discordance was indicated in the comment section. There were 1522 parts in agreement (98.5%) with 13 minor discordances and 10 major discordances for a combined discordance rate of 1.5% [Table 2]. The majority of the discordances involved the failure to identify granulocytes (neutrophils or eosinophils) on the digital image. Of note, one major discordance involved a small focus of carcinoid tumor that was missed on the digital image of a colon polyp biopsy. In addition, two parts had cropping errors in which tissue fragments were not scanned (0.13%).
| Comment|| |
In summary, we successfully validated a digital imaging system for primary diagnosis in surgical pathology using the guidelines developed by an expert panel and recommended to the CAP. We took the extra step of validating each pathologist because we wanted to ensure acceptance of, and facility with, the system across all users. The actual validation process took 4 months to complete. Regarding the parallel study phase, we concluded that the discordance rate was within an acceptable limit in that it was less than the average TM and WSI discordance rates (1.5% compared to 7.5 and 6.9%, respectively) and was less than previously reported intraobserver discordance rates.  In addition to the interpretive discordances identified during both the validation and parallel study phases, a variety of workflow issues also became apparent. As with most transformational technological systems, lack of familiarity with the new technology can hinder optimal workflow, and the process of examining tissue on a computer screen can reduce overall productivity. Slide review is performed more slowly, and confidence in the interpretation is reduced. In fact, all pathologists reported during the parallel study phase that they occasionally resorted to the glass slide before dictating the case. This, usually, occurred during the interpretation of diagnostically difficult cases. Other specific areas related to workflow are described below.
Mechanical Manipulation of the Image
For a surgical pathologist, the process of reviewing a glass slide by TM mainly involves two mechanical components: Moving the slide around the microscope stage and focusing up and down on a particular area of the slide. These components are performed simultaneously and seamlessly by the practiced pathologist in a fluid dance that allows for high diagnostic accuracy coupled with rapid throughput. We found that converting this process to the computer screen required both time and patience with each pathologist developing his/her own techniques, much the same as we have done with TM over many years of practice. Navigation of the digital image is user-dependent and involved different input devices and different magnification techniques. Some pathologists preferred traditional mouse devices while others adapted to newer devices such as the -3D connexion mouse. Some pathologists preferred to use the scroll feature on the mouse for magnification while several relied on the Image Scope magnification bar.
Color Fidelity of the Image
Even in the best of laboratories, the quality of the standard H & E stain can vary from day to day depending on, among other variables, the staining reagents used, the staining protocol employed and the degree to which the staining process is automated. Surgical pathologists have learned to adjust to these minor variations. But we found that the colors in a digital image introduced another variable that required adjustment. For example, the blue color of the transmitted image, as determined by the hematoxylin reagent, was more intense, with a proportionate intensity reduction in the pink color of the eosin reagent. This made it difficult to discern granulocytes, particularly eosinophils, in upper endoscopy biopsies. After we had made adjustments to the Aperio scanner and ImageScope settings, we were able to modify the red, blue and green gains in the transmitted image, resulting in brighter and more color-balanced images.
Image under High Magnification
Surgical pathologists use a variety of cognitive capacities to make an accurate diagnosis, including low-power pattern recognition and high-power perception of cellular morphology. Both capacities are typically employed in a given case, but we found that the degree to which a pathologist relied on cellular morphology influenced his/her acceptance of the ×20 scanning protocol. Some needle biopsies of the prostate gland, particularly cases involving atypical small acinar proliferations, were difficult for some of our pathologists without re-scanning at ×40 as were special histochemical stains for organisms, such as an acid-fast stain. IHC stains for organisms, such as the Helicobacter pylori immunostain, were easily enough interpreted at ×20 if the organisms were plentiful, but a single organism was very difficult to identify at ×20 and without the ability to focus up and down on the image.
One limitation of our study was that the parallel review phase only involved small biopsies of the GI tract. We selected this organ system because it represents a large proportion of our routine work and has the greatest degree of familiarity to our pathologists.
The unique features of our study included the fact that the WSI system was validated for all pathologists within the department and that we ran the system in parallel for 8 months after validation. Although the six pathologists within our group have subspecialty interests, we work as general pathologists capable, with rare exceptions, of functioning interchangeably. As with most community hospital groups, each of us signs out cases from all organ systems. This is in contrast to larger academic or nonacademic departments in which there may be separate organ system sign-out sections, which may or may not lend themselves to WSI. We felt strongly that, if we were going to implement WSI for primary diagnosis, all pathologists would need to accept and be able to demonstrate proficiency with the technology. Making digital diagnoses followed by review of the corresponding glass slides provided a safety net while we were learning to optimally use the WSI technology. Additionally, we validated and implemented an image cropping procedure that had not been previously reported.
The proportion of our routine work interpreted in the parallel phase of our study accounted for only approximately 10% of our total daily specimen volume. In order for WSI to be fully implemented in high volume surgical pathology laboratories, we feel that several additional refinements need to take place. The ×20 and ×40 scan times must be reduced, simply because it takes too long to scan hundreds or thousands of glass slides. Imaging software dedicated to more easily moving the image around the computer screen (similar to the way our hands move a glass slide on a light microscope stage) must also be developed, most probably using touch screen technology or other viewing devices such as smart eyewear. The ability to focus up and down on the image through so-called "Z-stacking" technology must be refined and implemented as well. We look forward to these improvements as WSI becomes more widely accepted in the community hospital setting.
Despite these limitations, we are now ready to "go live" with digital imaging for primary diagnosis. Our health care network has two hospitals with a centralized histology service processing specimens from an active laboratory outreach program and four separate endoscopy centers, one of which is in a remote location. We plan on using the Aperio system for more efficient distribution of cases among the pathologists, who are located in various sites on any given day, and for more real-time intradepartmental case review. The occasional need for deferral to glass slides may be a barrier to full implementation; however, we anticipate these cases to be sufficiently small in number so as not to impede effective utilization of the system. We are concurrently implementing an advanced barcoding system to more effectively interface the digital imaging system with our pathology management system (CoPath 2012). Our hospital system also anticipates becoming part of a statewide integrated delivery system in the near future and this technology may be used for greater operational efficiency among pathology groups in multiple hospital sites.
| Acknowledgements|| |
The authors would like to thank Dr. Michele Conlon for her editorial support, Matthew Green and Stephanie Hooper for their technical support, and Drs. Devbala Patel, Monica Srodon and Robert Schwartz for their patience and enthusiasm during the validation process.
| References|| |
Bauer TW, Schoenfield L, Slaw RJ, Yerian L, Sun Z, Henricks WH. Validation of whole slide imaging for primary diagnosis in surgical pathology. Arch Pathol Lab Med 2013;137:518-24.
Evans AJ, Chetty R, Clarke BA, Croul S, Ghazarian DM, Kiehl TR, et al.
Primary frozen section diagnosis by robotic microscopy and virtual slide telepathology: The University Health Network experience. Hum Pathol 2009;40:1070-81.
Fallon MA, Wilbur DC, Prasad M. Ovarian frozen section diagnosis: Use of whole-slide imaging shows excellent correlation between virtual slide and original interpretations in a large series of cases. Arch Pathol Lab Med 2010;134:1020-3.
Fine JL, Grzybicki DM, Silowash R, Ho J, Gilbertson JR, Anthony L, et al.
Evaluation of whole slide image immunohistochemistry interpretation in challenging prostate needle biopsies. Hum Pathol 2008;39:564-72.
Gilbertson JR, Ho J, Anthony L, Jukic DM, Yagi Y, Parwani AV. Primary histologic diagnosis using automated whole slide imaging: A validation study. BMC Clin Pathol 2006;6:4.
Jukic DM, Drogowski LM, Martina J, Parwani AV. Clinical examination and validation of primary diagnosis in anatomic pathology using whole slide digital images. Arch Pathol Lab Med 2011;135:372-8.
Molnar B, Berczi L, Diczhazy C, Tagscherer A, Varga SV, Szende B, et al.
Digital slide and virtual microscopy based routine and telepathology evaluation of routine gastrointestinal biopsy specimens. J Clin Pathol 2003;56:433-8.
Nielsen PS, Lindebjerg J, Rasmussen J, Starklint H, Waldstrøm M, Nielsen B. Virtual microscopy: An evaluation of its validity and diagnostic performance in routine histologic diagnosis of skin tumors. Hum Pathol 2010;41:1770-6.
Pantanowitz L, Sinard JH, Henricks WH, Fatheree LA, Carter AB, Contis L, et al.
Validating whole slide imaging for diagnostic purposes in pathology: Guideline from the College of American Pathologists Pathology and Laboratory Quality Center. Arch Pathol Lab Med 2013;137:1710-22.
[Table 1], [Table 2]
|This article has been cited by|
||Enterprise Implementation of Digital Pathology: Feasibility, Challenges, and Opportunities
| ||DJ Hartman,L Pantanowitz,JS McHugh,AL Piccoli,MJ OLeary,GR Lauro |
| ||Journal of Digital Imaging. 2017; |
|[Pubmed] | [DOI]|
||The Diagnostic Concordance of Whole Slide Imaging and Light Microscopy: A Systematic Review
| ||Edward Goacher,Rebecca Randell,Bethany Williams,Darren Treanor |
| ||Archives of Pathology & Laboratory Medicine. 2017; 141(1): 151 |
|[Pubmed] | [DOI]|
||The Empirical Foundations of Telepathology: Evidence of Feasibility and Intermediate Effects
| ||Rashid L. Bashshur,Elizabeth A. Krupinski,Ronald S. Weinstein,Matthew R. Dunn,Noura Bashshur |
| ||Telemedicine and e-Health. 2017; |
|[Pubmed] | [DOI]|
||Validity of whole slide images for scoring HER2 chromogenic in situ hybridisation in breast cancer
| ||Shaimaa Al-Janabi,Anja Horstman,Henk-Jan van Slooten,Chantal Kuijpers,Clifton Lai-A-Fat,Paul J van Diest,Mehdi Jiwa |
| ||Journal of Clinical Pathology. 2016; 69(11): 992 |
|[Pubmed] | [DOI]|
||Mobile Technology for the Practice of Pathology
| ||Douglas J. Hartman |
| ||Advances In Anatomic Pathology. 2016; 23(2): 118 |
|[Pubmed] | [DOI]|
||Overview of contemporary guidelines in digital pathology: what is available in 2015 and what still needs to be addressed?
| ||M. G. Hanna,L. Pantanowitz,A. J. Evans |
| ||Journal of Clinical Pathology. 2015; 68(7): 499 |
|[Pubmed] | [DOI]|
||Whole slide imaging: uses and limitations for surgical pathology and teaching
| ||Bf Boyce |
| ||Biotechnic & Histochemistry. 2015; 90(5): 321 |
|[Pubmed] | [DOI]|