|J Pathol Inform 2013,
Diagnostic digital cytopathology: Are we ready yet?
Jarret C House1, Evita B Henderson-Jackson2, Joseph O Johnson3, Mark C Lloyd3, Jasreman Dhillon2, Nazeel Ahmad1, Ardeshir Hakam2, Walid E Khalbuss4, Marino E Leon2, David Chhieng5, Xiaohui Zhang1, Barbara A Centeno2, Marilyn M Bui6
1 Department of Pathology and Cell Biology, University of South Florida College of Medicine, Tampa, FL, USA
2 Department of Pathology and Cell Biology, University of South Florida College of Medicine; Department of Anatomic Pathology, Cytology Division, Tampa, FL, USA
3 Analytic Microscopy Core, Moffitt Cancer Center, Tampa, FL, USA
4 Cytology Division, Shadyside Hospital, University of Pittsburgh, PA, USA
5 Department of Pathology, Cytology Division, Yale University, New Haven, CT, USA
6 Department of Pathology and Cell Biology, University of South Florida College of Medicine; Analytic Microscopy Core, Moffitt Cancer Center, Tampa, FL, USA
|Date of Submission||29-Jul-2013|
|Date of Acceptance||04-Sep-2013|
|Date of Web Publication||29-Oct-2013|
Marilyn M Bui
Department of Pathology and Cell Biology, University of South Florida College of Medicine; Analytic Microscopy Core, Moffitt Cancer Center, Tampa, FL
Source of Support: None, Conflict of Interest: None
| Abstract|| |
Background: The cytology literature relating to diagnostic accuracy using whole slide imaging is scarce. We studied the diagnostic concordance between glass and digital slides among diagnosticians with different profiles to assess the readiness of adopting digital cytology in routine practice. Materials and Methods: This cohort consisted of 22 de-identified previously screened and diagnosed cases, including non-gynecological and gynecological slides using standard preparations. Glass slides were digitalized using Aperio ScanScope XT (×20 and ×40). Cytopathologists with (3) and without (3) digital experience, cytotechnologists (4) and senior pathology residents (2) diagnosed the digital slides independently first and recorded the results. Glass slides were read and recorded separately 1-3 days later. Accuracy of diagnosis, time to diagnosis and diagnostician's profile were analyzed. Results: Among 22 case pairs and four study groups, correct diagnosis (93% vs. 86%) was established using glass versus digital slides. Both methods more (>95%) accurately diagnosed positive cases than negatives. Cytopathologists with no digital experience were the most accurate in digital diagnosis, even the senior members. Cytotechnologists had the fastest diagnosis time (3 min/digital vs. 1.7 min/glass), but not the best accuracy. Digital time was 1.5 min longer than glass-slide time/per case for cytopathologists and cytotechnologists. Senior pathology residents were slower and less accurate with both methods. Cytopathologists with digital experience ranked 2 nd fastest in time, yet last in accuracy for digital slides. Conclusions: There was good overall diagnostic agreement between the digital whole-slide images and glass slides. Although glass slide diagnosis was more accurate and faster, the results of technologists and pathologists with no digital cytology experience suggest that solid diagnostic ability is a strong indicator for readiness of digital adoption.
Keywords: Digital diagnostic cytopathology, virtual microscopy, whole slide imaging
|How to cite this article:|
House JC, Henderson-Jackson EB, Johnson JO, Lloyd MC, Dhillon J, Ahmad N, Hakam A, Khalbuss WE, Leon ME, Chhieng D, Zhang X, Centeno BA, Bui MM. Diagnostic digital cytopathology: Are we ready yet?. J Pathol Inform 2013;4:28
|How to cite this URL:|
House JC, Henderson-Jackson EB, Johnson JO, Lloyd MC, Dhillon J, Ahmad N, Hakam A, Khalbuss WE, Leon ME, Chhieng D, Zhang X, Centeno BA, Bui MM. Diagnostic digital cytopathology: Are we ready yet?. J Pathol Inform [serial online] 2013 [cited 2017 Mar 26];4:28. Available from: http://www.jpathinformatics.org/text.asp?2013/4/1/28/120727
| Background|| |
As whole slide imaging (WSI) technology has developed over the past decade, its use for diagnostic surgical pathology has increased to a point that the first recommendations for validation requirements for WSI systems to be used for diagnostic purposes were published by the College of American Pathologists Pathology and Laboratory Quality Center early this year.  WSI and virtual microscopy are increasingly being utilized in cytopathology for education, proficiency testing, archiving and telecytopathology. , However, diagnostic cytopathology and cytology specimens on the other hand offer unique challenges,  particularly for direct smear preparations. The greater depth of field and high-volume of cytopathology specimens have traditionally been barriers to the adoption of WSI for routine diagnostic cytopathology.
Literature on the use of WSI in diagnostic cytological application, especially for primary diagnosis, is limited. We studied the concordance in diagnosis between glass and digital (virtual) slides among diagnosticians with different training profiles using current WSI technology to assess our readiness for future adoption of WSI for diagnostic digital cytopathology in clinical practice. Two factors were considered: The readiness of our diagnosticians and the readiness of the technology. In this pilot study, we evaluated only the diagnostic-end (accuracy and time to diagnosis) of digital cytopathology and not the entire process of transitioning a cytopathology practice from traditional glass slides to digital slides.
| Materials and Methods|| |
This retrospective study was conducted following the guidelines established by the Moffitt Cancer Center Protocol Review Board and the Institutional Review Board at the University of South Florida. This pilot study consisted of 22 de-identified, previously screened and diagnosed cytopathology cases. The final diagnosis of each case was verified by the primary investigator prior to inclusion in this study. Only the most representative one or two diagnostic slides from each case as selected by the primary investigator were included in this study. The cases represented a diverse group of cytopathology specimen types and preparations that reflected the routine cytopathology cases seen in an National Cancer Institute-designated comprehensive cancer center. Both exfoliative cytology cases and aspiration cytology cases were included in this study. Exfoliative cytology cases included body fluids, urine, sputum, cerebrospinal fluid and Pap smear More Detailss (Thin Prep and conventional). Aspiration cytology cases included pancreas, lung, thyroid, neck, lymph node and brain. The preparations included smears, cytospins, Thin Prep and one cell block. The stains included Diff-Quik, Papanicolaou (Pap), hematoxylin and eosin and a fungal stain (Gomori methenamine silver).
De-identified glass slides were digitalized using the Aperio® ScanScope® XT slide scanner (Aperio Technologies Inc., Vista, CA, USA) at the Analytic Microscopy Core. The digital images were captured at both ×20 and ×40 magnifications with resolutions of 0.5 μm/pixel and 0.25 μm/pixel, respectively, using a ×20/0.75 NA Plan Apo objective with a ×2 magnification changer. The digital images were captured in single planes without Z-stacking or multi-plane imaging. The images were viewed using Spectrum™ and Imagescope™ software (Aperio Technologies Inc., Vista, CA, USA) through a secure intranet connection. The minimum system requirements for viewing the digital slides were: CPU speed of 500 MHz (2 GHz recommended), hard drive with 100 MB of free space, 256 MB of memory (1 GB recommended), 100 megabit network card, 24-bit color video card and Windows XP Pro (SP2) or Windows Vista Pro operating system.
The diagnosticians were cytopathologists with (3) and without (3) digital cytopathology experience, cytotechnologists (4) and senior pathology residents (2). [Table 1] lists all of the case histories and diagnoses, which were categorized as either positive (malignant, epithelial cell abnormalities or suspicious for malignancy), negative (benign, reactive, infection or no evidence of malignancy) or other (insufficient or a neoplasm of uncertain malignant potential). The diagnostic categories of the 22 cases were: 10 positive, 8 negative and 4 other.
The digital slides were independently diagnosed first: The diagnosticians were provided with brief case histories; a blank diagnosis sheet to record their diagnoses, diagnosis times and comments and instructions on how to access the digital slides via a secure intranet connection. After a washout period of 3 days, the diagnosticians were provided with the glass slides, brief case histories and a new blank diagnosis sheet to record their diagnoses, diagnosis times and comments. The accuracy of diagnosis (compared with the reference diagnosis), time taken to reach a diagnosis and diagnostician's performance/experience were analyzed.
All statistical values were determined with Microsoft Excel. These values included: percent values, mean values, standard deviations and P values (two sample paired test). Twenty-two case pairs and four study groups were analyzed. These values were used to evaluate the percent of correct diagnoses and time to diagnosis data by diagnostic group for glass slides compared with digital slides.
| Results|| |
Overall Diagnostic Accuracy
As shown in [Table 2], the correct diagnostic category was identified in 93% of glass slides and 86% of digital slides. For both glass slides and digital slides, positive cases were more accurately diagnosed (>95%) than negative cases (<90% with glass slides and <80% with digital slides). The P value was 0.05.
|Table 2: Percent correct diagnosis by slide type (digital vs. glass) and diagnostic category (positive vs. negative) |
Click here to view
The diagnosis time by diagnostician group for digital versus glass slides is shown in [Table 3] and [Figure 1]. Cytotechnologists had the fastest diagnosis time for both digital and glass slides (3.0 min per digital slide vs. 1.7 min/glass slide). For cytopathologists and cytotechnologists, the diagnosis time for digital slides was an average of 1.5 min longer than for glass slides. For senior pathology residents, the diagnosis time for digital slides was nearly 2.5 min longer than for glass slides.
|Figure 1: Diagnosis time by diagnostician group for digital versus glass slides|
Click here to view
|Table 3: Diagnosis time by diagnostician group for digital versus glass slides |
Click here to view
Diagnostician Group Accuracy
The percent correct diagnosis by diagnostician group for digital versus glass slides is illustrated in [Table 4] and [Figure 2]. The cytopathologists with digital cytopathology experience ranked fourth in accuracy for digital slides and third in accuracy for glass slides. The cytopathologists without digital cytopathology experience ranked first in accuracy for digital slides and second in accuracy for glass slides. Cytotechnologists ranked second in accuracy for digital slides and first in accuracy for glass slides. Senior pathology residents ranked third in accuracy for digital slides and fourth in accuracy for glass slides.
|Figure 2: Percent correct diagnosis by diagnostician group for digital versus glass slides|
Click here to view
|Table 4: Percent correct diagnosis by diagnostician group for digital versus glass slides |
Click here to view
Less than 10% of the digital slides were considered to be of poor quality due to contributing factors like hypocellularity, the presence of air bubbles, variations in smear thickness, air-drying artifacts and "blurry" images due to suboptimal focus. We also found that when considering all digital slide readings, the image was considered as poor in only 1.7% of the positive specimen images (n = 120) versus 16.7% of the negative specimen images (n = 96). These findings were not surprising given the fact that a digital slide of less than optimal quality might hinder the ability to reliably distinguish negative from positive, whereas, for example, a strikingly positive case is less prone to misdiagnosis. In addition, the thyroid and pancreatic neoplastic mucinous cyst cases were the most diagnostically challenging for both digital and glass slides. Case 17 (benign thyroid nodule) and case 21 (neoplastic mucinous cyst favor intraductal papillary mucinous neoplasm) were the most frequently misdiagnosed cases by digital method. The diagnosticians who correctly diagnosed these two cases when using glass slides, but misdiagnosed them when using digital slides commented that the corresponding digital slides appeared "blurry" due to suboptimal focus.
| Conclusions|| |
Only a few studies have compared the accuracy and diagnosis time of WSI versus glass slides in diagnostic cytology applications. ,, Our study was the first head-to-head comparison of primary cytological diagnosis utilizing a digital versus glass cytology slide set comprised of a variety of routine cytology specimens, which included both gynecological and non-gynecological specimens obtained by exfoliative and aspiration methods. Other studies have compared the diagnostic accuracy and time to diagnosis for digital versus glass slides exclusively for liquid-based preparations of gynecological cytology specimens. , Similar to our results, these studies also found that WSI underperformed glass slides in terms of diagnostic accuracy and time to diagnosis when images were captured in single planes without focusing capability. ,
Furthermore, our study was the first head-to-head comparison of a variety of diagnosticians that included cytopathologists with digital cytopathology experience, cytopathologists without digital cytopathology experience, cytotechnologists and senior pathology residents. Other such study groups have included only cytopathologists,  cytopathologists and cytopathology trainees  and cytopathologists and cytotechnologists.  At the beginning of the study, we assumed that senior pathology residents should be more technologically savvy than senior cytopathologists (practicing for >10 years) and thus may perform better in digital pathology. To our surprise, this was not true. Our practicing cytopathologists without any digital pathology experience performed the best overall likely due to their solid diagnostic ability. Thus, it is most likely that as they become more familiar with virtual microscopy, their performance in ascertaining accurate diagnoses in routine practice will improve.
One might ask why a 100% concordance between the glass slides and the reference diagnosis was not observed in our study. One reason is that only one or two representative slides of each case were utilized in our study to facilitate the collection of data. In contrast, each of the reference diagnoses was made using the complete complement of glass slides representing all of the material collected, along with access to the complete electronic medical record. This point emphasizes that our pilot study was indeed a simulation, but not equal to a real case scenario. A multimodal approach to a case including multiple slides of various preparation, multiple observers and correlation with detailed clinical information are warranted for a better diagnosis.
A limitation of our study was the lack of z-axis depth focusing in our digital slides. Each of the digital slides in our study was composed of a whole slide image taken at a single focal plane. However, WSI systems have recently become available in the United States capable of scanning at multiple focal planes to create a virtual depth of vision known as a "z-stack."  This technology will likely further improve the diagnostic accuracy of digital cytopathology slides. Another limitation of our study was the short washout period of 3 days between review of the digital and glass slides. The level of agreement may have decreased with a longer interval between the review of the digital and glass slides. We understand that the College of American Pathologists guidelines for validating WSI suggest a washout period of at least 2 weeks;  however, our study was conducted prior to their publication. In addition, the small sample size in our study limits its statistical power and consequently small differences would not be detectable.
Overall, agreement in diagnosis between digital slides and glass slides was good (6.6% difference). Although glass slide interpretations were more accurate and took less time per case to determine the diagnosis, the results from cytotechnologists, cytopathologists and senior cytopathologists are promising. The challenges lie not in our ability, but in the current state of the technology of WSI for diagnostic cytology applications. As the access speed and quality of digital slides improve so too will the diagnostician's accuracy and speed of diagnosis.
| Acknowledgments|| |
- This project was partially funded by the Amandalee Weiss Sarcoma Fund.
- Thanks to the cytotechnologists and senior pathology residents who participated in this study: Nancy Mela, CT (ASCP); Aaron Pacholke, CT (ASCP); Karen Clanton, CT (ASCP); Jane Grieble, CT (ASCP); and Brian Quigley, MD.
- Thanks to Agnieszka Kasprzak at Analytic Microscopy Core for her assistance in scanning the slides.
| References|| |
|1.||Pantanowitz L, Sinard JH, Henricks WH, Fatheree LA, Carter AB, Contis L, et al. Validating whole slide imaging for diagnostic purposes in pathology: guideline from the college of american pathologists pathology and laboratory quality center. Arch Pathol Lab Med 2013; doi: 10.5858/arpa. 2013-0093-CP. |
|2.||Alsharif M, Carlo-Demovich J, Massey C, Madory JE, Lewin D, Medina AM, et al. Telecytopathology for immediate evaluation of fine-needle aspiration specimens. Cancer Cytopathol 2010;118:119-26. |
|3.||Kaplan KJ. Telecytopathology for immediate evaluation of fine-needle aspiration specimens. Cancer Cytopathol 2010;118:115-8. |
|4.||Wilbur DC. Digital cytology: Current state of the art and prospects for the future. Acta Cytol 2011;55:227-38. |
|5.||Evered A, Dudding N. Accuracy and perceptions of virtual microscopy compared with glass slide microscopy in cervical cytology. Cytopathology 2011;22:82-7. |
|6.||Giovagnoli MR, Giarnieri E, Carico E, Giansanti D. How do young and senior cytopathologists interact with digital cytology? Ann Ist Super Sanita 2010;46:123-9. |
|7.||Wright AM, Smith D, Dhurandhar B, Fairley T, Scheiber-Pacht M, Chakraborty S, et al. Digital slide imaging in cervicovaginal cytology: A pilot study. Arch Pathol Lab Med 2013;137:618-24. |
[Figure 1], [Figure 2]
[Table 1], [Table 2], [Table 3], [Table 4]
|This article has been cited by|
||The College of American Pathologists Guidelines for Whole Slide Imaging Validation Are Feasible for Pediatric Pathology: A Pediatric Pathology Practice Experience
| ||Michael A. Arnold,Emily Chenever,Peter B. Baker,Daniel R. Boué,Bonita Fung,Sue Hammond,Brett W. Hendrickson,Samir B. Kahwash,Christopher R. Pierson,Vinay Prasad,Kathleen K. Nicol,Thomas Barr |
| ||Pediatric and Developmental Pathology. 2015; 18(2): 109 |
|[Pubmed] | [DOI]|
||Practical issues concerning the implementation of Ki-67 proliferative index measurement in breast cancer reporting
| ||Jennet Harvey,Carla Thomas,Ben Wood,Mireille Hardie,Ben Dessauvagie,Marais Combrinck,Felicity A. Frost,Greg Sterrett |
| ||Pathology. 2015; 47(1): 13 |
|[Pubmed] | [DOI]|
||Circulating Tumor Cells: What Is in It for the Patient? A Vision towards the Future
| ||Anja van de Stolpe,Jaap den Toonder |
| ||Cancers. 2014; 6(2): 1195 |
|[Pubmed] | [DOI]|