Journal of Pathology Informatics

RESEARCH ARTICLE
Year
: 2015  |  Volume : 6  |  Issue : 1  |  Page : 38-

Validation of natural language processing to extract breast cancer pathology procedures and results


Arika E Wieneke, Erin J. A. Bowles, David Cronkite, Karen J Wernli, Hongyuan Gao, David Carrell, Diana S. M. Buist 
 Group Health Research Institute, Seattle, WA, USA

Correspondence Address:
Erin J. A. Bowles
Group Health Research Institute, Seattle, WA
USA

Background: Pathology reports typically require manual review to abstract research data. We developed a natural language processing (NLP) system to automatically interpret free-text breast pathology reports with limited assistance from manual abstraction. Methods: We used an iterative approach of machine learning algorithms and constructed groups of related findings to identify breast-related procedures and results from free-text pathology reports. We evaluated the NLP system using an all-or-nothing approach to determine which reports could be processed entirely using NLP and which reports needed manual review beyond NLP. We divided 3234 reports for development (2910, 90%), and evaluation (324, 10%) purposes using manually reviewed pathology data as our gold standard. Results: NLP correctly coded 12.7% of the evaluation set, flagged 49.1% of reports for manual review, incorrectly coded 30.8%, and correctly omitted 7.4% from the evaluation set due to irrelevancy (i.e. not breast-related). Common procedures and results were identified correctly (e.g. invasive ductal with 95.5% precision and 94.0% sensitivity), but entire reports were flagged for manual review because of rare findings and substantial variation in pathology report text. Conclusions: The NLP system we developed did not perform sufficiently for abstracting entire breast pathology reports. The all-or-nothing approach resulted in too broad of a scope of work and limited our flexibility to identify breast pathology procedures and results. Our NLP system was also limited by the lack of the gold standard data on rare findings and wide variation in pathology text. Focusing on individual, common elements and improving pathology text report standardization may improve performance.


How to cite this article:
Wieneke AE, Bowles EJ, Cronkite D, Wernli KJ, Gao H, Carrell D, Buist DS. Validation of natural language processing to extract breast cancer pathology procedures and results.J Pathol Inform 2015;6:38-38


How to cite this URL:
Wieneke AE, Bowles EJ, Cronkite D, Wernli KJ, Gao H, Carrell D, Buist DS. Validation of natural language processing to extract breast cancer pathology procedures and results. J Pathol Inform [serial online] 2015 [cited 2020 Aug 13 ];6:38-38
Available from: http://www.jpathinformatics.org/article.asp?issn=2153-3539;year=2015;volume=6;issue=1;spage=38;epage=38;aulast=Wieneke;type=0