|J Pathol Inform 2019,
Differentiating noninvasive follicular thyroid neoplasm with papillary-like nuclear features from classic papillary thyroid carcinoma: Analysis of cytomorphologic descriptions using a novel machine-learning approach
Sara Maleki1, Amin Zandvakili2, Shweta Gera3, Seema D Khutti4, Adam Gersten3, Samer N Khader3
1 Department of Pathology, Montefiore Medical Center, Albert Einstein College of Medicine, Bronx, New York; Department of Pathology, Rhode Island Hospital, Alpert Medical School of Brown University, Providence, RI, USA
2 Department of Psychiatry and Human Behavior, Alpert Medical School of Brown University, Providence, RI, USA
3 Department of Pathology, Montefiore Medical Center, Albert Einstein College of Medicine, Bronx, New York, USA
4 Department of Pathology and Laboratory Medicine, Hofstra Northwell Health School of Medicine, New York, USA
|Date of Submission||26-Apr-2019|
|Date of Acceptance||20-Jun-2019|
|Date of Web Publication||18-Sep-2019|
Rhode Island Hospital 593 Eddy Street APC-12, Providence 02903, RI
Source of Support: None, Conflict of Interest: None
| Abstract|| |
Background: Recent studies show various cytomorphologic features that can assist in the differentiation of classic papillary thyroid carcinoma (cPTC) from noninvasive follicular thyroid neoplasm with papillary-like nuclear features (NIFTP). Differentiating these two entities changes the clinical management significantly. We evaluated the performance of support vector machine (SVM), a machine learning algorithm, in differentiating cases of NIFTP and encapsulated follicular variant of papillary thyroid carcinoma with no capsular or lymphovascular invasion (EFVPTC) from cases of cPTC with the use of microscopic descriptions. SVM is a supervised learning algorithm used in classification problems. It assigns the input data to one of two categories by building a model based on a set of training examples (learning) and then using that learned model to classify new examples. Methods: Surgical pathology cases with the diagnosis of cPTC, NIFTP, and EFVPTC, were obtained from the laboratory information system. Only cases with existing fine-needle aspiration matching the tumor and available microscopic description were included. NIFTP cases with ipsilateral micro-PTC were excluded. The final cohort consisted of 59 cases (29 cPTCs and 30 NIFTP/EFVPTCs). Results: SVM successfully differentiated cPTC from NIFTP/EFVPTC 76.05 ± 0.96% of times (above chance,P < 0.05) with the sensitivity of 72.6% and specificity of 81.6% in detecting cPTC. Conclusions: This machine learning algorithm was successful in distinguishing NIFTP/EFVPTC from cPTC. Our results are compatible with the prior studies, which show cytologic features are helpful in differentiating these two entities. Furthermore, this study shows the power and potential of this approach for clinical use and in developing data-driven scoring systems, which can guide cytopathology and surgical pathology diagnosis.
Keywords: Cytology, machine-learning, noninvasive follicular thyroid neoplasm with papillary-like nuclear features, papillary thyroid carcinoma, support vector machine
|How to cite this article:|
Maleki S, Zandvakili A, Gera S, Khutti SD, Gersten A, Khader SN. Differentiating noninvasive follicular thyroid neoplasm with papillary-like nuclear features from classic papillary thyroid carcinoma: Analysis of cytomorphologic descriptions using a novel machine-learning approach. J Pathol Inform 2019;10:29
|How to cite this URL:|
Maleki S, Zandvakili A, Gera S, Khutti SD, Gersten A, Khader SN. Differentiating noninvasive follicular thyroid neoplasm with papillary-like nuclear features from classic papillary thyroid carcinoma: Analysis of cytomorphologic descriptions using a novel machine-learning approach. J Pathol Inform [serial online] 2019 [cited 2020 Apr 2];10:29. Available from: http://www.jpathinformatics.org/text.asp?2019/10/1/29/267097
| Introduction|| |
Papillary thyroid carcinoma (PTC) constitutes approximately 84% of all thyroid malignancies. There has been a substantial increase in the incidence of PTC, in part due to early detection of mainly microcarcinomas and also more relaxed histologic criteria for detection of follicular variant of papillary thyroid carcinoma (FVPTC). Histologic diagnosis of FVPTC is based on detection of nuclear features of PTC, which are frequently less evident than classic PTC (cPTC). Papillary structures and psammomatous calcifications are absent, and nuclear pseudoinclusions are rare. Until recently, the encapsulated FVPTC without invasion (EFVPTC) represented up to 20% of all PTCs. Encapsulated or partially encapsulated/well-circumscribed FVPTCs with no associated capsular penetration or lymphovascular invasion have virtually no metastatic potential or risk of recurrence. Moreover, although there is some variability in results, in most cases, FVPTC harbors RAS mutations and PAX8/PPARG rearrangements and lacks BRAF mutations., This cancer has recently been reclassified as a neoplasm of low malignant potential and is now termed “noninvasive follicular thyroid neoplasm with papillary-like nuclear features” (NIFTP). This change will result in a corresponding reduction in the number of thyroid malignancies. Before the introduction of NIFTP, many of these patients were treated with total thyroidectomy followed by radioactive iodine therapy. Now, these patients are spared the anxiety of cancer diagnosis, and they will be managed more conservatively, requiring only lobectomy without the need for radioactive iodine therapy.
Fine-needle aspiration (FNA) is the most essential and reliable procedure for preoperative evaluation of thyroid nodules. The cytopathologists follow the Bethesda system for reporting thyroid cytopathology (TBS). One important objective of TBS is to link each diagnostic category with a risk of malignancy (ROM) and an evidence-based clinical management guideline. NIFTP and tumors historically classified as FVPTC do not typically exhibit the full range of cytologic features associated with cPTC and as a result have tended to be classified in the indeterminate categories of TBS (i.e., atypia of undetermined significance/follicular lesion of undetermined significance [category III, AUS/FLUS], suspicious for a follicular neoplasm/follicular neoplasm [category IV, SFN/FN], and suspicious for malignancy [category V]). The two studies done by Strickland et al. and Faquin et al.,, showed a significant decrease in the ROM in the indeterminate categories of TBS after this new classification with very little change in non-diagnostic, benign, and positive for malignancy categories (categories I, II, and VI). Therefore, this reclassification will change the ROM associated with some of TBS diagnostic categories and to triage patients with NIFTP to more conservative clinical management, cytopathologists need to distinguish NIFTP/FVPTC from cPTC. Recent studies show various cytomorphological features that can assist in the cytologic differentiation of cPTC from EFVPTC (previously) and NIFTP. Howitt et al. examined a cohort of 39 cases including 11 EFVPTCs and 28 cPTCs. Cases of cPTC were significantly more frequently associated with a predominance of tumor sheets, papillae, and nuclear pseudoinclusions compared with EFVPTCs. Maletta et al. also showed that although the most common diagnosis that is given in the cases of NIFTP is “follicular neoplasm,” a number of these cases do have nuclear features of PTC and these nuclear features are significantly different from those of benign follicular tumors and hyperplastic nodules, but not from those of invasive FVPTC. Therefore, the presence of PTC nuclear features in a follicular patterned nodule may indicate the possibility of both FVPTC and NIFTP. Strickland et al. also reinforced the results of previous studies by indicating that simple cytologic and architectural features (i.e., papillae, nuclear pseudoinclusions, psammomatous calcifications, and microfollicular architecture) can be used to distinguish the majority of NIFTPs and other follicular-patterned lesions from cPTCs prospectively.
In recent years, attempts have been made to use machine learning algorithms in digital pathology for computer-assisted diagnosis. Machine learning techniques often used in digital pathology image analysis include supervised learning (e.g., support vector machine [SVM], random forest, and convolutional neural networks) and unsupervised learning (e.g., k-means, autoencoders, and principal component analysis). However, there have been few attempts on using machine learning algorithms on pathology reports. The benefit of such an algorithm is to help the pathologists with decision-making in challenging cases and as an aid to pattern recognition software for computer-assisted diagnosis., Herein, we took an alternative approach and evaluated the performance of SVM, a machine learning algorithm, in differentiating NIFTP from cPTC by analysis of the microscopic description part of cytopathology reports. SVM is a supervised learning algorithm that is used in classification problems. It assigns the input data to one of two categories by building a model based on a set of training examples (learning) and then using that learned model to classify new examples.
| Methods|| |
The study was approved by our institution's institutional review board. The dataset comprised surgical pathology cases with the diagnosis of cPTC, NIFTP, or EFVPTC on the final pathology report, 1 cm in size or larger, obtained from the laboratory information system. Only cases with existing FNA matching the tumor and available microscopic description were included. NIFTP cases with ipsilateral micro-PTC were excluded. For cases with multiple nodules, clinical history, radiologic findings, and surgical pathology report were reviewed to ensure the FNA correlated with the correct nodule. The final cohort consisted of 59 cases (29 cPTCs and 30 NIFTP/EFVPTCs) resected in the time frame of 2010–2016. A total of nine different cytopathologists had diagnosed the FNAs in the department of pathology in that period.
Data processing and machine learning algorithms
The data processing was done using Matlab Text Analytics Toolbox. We preprocessed the cytomorphological descriptions, by making all text lowercase, removing common words, such as “the, a, at, to” and reduce words into their root forms by trimming their endings. We also removed language that pertained to the differential diagnosis and only focused on the description.
To digitize and convert the microscopic descriptions to a format usable by the machine learning algorithm, the preprocessed microscopic descriptions were reviewed, and 59 cytomorphologic keywords/phrases that were used by the pathologists were identified. The synonymous phrases were then grouped into 32 categories [Table 1]. The presence or absence of each category in each case was recorded by values of 1 or 0 respectively in arrays 32 bins in length.
|Table 1: Keywords or phrases found commonly in the microscopic descriptions|
Click here to view
We used a supervised machine learning algorithm, (SVM, specifically an L1 regularized SVM implementation through LIBLINEAR), to train a linear classifier on the digitized microscopic descriptions, assigning them as NIFTP/EFVPTC or cPTC.
Linear SVM has one modifying parameter, C (i.e., misclassification cost and error penalty). When a small misclassification cost is used, the algorithm can find more global and generalizable trends in the data but is prone to making errors. A higher misclassification cost is associated with more specific solutions, a lower error rate, but less generalizability. The goal is a misclassification cost best balancing error and generalizability. We thus explored misclassifications (C) values ranging from 0.1 to 100 (log scale) to find the C with the best balance between error and generalizability. In addition, we used a regularized version of SVM here. This means that we used a penalty for model complexity and forced the SVM algorithm to use the fewest number of keywords in making the prediction. Limiting complexity would make model interpretation easier and also increases its generalizability.
The classifier was trained on a training set comprising all cases except for one random case of NIFTP/EFVPTC and one random case of cPTC. The trained model's ability to classify (performance) was then assessed on predicting the left-out cases (i.e., leave-2-out cross-validation). We repeated this process for all possible iterations of leave-2-out. As a post hoc analysis, we interrogated the classifier post training to identify the keywords that were used to assign cases to NIFTP/EFVPTC versus cPTC.
| Results|| |
The demographic data are shown in [Table 2]. The Bethesda categories ranged from II to V in the NIFTP/EFVPTC cases with 66% being Bethesda categories III and IV. The Bethesda categories ranged from III to VI in the cPTC cases with category VI being the most common (59%) [Table 3]. The average length of the microscopic description was 22.17 ± 9.38 words for NIFTP/EFVPTC and 26.31 ± 10.60 words for cPTC. Word counts did not differ significantly between NIFTP/EFVPTC and cPTC microscopic descriptions (Student's t-test, P= 0.12). Frequency of various cytomorphologic features as mentioned in the microscopic description part of the cytology reports are shown in [Table 4]. Microfollicles were more commonly described in the NIFTP/EFVPTC cases, and sheets were more commonly described in the cPTC cases. Nuclear grooves, pseudoinclusions, and papillary structures were more commonly described in the cPTC cases compared to the cases of NIFTP/EFVPTC.
|Table 2: Age, gender, tumor location, and tumor size in patients with noninvasive follicular thyroid neoplasm with papillary-like nuclear features/encapsulated follicular variant of papillary thyroid carcinoma with no capsular or lymphovascular invasion and classic papillary thyroid carcinoma|
Click here to view
|Table 3: The frequency of the Bethesda categories in cases of noninvasive follicular thyroid neoplasm with papillary-like nuclear features/encapsulated follicular variant of papillary thyroid carcinoma with no capsular or lymphovascular invasion and classic papillary thyroid carcinoma (%)|
Click here to view
|Table 4: The frequency of various cytomorphologic features noted in the microscopic descriptions of noninvasive follicular thyroid neoplasm with papillary-like nuclear features/encapsulated follicular variant of papillary thyroid carcinoma with no capsular or lymphovascular invasion and classic papillary thyroid carcinoma cases (%)|
Click here to view
The SVM-trained classifier correctly identified cPTC from NIFTP/EFVPTC in 76.05 ± 0.96% of cases in a cross-validated sample (predictions were done on test set that the model was not trained on). This performance was above chance (>50%, P < 0.001). For detection of cPTC, the sensitivity of prediction was 72.6% and the specificity was 81.6%. Area under the receiver operating characteristic curve was 0.79 (95% confidence interval 0.77–0.81) [Figure 1].
|Figure 1: Receiver operating characteristic curve, representing the support vector machine algorithm's performance in distinguishing classic papillary thyroid carcinoma from noninvasive follicular thyroid neoplasm with papillary-like nuclear features cases (cross-validated). The shaded area represents 95% confidence interval calculated through bootstrapping|
Click here to view
We then conducted a post hoc analysis, assessing the strategy that was used by the machine learning algorithm to make the decision. Through its training, SVM assigns a weight to each of the phrases in the text: a positive weight means that the occurrence of that word is associated with cPTC diagnosis, and a negative weight implies that the occurrence of that phrase is associated with NIFTP/EFVPTC diagnosis. [Figure 2] shows the categories with a nonzero weight that were used by the classifier to differentiate NIFTP/EFVPTC from cPTC.
|Figure 2: The plot represents the (normalized to the largest) weights that are used for the classification. Only categories that had a non-zero weight were plotted. The weights with a negative value were suggestive of noninvasive follicular thyroid neoplasm with papillary-like nuclear features, and the weights with a positive value were indicative of classic papillary thyroid carcinoma. Greater weights were more strongly associated with a noninvasive follicular thyroid neoplasm with papillary-like nuclear features versus classic papillary thyroid carcinoma diagnosis|
Click here to view
| Conclusions|| |
Reclassification of EFVPTC as a neoplasm of low malignant potential (NIFTP) will result in a smaller number of diagnosed thyroid cancer cases and more conservative management of these patients. Therefore, preoperative differentiation of cPTC from FVPTC is of utmost importance. Cytopathologic evaluation according to TBS is the most reliable diagnostic procedure used in the preoperative assessment of thyroid nodules. Due to subtle nuclear features of FVPTC, these tumors are usually classified in the indeterminate categories of TBS, and as described by Strickland et al. and Faquin et al.,, this reclassification will change the ROM associated with these diagnostic categories. Various cytomorphological features such as the predominance of tumor sheets versus microfollicular architecture, papillae, nuclear pseudoinclusions, and psammomatous calcifications ,, can assist in the cytologic differentiation of cPTC from FVPTC and NIFTP.
In this study, we evaluated the performance of SVM, a supervised machine learning algorithm, in differentiating NIFTP/EFVPTC from cPTC by analyzing the microscopic description part of cytopathology reports using a cohort of cPTCs and NIFTP/EFVPTCs with a corresponding FNA. We demonstrated that a machine learning algorithm was able to successfully distinguish microscopic descriptions of NIFTP/EFVPTC and cPTC FNAs in 76.05% ± 0.96% of cases with high sensitivity and specificity.
The significant aspect of this study was applying predictive analytics and machine learning to analyze cytomorphologic descriptions, written by practicing cytopathologists. In distinguishing cPTC versus NIFTP/EFVPTC, the model relied on keywords that were not unexpected, but the analysis provided relative weights (i.e., importance) for each of these keywords. Machine learning algorithms can be used as a decision-making tool in challenging cases or as an aid to pattern recognition software for computer-assisted diagnosis. Although there is a lot of interest in applying machine learning algorithms in digital pathology, most studies have focused on machine vision and image analysis. We, however, employed semantic and text-based analysis methods. Semantic methods cannot be deployed as an independent tool in making diagnostic decisions, but they can help provide a better understanding of the intuitions and the tacit approaches that are used by pathologists when making diagnoses. These intuitions then can turn into tangible criteria, guiding others. They can also be used in providing feedback and confirmation when diagnoses are made. Although the diagnosis of NIFTP/EFVPTC is not a cytological diagnosis and relies on tumor excision to exclude capsular and vascular invasion and the presence of papillary structures, it will be helpful to cytologically distinguish NIFTP/EFVPTC from cPTC as these patients require a more conservative surgery (hemithyroidectomy versus total thyroidectomy). The model introduced here was able to correctly categorize NIFTP/EFVPTC and cPTC in three-quarter of the cases. Similar to many other medical diagnostic tools, this approach has predictive value, but the predictive power is not absolute. However, it can help better stratify the risk of cancer and can be used in making clinical decisions by providers and patients.
This work is based on modest sample size, collected in one center. This raises the possibility that the performance we found could represent overfitting, which means that the algorithm learns trends in the data not generalizable to other samples. We addressed this by (1) using regularization (limiting model complexity), and (2) using cross-validation by training and testing the model on two separate datasets. Replication with larger samples, however, would be required to bolster the strength of our observation. Despite these methodological limitations, we found validated and clinically meaningful results. In conclusion, this study shows the power and potential of this method for clinical use. Ideally, this approach can be used to develop data-driven scoring systems, which can act as a guide for cytopathology and surgical pathology diagnosis.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Fagin JA, Wells SA Jr. Biologic and clinical perspectives on thyroid cancer. N Engl J Med 2016;375:1054-67.
Lloyd RV, Osamura RY, Klöppel G, Rosai J. WHO classification of tumours of endocrine organs. Lyon, France: International Agency for Research on Cancer (IARC), 2017.
Krane JF, Alexander EK, Cibas ES, Barletta JA. Coming to terms with NIFTP: A provisional approach for cytologists. Cancer Cytopathol 2016;124:767-72.
Paulson VA, Shivdasani P, Angell TE, Cibas ES, Krane JF, Lindeman NI, et al.
Noninvasive follicular thyroid neoplasm with papillary-like nuclear features accounts for more than half of “Carcinomas” harboring RAS mutations. Thyroid 2017;27:506-11.
Thompson LD. Ninety-four cases of encapsulated follicular variant of papillary thyroid carcinoma: A name change to noninvasive follicular thyroid neoplasm with papillary-like nuclear features would help prevent overtreatment. Mod Pathol 2016;29:698-707.
Nikiforov YE, Seethala RR, Tallini G, Baloch ZW, Basolo F, Thompson LD, et al.
Nomenclature revision for encapsulated follicular variant of papillary thyroid carcinoma: A paradigm shift to reduce overtreatment of indolent tumors. JAMA Oncol 2016;2:1023-9.
Haugen BR, Sawka AM, Alexander EK, Bible KC, Caturegli P, Doherty GM, et al.
American Thyroid Association Guidelines on the Management of Thyroid Nodules and Differentiated Thyroid Cancer Task Force review and recommendation on the proposed renaming of encapsulated follicular variant papillary thyroid carcinoma without invasion to noninvasive follicular thyroid neoplasm with papillary-like nuclear features. Thyroid 2017;27:481-3.
Ali SZ, Cibas ES. The Bethesda system for reporting thyroid cytopathology: Definitions, criteria, and explanatory notes. Cham, Switzerland: Springer, 2018.
Baloch ZW, Seethala RR, Faquin WC, Papotti MG, Basolo F, Fadda G, et al.
Noninvasive follicular thyroid neoplasm with papillary-like nuclear features (NIFTP): A changing paradigm in thyroid surgical pathology and implications for thyroid cytopathology. Cancer Cytopathol 2016;124:616-20.
Strickland KC, Howitt BE, Marqusee E, Alexander EK, Cibas ES, Krane JF, et al.
The impact of noninvasive follicular variant of papillary thyroid carcinoma on rates of malignancy for fine-needle aspiration diagnostic categories. Thyroid 2015;25:987-92.
Faquin WC, Wong LQ, Afrogheh AH, Ali SZ, Bishop JA, Bongiovanni M, et al.
Impact of reclassifying noninvasive follicular variant of papillary thyroid carcinoma on the risk of malignancy in the Bethesda system for reporting thyroid cytopathology. Cancer Cytopathol 2016;124:181-7.
Ohori NP, Wolfe J, Carty SE, Yip L, LeBeau SO, Berg AN, et al.
The influence of the noninvasive follicular thyroid neoplasm with papillary-like nuclear features (NIFTP) resection diagnosis on the false-positive thyroid cytology rate relates to quality assurance thresholds and the application of NIFTP criteria. Cancer Cytopathol 2017;125:692-700.
Howitt BE, Chang S, Eszlinger M, Paschke R, Drage MG, Krane JF, et al.
Fine-needle aspiration diagnoses of noninvasive follicular variant of papillary thyroid carcinoma. Am J Clin Pathol 2015;144:850-7.
Maletta F, Massa F, Torregrossa L, Duregon E, Casadei GP, Basolo F, et al.
Cytological features of “noninvasive follicular thyroid neoplasm with papillary-like nuclear features” and their correlation with tumor histology. Hum Pathol 2016;54:134-42.
Strickland KC, Vivero M, Jo VY, Lowe AC, Hollowell M, Qian X, et al.
Preoperative cytologic diagnosis of noninvasive follicular thyroid neoplasm with papillary-like nuclear features: A prospective analysis. Thyroid 2016;26:1466-71.
Komura D, Ishikawa S. Machine learning methods for histopathological image analysis. Comput Struct Biotechnol J 2018;16:34-42.
Whalen PE, Muralidharan AT, Kiddy JR, Duncan WD. Enhancing semantic analysis of pathology reports. In: Proceedings of the 9th
International Conference on Biological Ontology. Corvallis, Oregon, USA: International Conference on Biological Ontology; 2018.
Vega-Barbas M, Seoane F. A different approach for digital pathology: Lexicon-semantic analysis of histopathological reports for the assessment of their quality. Conf Proc IEEE Eng Med Biol Soc 2018;2018:4054-7.
Cortes C, Vapnik V. Support-vector networks. Mach Learn 1995;20:273-97.
Fan RE, Chang KW, Hsieh CJ, Wang XR, Lin CJ. LIBLINEAR: A library for large linear classification. J Mach Learn Res 2008;9:1871-4.
[Figure 1], [Figure 2]
[Table 1], [Table 2], [Table 3], [Table 4]