Journal of Pathology Informatics

: 2020  |  Volume : 11  |  Issue : 1  |  Page : 35-

Computerized image analysis of tumor cell nuclear morphology can improve patient selection for clinical trials in localized clear cell renal cell carcinoma

In Hwa Um1, Lindesay Scott-Hayward2, Monique Mackenzie2, Puay Hoon Tan3, Ravindran Kanesvaran4, Yukti Choudhury5, Peter D Caie1, Min-Han Tan6, Marie O’Donnell7, Steve Leung8, Grant D Stewart9, David J Harrison10,  
1 School of Medicine, University of St Andrews, St Andrews, Scotland
2 School of Mathematics and Statistics, University of St Andrews, St Andrews, Scotland
3 Department of Pathology, Singapore General Hospital, Singapore
4 Division of Medical Oncology, National Cancer Centre, Singapore
5 Institute of Bioengineering and Nanotechnology, Singapore
6 Lucence Diagnostics Pte Ltd, Singapore
7 Department of Pathology, Western General Hospital, Edinburgh, Scotland
8 Department of Urology, Western General Hospital, Edinburgh, Scotland
9 Department of Surgery, University of Cambridge, Cambridge, England
10 School of Medicine, University of St Andrews and Lothian NHS University Hospitals, St Andrews, Scotland

Correspondence Address:
Dr. In Hwa Um
School of Medicine, University of St Andrews, St Andrews


Background: Clinicopathological scores are used to predict the likelihood of recurrence-free survival for patients with clear cell renal cell carcinoma (ccRCC) after surgery. These are fallible, particularly in the middle range. This inevitably means that a significant proportion of ccRCC patients who will not develop recurrent disease enroll into clinical trials. As an exemplar of using digital pathology, we sought to improve the predictive power of “recurrence free” designation in localized ccRCC patients, by precise measurement of ccRCC nuclear morphological features using computational image analysis, thereby replacing manual nuclear grade assessment. Materials and Methods: TNM 8 UICC pathological stage pT1-pT3 ccRCC cases were recruited in Scotland and in Singapore. A Leibovich score (LS) was calculated. Definiens Tissue studio® (Definiens GmbH, Munich) image analysis platform was used to measure tumor nuclear morphological features in digitized hematoxylin and eosin (H&E) images. Results: Replacing human-defined nuclear grade with computer-defined mean perimeter generated a modified Leibovich algorithm, improved overall specificity 0.86 from 0.76 in the training cohort. The greatest increase in specificity was seen in LS 5 and 6, which went from 0 to 0.57 and 0.40, respectively. The modified Leibovich algorithm increased the specificity from 0.84 to 0.94 in the validation cohort. Conclusions: CcRCC nuclear mean perimeter, measured by computational image analysis, together with tumor stage and size, node status and necrosis improved the accuracy of predicting recurrence-free in the localized ccRCC patients. This finding was validated in an ethnically different Singaporean cohort, despite the different H and E staining protocol and scanner used. This may be a useful patient selection tool for recruitment to multicenter studies, preventing some patients from receiving unnecessary additional treatment while reducing the number of patients required to achieve adequate power within neoadjuvant and adjuvant clinical studies.

How to cite this article:
Um IH, Scott-Hayward L, Mackenzie M, Tan PH, Kanesvaran R, Choudhury Y, Caie PD, Tan MH, O’Donnell M, Leung S, Stewart GD, Harrison DJ. Computerized image analysis of tumor cell nuclear morphology can improve patient selection for clinical trials in localized clear cell renal cell carcinoma.J Pathol Inform 2020;11:35-35

How to cite this URL:
Um IH, Scott-Hayward L, Mackenzie M, Tan PH, Kanesvaran R, Choudhury Y, Caie PD, Tan MH, O’Donnell M, Leung S, Stewart GD, Harrison DJ. Computerized image analysis of tumor cell nuclear morphology can improve patient selection for clinical trials in localized clear cell renal cell carcinoma. J Pathol Inform [serial online] 2020 [cited 2020 Nov 26 ];11:35-35
Available from:

Full Text


In recent decades, the incidence of renal cell carcinoma (RCC) has increased almost thirty-fold and a more dramatic increase is expected by 2030, related to obesity and hypertension, but also partly due to the increased use of medical imaging systems such as computed tomography, ultrasound scan, and magnetic resonance imaging for unrelated conditions.[1],[2],[3],[4] A significant part of this increase may be the result of more use of imaging which detects earlier, pre-symptomatic renal masses, whose prognosis may be uncertain.[4] Around 70% of newly diagnosed RCC cases are discovered when initially localized to the kidney, while the remaining RCC cases are locally advanced or have primary metastatic disease.[3] Renal masses may be removed by nephrectomy, but increasingly small masses may be removed by nephron-sparing partial nephrectomy.[5] Ablative therapy and active surveillance are also options, particularly for small masses detected incidentally and these can also benefit elderly patients who are too infirm to undergo surgery.[3],[6],[7] However, approximately 30%–40% of localized RCC patients (stage I to III) experience disease recurrence within 5 years after nephrectomy.[8],[9],[10],[11] This establishes that surgical intervention is not sufficient for treating all localized RCC patients. Improved management may include neoadjuvant or adjuvant therapy, although no such treatments have yet been proven to be efficacious.[12],[13]

A number of different integrated staging systems such as Mayo clinic stage, size, grade, and necrosis (SSIGN) scoring algorithm, University of California-Los Angeles integrated staging system (UISS), and Leibovich score (LS) have been developed and utilized to improve the prognosis of RCC patients.[9],[14],[15] In particular, they have been utilized as an enrolment criterion. For example, Phase III, SORCE trial (NCT00492258), scrutinizing the benefit of 1 or 3 years of sorafenib in the intermediate- and high-risk group of RCC patients, calculated the risk of recurrence using LS, whereas Phase III, S-TRACT (NCT00375674) trial which investigated sunitinib's effect as an adjuvant treatment in high risk group utilized a modified-UISS score.[16] It is important to accurately stratify the high or low risk of recurrence after surgical management. There is a 3-25% chance of the low and intermediate recurring: these patients may not have had potentially beneficial adjuvant therapy. By contrast 30% of the high risk group may never recur and therefore may have received unnecessary treatment.[9],[16] Therefore, it is clinically important to develop a more precise stratification tool predicting recurrence in nonmetastatic clear cell RCC (ccRCC). This will require different parameters, as current prognostic and predictive models based entirely on standard clinical and pathological factors have a limited potential for improvement.

In order to investigate the utility of objective pathological assessment using centralized, digital pathology, with image analysis and machine learning, we investigated the contribution of nuclear grade to LS.

Subjective nuclear or nucleolar grading contributes to integrated staging systems such as SSIGN, UISS, and LS as a part of prognostic tool, but there are drawbacks.[17],[18],[19],[20],[21]

Interobserver variation is also a significant issue.[22] There may be heterogeneity in nuclear size, shape, and nucleolar features in different parts of the tumor, which results in lack of concordance between pathologists.[18],[22],[23],[24] Computer-aided image analysis offers more consistency as well as the ability to quantify heterogeneity.[25],[26],[27]

Automated whole-slide image (WSI) acquisition system has developed rapidly in terms of the resolution, speed, affordability, and functional versatility.[28] In addition, many computational image analysis platforms have enabled pathologists to obtain more biological information by transforming pixels to numbers, which can measure a number of various intriguing features such as size, shape, and intensity within not only tumor cells but also in the tumor microenvironment.[25],[28],[29],[30],[31]

Therefore, we sought to explore the ability to predict recurrence-free survival in localized ccRCC by modifying original LS with more precise and accurate measurement of ccRCC nuclear morphological features. This was achieved by utilizing a binomial regression framework with spatially adaptive smoothing to evaluate the usefulness of the morphological features in predicting disease recurrence.

 Materials and Methods

Training set patient cohort

A consecutive cohort of 120 ccRCC patients with TNM8 UICC pathological stage pT1–3 disease, having extirpative surgery between 2007 and 2010, were identified from the Scottish Collaboration on Translational Research into Renal Cell Carcinoma (SCOTRRCC) study (South East Scotland Research Ethics Committee 02: 10/S1102/68).[32] Approval for tissue and clinical data usage was achieved by NHS Lothian SCOTRRCC tissue bank (approval number-RR011, RR012, and TGU-LAB-504). Patient clinicopathological characteristics are shown in [Table 1].{Table 1}

Validation set patient cohort

A consecutive cohort of 217 ccRCC patients with TNM8 UICC pathological stage pT1-3 disease, having extirpative surgery between 2000 and 2012, were selected for the validation set from the tissue bank of Singaporean cancer research [Table 2].{Table 2}

Representative sample collection and image acquisition

Archived hematoxylin and eosin (H&E)-stained slides from the training set were reviewed by authors (MO'D) and a slide with adequate ccRCC was selected (authors DJH, IHU). The representative tumor blocks were selected on the basis of having sufficient tumor present, thus excluding small blocks for resection margins. No formal record of heterogeneity of manual grading was made. The assessment mirrored clinical practice. Both the training and validation cohorts were retrospective cases between 2000 and 2012 to ensure that adequate follow-up was available. Fuhrman nuclear grade had been utilized in these cases which preceded the update to ISUP (International Society of Urological pathology) nucleolar grade in 2013.[24],[33] We found that image analysis on H and E was not efficient at classifying nucleoli, and so, for the purposes of this study, Fuhrman grade was used to contribute to LS. The SCOTRRCC cohort WSIs were acquired in Zeiss Axio Z1 scanner with objective 20x magnification. H and E WSIs from the validation set were digitized using Leica SCN scanner at 20x magnification, carried out in Singapore. In this way, we sought to explore how interoperable any algorithm would be when using images obtained from different scanners.

Image analysis-ccRCC nuclear morphology analysis

CZI and SVS formatted H and E digitized images were imported into Definiens Tissue studio® (Definiens GmbH, Munich, Germany), then ccRCC tumor area was chosen as a region of interest, followed by classification of ccRCC cells from non-ccRCC such as stromal cells, immune cells, necrotic cells, and hemorrhage area via their distinct shapes and sizes. ccRCC tumor nuclei were then segmented by hematoxylin intensity on the individual level. In this step immune cells were excluded on the basis of size, being smaller than tumour cell nuclei. Fibroblasts and other stromal cells were automatically excluded on the basis of length to width ratio being greater than 2:1. Then, the pixels of ccRCC tumor nuclear morphometric features, such as area (μm 2), roundness, shape index, length, width, the ratio of length to width, elliptic fit, circularity, ellipticity, and perimeter, were calculated to a numeric dataset [Figure 1].[34]{Figure 1}

Data collection

The primary end clinical point was disease recurrence-free status. If the patient had a disease recurrence by imaging or a biopsy either locally or remotely at the time of data collection, it was deemed to be a recurrence; otherwise, the patient was censored as non-recurrence. LSs were recalculated as a part of quality control in data collection on the basis of the pathology reports, related to tumor, node, metastasis (TNM) staging, tumor size, pathologist-assessed nuclear grade, and necrosis [Table 3].[9],[24],[33],[35] The image analysis data from individual nuclei were averaged over the whole section and standard deviation was used to record variance in feature measurement [Figure 2] as the data were distributed as a continuous variable. Both mean and standard deviation of the ten different nuclear morphological features detailed above were regarded as individual attributes for statistical analysis.{Table 3}{Figure 2}

Statistical analysis

Statistical models were fitted to the dataset from the Scottish training cohort using a binomial generalized linear model framework. The response variable was a binary indicator of recurrence (1) or non-recurrence (0). Two initial models were constructed, (i) using LS only and (ii) with a partial LS that excluded nuclear grade. The first model acted as a comparative tool for the second one. In addition to the partial LS, model (ii) allowed for the inclusion of the nuclear morphological features as replacement for the excluded nuclear grade. The features were added in turn to the partial LS model to formulate a modified Leibovich model.

Model selection was done using forward selection with the addition of a covariate only permitted if it was not highly correlated with one already chosen. Colinearity was determined by a variance inflation factor score of < 5. At each stage, the flexibility of each covariate was assessed for being a smooth or linear term. Smooth terms were fitted using quadratic B-splines with flexibility permitted between 1 (linear) and 8 degrees of freedom. Flexibility was chosen using a spatially adaptive local smoothing algorithm (SALSA).[36]

Model fit and covariate flexibility were determined using 5-fold cross-validation (CV) with a partial area under the curve (AUCp) score as the cost function. The AUCp score was calculated as the area under the receiver operating characteristic (ROC) where specificity (true negativity – prediction of “No disease recurrence”) was between 0.8 and 1. The larger the AUCp, the better the model fit.

Once the best model was chosen, a ROC curve was used to determine the threshold to convert the proportion-based predictions back onto the 0/1 scale.

The Singapore dataset was used for validation purposes. Predictions were made using the best modified Leibovich algorithm (result of model ii) and converted using the estimated threshold from the same model.

All analyses were undertaken in R and using the packages MRSea, pROC, and caret.[37],[38],[39],[40] Plotting was done using the ggplot2 package.[41]


Leibovich score is good at predicting the risk of disease recurrence especially in low- and high-risk groups (model i)

In the training cohort, 93 cases (78%) out of 120 ccRCC had no recurrence at the time of data collection, with 50.5 months median recurrence-free survival, whereas 27 cases (22%) had either local or distant recurrence. LS (model i) predicted “No recurrence” with 76% of specificity (that is, a true negative rate = 71 out of 93 cases) and recurrence with 74% of sensitivity (that is, a true positive rate = 20 out of 27 cases) [Table 4]. This was seen similar in Pichler et al. who tested LS in an external cohort and showed concordance index, 0.778.[42]{Table 4}

[Figure 3] shows how the prediction accuracy varies across different LSs. A blue triangle shows a correct prediction of “No recurrence” or “Recurrence” and a red dot to indicate a prediction error for “No recurrence.” The [Figure 3] shows that most of the patients incorrectly predicted as “No recurrence,” were found in LS 5 and 6 categories supporting the previous study performed by Leibovich et al.[9] In particular, 14 cases of LS 5 were “No recurrence,” but all were predicted as “Recurrence,” giving a specificity of zero. Similarly, LS 6 showed specificity 0 for five patients.{Figure 3}

Replacement of manual nuclear grade with computational image analysis measurements in clear cell renal cell carcinoma nuclei improved the specificity of Leibovich score (model ii)

The variable “mean perimeter” was the only variable selected for model (ii), and so the best prediction of disease recurrence included this value together with the partial LS. This was termed the “modified Leibovich algorithm. The modified Leibovich algorithm improved the specificity by 86% compared to the original LS (model (i) specificity 76%) in this training cohort.

In particular, modified Leibovich algorithm predicted 8 cases out of 14 of LS 5 as “No recurrence” correctly, which improved the specificity from 0% to 57%. Similarly, the specificity of the LS 6 was improved from 0% to 40% [Figure 4].{Figure 4}

Validation of the modified Leibovich algorithm's “No recurrence” prediction in the Singaporean cohort

Of 217 ccRCC cases, 150 patients (69%) were designated as “No recurrence” at the time of data collection with 96.2 months of the median recurrence-free survival. Sixty-seven patients (31%) were classed as “Recurrence.”

Application of the modified Leibovich algorithm increased specificity from 84% to 94%, equivalent to correctly classifying and reassuring 15 more patients that their disease was unlikely to recur within 5 years. In particular, the modified Leibovich algorithm predicted 11 out of 12 cases of LS 5 as “No recurrence” correctly, which improved the specificity from 0% to 92% [Figure 5].{Figure 5}


We have investigated how digitization of slides using different scanners, computational image analysis and a statistical model can improve the specificity (correct prediction of “No recurrence”) of original LS in a proof of principle exercise. Replacement of manually assessed nuclear grade with mean perimeter, referred to as “Modified Leibovich algorithm differentiated ccRCC patients with no risk of disease recurrence for 5 years after surgery, particularly in the equivocal cases of LS 5 and 6. In the Scottish SCOTRRCC training cohort, the statistical model correctly predicted 9 more cases out of 19 with no risk of disease recurrence. This finding was also confirmed in an external independent Singaporean validation set, which had 13 more cases out of 15 correctly predicted as non-recurrence compared to LS prediction.

In this study, the number of nuclei assessed per section varied from 30,000 to more than 1000,000, depending on how much tumor was present in an individual block. ccRCC tumor nuclear profiles and sizes were measured in individual cells, which highlighted the variation of ccRCC at the level of morphology, as is well recognized at a genetic level.[43],[44],[45] We were able to use knowledge of the distribution of features in our statistical model, rather than having to commit to a single number as is the case for Fuhrman or ISUP grading.[24],[33],[35]

Other efforts have been made to improve the stratification of the risk of disease recurrence in localized ccRCC patients. Rini et al. developed a recurrence score using gene expression analysis in RNAs extracted from in total 1568 (sum of developmental and validation cohort) localized ccRCC patients' tissue blocks.[46] Out of 723 test genes, 11 cancer related genes were selected and used to calculate the recurrence score, which improved C statistics, an indicator of the prediction fitness for a binary outcome (e.g., “No recurrence” vs. “Recurrence”), up to 0.79 from 0.74 of LS alone.[46] Furthermore, when this recurrence score was combined with LS, its C statistics was increased to 0.81.[46] Brooks et al. also developed “ClearCode34” as a disease recurrence predictor in 95 localized ccRCC patients using gene expression analysis in RNA samples.[47] “ClearCode34” model was applied to 266 ccRCC patients genomic data from the Cancer Genome Atlas (TCGA) to predict the risk of disease recurrence and showed C-index (equal to C statistics), 0.65 and 0.70 when compared to UISS (0.575) and SSIGN score (0.625), respectively.[47] Our modified Leibovich algorithm compares favorably with these prediction scores and algorithms, and the approach we have outlined could readily be applied to these previous studies to determine whether there is any additional benefit to prognostic accuracy.

H&E-stained microscopic slides, routinely prepared in all histopathology laboratories for diagnosis, were used for computational image analysis. The H and E staining protocol, H and E reagents and scanners used in the test and validation cohorts were different but they showed consistent results, in part because we excluded features such as staining intensity that would be affected by different staining protocols between the Scottish training and Singaporean validation cohorts. Thus, computer-aided image analysis could be deployed at a central site to objectively assess patients for inclusion into trials, and potentially to reduce numbers required because of more accurate prediction of disease recurrence, thereby increasing the statistical power to detect differences in response to experimental treatment.


The authors would like to thank Frances Rae and Craig Marshall in the NRS BioResource and Tissue Governance Unit with tissue collection.

Financial support and sponsorship

The study was supported by Laboratory Medicine R&D Fund and iCAIRD.

Conflicts of interest

There are no conflicts of interest.


1Silverberg E, Grant RN. Cancer statistics, 1970. CA Cancer J Clin 1970;20:11-23.
2Kane CJ, Mallin K, Ritchey J, Cooperberg MR, Carroll PR. Renal cell cancer stage migration: Analysis of the National Cancer Data Base. Cancer 2008;113:78-83.
3Campbell SC, Rini BI. Current clinical urology. In: Campbell SC, Rini BI, editors. Renal Cell Carcinoma [electronic resource]: Clinical Management. New York: Humana Press; 2013.
4Chow WH, Devesa SS, Warren JL, Fraumeni JF Jr. Rising incidence of renal cell cancer in the United States. JAMA 1999;281:1628-31.
5Gonçalves PD, Srougi M, Dall'lio MF, Leite KR, Ortiz V, Hering F. Low clinical stage renal cell carcinoma: Relevance of microvascular tumor invasion as a prognostic parameter. J Urol 2004;172:470-4.
6Feng B, Liang P. Local thermal ablation of renal cell carcinoma. Eur J Radiol 2012;81:437-40.
7Russo P. Renal cell carcinoma: Presentation, staging, and surgical treatment. Semin Oncol 2000;27:160-76.
8Escudier B, Porta C, Schmidinger M, Algaba F, Patard JJ, Khoo V, et al. Renal cell carcinoma: ESMO Clinical Practice Guidelines for diagnosis, treatment and follow-up. Ann Oncol 2014;25 Suppl 3:iii49-56.
9Leibovich BC, Blute ML, Cheville JC, Lohse CM, Frank I, Kwon ED, et al. Prediction of progression after radical nephrectomy for patients with clear cell renal cell carcinoma: A stratification tool for prospective clinical trials. Cancer 2003;97:1663-71.
10Warren AY, Harrison D. WHO/ISUP classification, grading and pathological staging of renal cell carcinoma: Standards and controversies. World J Urol 2018;36:1913-26.
11Amin MB, Greene FL, Edge SB, Compton CC, Gershenwald JE, Brookland RK, et al. The eighth edition AJCC cancer staging manual: Continuing to build a bridge from a population-based to a more “personalized” approach to cancer staging. CA Cancer J Clin 2017;67:93-9.
12Figlin RA, Leibovich BC, Stewart GD, Negrier S. Adjuvant therapy in renal cell carcinoma: Does higher risk for recurrence improve the chance for success? Ann Oncol 2018;29:324-31.
13Blick C, Ritchie AW, Eisen T, Stewart GD. Improving outcomes in high-risk, nonmetastatic renal cancer: New data and ongoing trials. Nat Rev Urol 2017;14:753-9.
14Frank I, Blute ML, Cheville JC, Lohse CM, Weaver AL, Zincke H. An outcome prediction model for patients with clear cell renal cell carcinoma treated with radical nephrectomy based on tumor stage, size, grade and necrosis: The SSIGN score. J Urol 2002;168:2395-400.
15Zisman A, Pantuck AJ, Dorey F, Said JW, Shvarts O, Quintana D, et al. Improved prognostication of renal cell carcinoma using an integrated staging system. J Clin Oncol 2001;19:1649-57.
16Janowitz T, Welsh SJ, Zaki K, Mulders P, Eisen T. Adjuvant therapy in renal cell carcinoma-past, present, and future. Semin Oncol 2013;40:482-91.
17Novara G, Martignoni G, Artibani W, Ficarra V. Grading systems in renal cell carcinoma. J Urol 2007;177:430-6.
18Delahunt B. Advances and controversies in grading and staging of renal cell carcinoma. Mod Pathol 2009;22 Suppl 2:S24-36.
19Young MD, Mitchell TJ, Vieira Braga FA, Tran MG, Stewart BJ, Ferdinand JR, et al. Single-cell transcriptomes from human kidneys reveal the cellular identity of renal tumors. Science 2018;361:594-9.
20Patard JJ, Leray E, Rioux-Leclercq N, Cindolo L, Ficarra V, Zisman A, et al. Prognostic value of histologic subtypes in renal cell carcinoma: A multicenter experience. J Clin Oncol 2005;23:2763-71.
21Sika-Paotonu D, Bethwaite PB, McCredie MR, Jordan TW, Delahunt B. Nucleolar grade but not Fuhrman grade is applicable to papillary renal cell carcinoma. Am J Surg Pathol 2006;30:1091-6.
22Delahunt B, Egevad L, Samaratunga H, Martignoni G, Nacey JN, Srigley JR. Gleason and Fuhrman no longer make the grade. Histopathology 2016;68:475-81.
23Engers R. Reproducibility and reliability of tumor grading in urological neoplasms. World J Urol 2007;25:595-605.
24Trpkov K, Grignon DJ, Bonsib SM, Amin MB, Billis A, Lopez-Beltran A, et al. Handling and staging of renal cell carcinoma: The International Society of Urological Pathology Consensus (ISUP) conference recommendations. Am J Surg Pathol 2013;37:1505-17.
25Webster JD, Dunstan RW. Whole-slide imaging and automated image analysis: Considerations and opportunities in the practice of pathology. Vet Pathol 2014;51:211-23.
26Cardesa A, Zidar N, Alos L, Nadal A, Gale N, Klöppel G. The Kaiser's cancer revisited: Was Virchow totally wrong? Virchows Arch 2011;458:649-57.
27García-Rojo M. International clinical guidelines for the adoption of digital pathology: A review of technical aspects. Pathobiology 2016;83:99-109.
28Hamilton PW, Bankhead P, Wang Y, Hutchinson R, Kieran D, McArt DG, et al. Digital pathology and image analysis in tissue biomarker research. Methods 2014;70:59-73.
29Beck AH, Sangoi AR, Leung S, Marinelli RJ, Nielsen TO, Van De Vijver MJ, et al. Systematic analysis of breast cancer morphology uncovers stromal features associated with survival. Sci Transl Med 2011;3:108ra113.
30Yeh FC, Parwani AV, Pantanowitz L, Ho C. Automated grading of renal cell carcinoma using whole slide imaging. J Pathol Inform 2014;5:23.
31Bankhead P, Loughrey MB, Fernández JA, Dombrowski Y, McArt DG, Dunne PD, et al. QuPath: Open source software for digital pathology image analysis. Sci Rep. 2017;7:16878.
32Stewart GD, Riddick AC, Rae F, Marshall C, MacLeod L, O'Mahony FC, et al. Translational research will fail without surgical leadership: SCOTRRCC a successful surgeon-led Nationwide translational research infrastructure in renal cancer. Surgeon 2015;13:181-6.
33Samaratunga H, Gianduzzo T, Delahunt B. The ISUP system of staging, grading and classification of renal cell neoplasia. J Kidney Cancer VHL 2014;1:26-39.
34Horai Y, Kakimoto T, Takemoto K, Tanaka M. Quantitative analysis of histopathological findings using image processing software. J Toxicol Pathol 2017;30:351-8.
35Fuhrman SA, Lasky LC, Limas C. Prognostic significance of morphologic parameters in renal cell carcinoma. Am J Surg Pathol 1982;6:655-63.
36Walker CG, Mackenzie ML, Donovan CR, O'Sullivan MJ. SALSA – A spatially adaptive local smoothing algorithm. J Stat Comput Simul 2011;81:179-91.
37Robin X, Turck N, Hainard A, Tiberti N, Lisacek F, Sanchez JC, et al. pROC: An open-source package for R and S+to analyze and compare ROC curves. BMC Bioinformatics 2011;12:77.
38Kuhn M, Wing J, Weston S, Williams A, Keefer C, Engelhardt A, et al. Caret: Classification and Regression Training. R package version 6.0-71. 2016.
39Team RC. A language and environment for statistical computing. In: R Foundation for Statistical Computing. Vienna, Austria: 2019. Available online at
40Scott-Hayward L, Oedekoven CS, Mackenzie ML, Walker CG. Vignette for the MRSea Package v1.01: Statistical modelling of bird and cetacean distributions in offshore renewables development areas. In: Centre for Research into Ecological and Environmental Modelling. published in University of St Andrews, CREEM: University of St. Andrews; 2019.
41Wickham H. Ggplot2: Elegant Graphics for Data Analysis. New York, NY: Springer; 2016.
42Pichler M, Hutterer GC, Chromecki TF, Jesche J, Kampel-Kettner K, Rehak P, et al. External validation of the Leibovich prognosis score for nonmetastatic clear cell renal cell carcinoma at a single European center applying routine pathology. J Urol 2011;186:1773-7.
43Gerlinger M, Rowan AJ, Horswell S, Math M, Larkin J, Endesfelder D, et al. Intratumor heterogeneity and branched evolution revealed by multiregion sequencing. N Engl J Med 2012;366:883-92.
44Turajlic S, Xu H, Litchfield K, Rowan A, Chambers T, Lopez JI, et al. Tracking cancer evolution reveals constrained routes to metastases: TRACERx renal. Cell 2018;173: 581-94.e12.
45Ball MW, Bezerra SM, Gorin MA, Cowan M, Pavlovich CP, Pierorazio PM, et al. Grade heterogeneity in small renal masses: Potential implications for renal mass biopsy. J Urol 2015;193:36-40.
46Rini B, Goddard A, Knezevic D, Maddala T, Zhou M, Aydin H, et al. A 16-gene assay to predict recurrence after surgery in localised renal cell carcinoma: Development and validation studies. Lancet Oncol 2015;16:676-85.
47Brooks SA, Brannon AR, Parker JS, Fisher JC, Sen O, Kattan MW, et al. ClearCode34: A prognostic risk predictor for localized clear cell renal cell carcinoma. Eur Urol 2014;66:77-84.