|
 |
| ORIGINAL ARTICLE |
|
|
|
| J Pathol Inform 2020,
11:22 |
A regulatory science initiative to harmonize and standardize digital pathology and machine learning processes to speed up clinical innovation to patients
Hetal Desai Marble1, Richard Huang1, Sarah Nixon Dudgeon2, Amanda Lowe3, Markus D Herrmann4, Scott Blakely5, Matthew O Leavitt6, Mike Isaacs7, Matthew G Hanna8, Ashish Sharma9, Jithesh Veetil10, Pamela Goldberg10, Joachim H Schmid11, Laura Lasiter12, Brandon D Gallas2, Esther Abels13, Jochen K Lennerz1
1 Department of Pathology, Center for Integrated Diagnostics, Harvard Medical School, Massachusetts General Hospital, Boston, MA, USA
2 Division of Imaging, Diagnostics, and Software Reliability, Center for Devices and Radiological Health, Food and Drug Administration, Office of Science and Engineering Laboratories, Silver Spring, MD, USA
3 Visiopharm Americas, Westminster, CO, USA
4 Department of Pathology, Harvard Medical School, Massachusetts General Hospital, Boston, MA, USA
5 Hamamatsu Corporation, Pittsburgh, PA, USA
6 LUMEA, Salt Lake City, UT, USA
7 Department of Pathology and Immunology, Washington University School of Medicine, St. Louis, MO, USA
8 Department of Pathology, Memorial Sloan Kettering Cancer Center, New York, NY, USA
9 Department of Biomedical Informatics, Emory University School of Medicine, Atlanta, GA, USA
10 Medical Device Innovation Consortium, Arlington, VA, USA
11 Roche Tissue Diagnostics, Santa Clara, USA
12 Friends of Cancer Research, Washington, DC, USA
13 PathAI, Boston, MA, USA
| Date of Submission | 01-Apr-2020 |
| Date of Decision | 20-Apr-2020 |
| Date of Acceptance | 16-Jun-2020 |
| Date of Web Publication | 06-Aug-2020 |
Correspondence Address:
Dr. Jochen K Lennerz
Department of Pathology, Center for Integrated Diagnostics, Harvard Medical School, Massachusetts General Hospital, 55 Fruit Street, GRJ1015 Boston, MA 02114
USA
 Source of Support: None, Conflict of Interest: None  | Check |
DOI: 10.4103/jpi.jpi_27_20
 Abstract | | |
Unlocking the full potential of pathology data by gaining computational access to histological pixel data and metadata (digital pathology) is one of the key promises of computational pathology. Despite scientific progress and several regulatory approvals for primary diagnosis using whole-slide imaging, true clinical adoption at scale is slower than anticipated. In the U.S., advances in digital pathology are often siloed pursuits by individual stakeholders, and to our knowledge, there has not been a systematic approach to advance the field through a regulatory science initiative. The Alliance for Digital Pathology ( the Alliance) is a recently established, volunteer, collaborative, regulatory science initiative to standardize digital pathology processes to speed up innovation to patients. The purpose is: (1) to account for the patient perspective by including patient advocacy; (2) to investigate and develop methods and tools for the evaluation of effectiveness, safety, and quality to specify risks and benefits in the precompetitive phase; (3) to help strategize the sequence of clinically meaningful deliverables; (4) to encourage and streamline the development of ground-truth data sets for machine learning model development and validation; and (5) to clarify regulatory pathways by investigating relevant regulatory science questions. The Alliance accepts participation from all stakeholders, and we solicit clinically relevant proposals that will benefit the field at large. The initiative will dissolve once a clinical, interoperable, modularized, integrated solution (from tissue acquisition to diagnostic algorithm) has been implemented. In times of rapidly evolving discoveries, scientific input from subject-matter experts is one essential element to inform regulatory guidance and decision-making. The Alliance aims to establish and promote synergistic regulatory science efforts that will leverage diverse inputs to move digital pathology forward and ultimately improve patient care.
Keywords: Artificial intelligence, digital pathology, machine learning, regulatory science, slide scanning
How to cite this article:
Marble HD, Huang R, Dudgeon SN, Lowe A, Herrmann MD, Blakely S, Leavitt MO, Isaacs M, Hanna MG, Sharma A, Veetil J, Goldberg P, Schmid JH, Lasiter L, Gallas BD, Abels E, Lennerz JK. A regulatory science initiative to harmonize and standardize digital pathology and machine learning processes to speed up clinical innovation to patients. J Pathol Inform 2020;11:22 |
How to cite this URL:
Marble HD, Huang R, Dudgeon SN, Lowe A, Herrmann MD, Blakely S, Leavitt MO, Isaacs M, Hanna MG, Sharma A, Veetil J, Goldberg P, Schmid JH, Lasiter L, Gallas BD, Abels E, Lennerz JK. A regulatory science initiative to harmonize and standardize digital pathology and machine learning processes to speed up clinical innovation to patients. J Pathol Inform [serial online] 2020 [cited 2021 Nov 27];11:22. Available from: https://www.jpathinformatics.org/text.asp?2020/11/1/22/291538 |
| Introduction | |  |
“The scientist and science provide the means, the politician and politics decide the ends.”
-Alvin M. Weinberg[1]
Regulatory science is an established discipline that entails the application of the scientific method to support regulatory and other policy objectives.[2] Simply put, when medical research provides a novel solution to a health need, regulatory science applies the scientific method to assess benefits and risks before marketing for clinical use. To assess benefits and risks, regulatory scientists develop new tools, standards, and approaches to evaluate the effectiveness, safety, and quality of medical products. A primary challenge in the field of digital pathology is the lack of understanding that strong relationships between regulatory, basic, and translational scientists can substantially improve clinical innovation.[3],[4],[5],[6] For example, regulatory science is not restricted to regulatory agencies.[2],[4],[5],[6] As a scientific discipline, regulatory science challenges current concepts of benefit and risk assessments, submission and approval strategies, patient involvement, and various ethical aspects. Regulatory science includes the creation of a scientific dialog for launching new ideas – not only derived from industry and regulatory authorities but also by, for example, academics, clinicians, and patients.[7] It has been recognized that regulatory science can have a significant impact in bringing new devices to patients in need.[7]
Here, we outline a recently established, volunteer, collaborative regulatory science initiative termed the Alliance for Digital Pathology (the Alliance). To prevent confusion, our intent is to familiarize the community with the aims, scope, and rationale of the Alliance. The Alliance aims to move the field of digital pathology forward by systematically assessing relevant aspects and providing publicly available resources (e.g., data, tools, and methods) to inform and improve the relevant regulatory guidance landscape.[8] Our premise (thesis) is that the Alliance promotes regulatory science as a bridge between digital pathology (the means) and moving the field of diagnostic pathology forward (the ends). By promoting regulatory science, the Alliance helps to unlock the potential of new technologies and thereby overcomes the dichotomy illustrated in the epigraph by Dr. Weinberg.[1]
| Toward an Operational Definition of a Clinical, Interoperable, Integrated Solution for Digital Pathology | | |
The key aim of the Alliance is to help convert the existing (traditional) pathology technologies and workflows into interoperable, digitally enhanced solutions by contributing regulatory science deliverables that can be used to inform and improve the applicable regulatory guidance landscape. Numerous groups have attempted to specify the relevant components of digital pathology solutions;[9],[10],[11],[12],[13],[14],[15],[16],[17],[18] however, given the modularized nature of diagnostic pathology, defining the specific scope of a digital pathology solution is highly context dependent. For example, the variability of a stain (e.g., hematoxylin and eosin across or within laboratories) may influence the performance of a downstream mutation prediction algorithm.[19],[20],[21] In this example, one may consider drawing an arbitrary boundary before the staining step; however, the fixation and processing method (e.g., formalin fixed, paraffin embedded) or even the tissue acquisition, handling, or image acquisition[22] may influence the performance of the predictor as well. Thus, for the purpose of the Alliance, we considered three descriptors for the solution. First, we aim toward a clinical (as opposed to a research-based) solution. Second, due to the modularized nature of the various subprocesses within the main workflows in pathology, we aim for interoperability of systems. Third, to account for the various and arbitrary boundaries of workflow steps (modules) and technologies relevant for a given task (intended use), we consider every step, from the medical procedure acquiring the cell or tissue sample all the way to the fully integrated diagnostic output (e.g., report or model output), as relevant. As opposed to an end-to-end solution, where the supplier of an application or system will provide all the hardware and/or software to meet specific requirements, we are aiming for modularized solutions within the main workflow. We refer to these three solution descriptors (clinical, interoperable, and modularized) as an “integrated solution” for digital pathology. We acknowledge that this definition is operational and arguably incomplete yet represents a technique that enables flexible modeling to solve challenging problems.[23],[24],[25],[26]
| The Multifaceted Nature of Digital Pathology Needs Increased Regulatory Clarity | | |
Digital pathology has grown into a multimillion-dollar vendor landscape,[27] and the application of machine learning algorithms holds big promise for improving diagnostics in numerous ways.[28],[29],[30] Despite this active and promising research, the Food and Drug Administration (FDA) has only recently authorized two digital pathology whole-slide imaging (WSI) systems for primary diagnosis.[3],[9],[11],[31],[32] Even with the authorization of two WSI systems and numerous use cases,[12],[13],[14],[18],[33],[34],[35],[36],[37],[38] in the U.S., we see few hospitals changing their daily clinical operations to integrate WSI for primary diagnosis.[39],[40],[41],[42],[43] Clinical laboratories face additional challenges when implementing high complexity and/or high-risk medical devices coupled with software solutions as laboratory-developed tests (LDTs).[44],[45],[46] For example, even when using an FDA-authorized whole-slide imaging device, the approval or clearance does not eliminate the need for an individual laboratory to verify the performance of these systems for the specific intended diagnostic purpose. Specifically, Clinical Laboratory Improvement Amendments of 1988 or CLIA '88 in the US requires at least verification[47] and substantial adaptation to implement.[48],[49],[50],[51],[52]
One value proposition for digital pathology is to take advantage of the digital nature of WSI and use artificial intelligence/machine learning (AI/ML) algorithms to support clinical decisions.[11],[53] In fact, several groups have proposed that AI/ML will unlock the full potential of digital pathology.[53],[54]
To examine the current regulatory guidance landscape related to digital pathology and AI, four authors (HDM, RH, EA, and JKL) performed a review of pertinent documents from the FDA. We noted the official release dates and assigned each document to one of five dimensions [Figure 1] and [Supplemental Table 1 [Additional file 1]]. By plotting these documents and dimensions over time, we show how the regulatory guidance landscape evolves. A novice in the field may look for one comprehensive guidance document for digital pathology and may be discouraged by the initial complexity; however, we hope that [Figure 1] provides a reasonable starting point for learning the current regulatory guidance landscape. As we show [Figure 1], arrows], the regulatory guidance landscape adapts over time as technologies and the associated regulatory science matures. One key element in the multistep process to improve the regulatory guidance landscape is critical scientific input from subject-matter experts.[3],[4],[5],[10],[11],[15],[53] We strongly believe that “watching and waiting” will not help the case of digital pathology. Similarly, workarounds[84],[85],[86],[87],[88],[89] turn into long and winding roads that ultimately end at the FDA and within the FDA's regulatory framework.[83] The Alliance intends to organize subject-matter experts and provide scientific input. | Figure 1: Overview of selected FDA guidance documents. Four of the authors (HM, RH, EA, and JKL) performed a meta-review of selected FDA guidance documents relevant to the scope and aims of the Alliance. The figure shows grouping of these guidance documents across five dimensions over time. Please note: the numbers refer to the order of review during the meta-review process; Supplemental Table 1 provides the original release dates, the official FDA guidance title, and the issuer. AI/ML: Artificial intelligence/machine learning; CMS: Centers for Medicare and Medicaid Services; FDA: Food and Drug Administration; IMDRF: International Medical Device Regulators Forum; MDDT: Medical Device Development Tools; SaMD: Software as a Medical Device; QMS: Quality management system; WSI: Whole-slide imaging
Click here to view |
Simply put, the practical dilemma in digital pathology is that developers are challenged to create an FDA submission following the evolving and complex regulatory guidance landscape, and the adoption of WSI by pathologists is slowed because they cannot realize the full potential and utility of digital pathology and AI/ML without full clinical integration. The field of digital pathology is looking for broader guidance, practical advice, and streamlined regulatory pathways to help navigate this uncharted and exciting territory.
| Regulatory Science, the Precompetitive Space, and Real-World Evidence | | |
FDA clearance of a medical device offers a vendor market access. Once introduced, market forces tend not to encourage the vendor to make the device or its subsystems interoperable.[55],[56],[57],[58],[59],[60],[61] We like to emphasize that routine diagnostic pathology is highly modularized and the practice does not lend itself easily to nonmodular, locked down solutions.[3],[9],[10],[11],[27],[50],[51],[54],[62] The Alliance believes that it can promote interoperability and innovation by launching initiatives and creating deliverables (data, standards, tools, and methods) in the precompetitive space. Organizing industry to work collaboratively in the precompetitive space will eliminate unnecessary or duplicative (proprietary) efforts and thereby save all parties' time, money, and resources when pursuing device authorizations.[63] The Alliance initiatives and deliverables will speed clinical integration and carry mutual benefit to all stakeholders, including regulators, clinicians, manufacturers, and most importantly, patients.
Real-world evidence (RWE) comes from the competitive, postmarket space. RWE can identify trends in adverse events, summarize where resources are being spent, and track the impact of a new diagnostic device or therapy in terms of patient outcomes. RWE can support clinical practice guidelines and decisions about reimbursement and policy. Furthermore, RWE can inform regulatory decision making, as effectively demonstrated by the Medical Device Innovation Consortium,[64],[65] the National Evaluation System for health Technology Coordinating Center,[66] the Patient-Centered Outcomes Research Institute,[67],[68] Friends of Cancer Research,[69],[70] and others.[3],[5],[6],[9],[71],[72],[73],[74]
| From Key Mission Elements to a Delivery Process | | |
Accomplishing mutual benefit to multiple stakeholders is a daunting value proposition that requires a unique regulatory science approach and stakeholder involvement for selection and prioritization of deliverables. The approach of the Alliance [Figure 2]a is to deliver tools by harnessing existing, precompetitive FDA programs and use the gained experience to inform effective regulation. The approach thereby aims to streamline precompetitive and eventually competitive submissions that enable faster time to market to improve patient care. Regulatory science deliverables, including tools and the experience from precompetitive submissions, will be shared, and when one integrated solution has been enabled, the Alliance can dissolve [Figure 2]a. The key mission elements of the Alliance are summarized in [Table 1].[75] | Figure 2: Concept, process, role, and proposed benefits of the Alliance. (a) The approach of the Alliance is to deliver tools via precompetitive FDA programs and use the gained experience to support effective FDA review. The concept also includes a predetermined exit strategy (i.e., one fully integrated solution for digital pathology). (b) The process of moving Alliance projects forward is essentially a two-step, multidisciplinary peer review by subject-matter experts. First, projects are reviewed, and after a multidisciplinary selection process that emphasizes the patient perspective and relevance for patient care, the steering committee (jointly with relevant partners) attempts to allocate resources. (c) Role and proposed benefits of the Alliance exemplified using the high-throughput truthing project for tumor-infiltrating lymphocytes as a biomarker in breast cancer. AMCs: Academic medical centers; MDDT: Medical Device Development Tools (precompetitive FDA submission program); Mock: mock submission program (precompetitive FDA submission program); OIR: Office of In vitro Diagnostics and Radiological Health; OPEQ: Office of Product Evaluation and Quality; OSEL: Office of Science and Engineering Laboratories; FDA: Food and Drug Administration
Click here to view |
To align stakeholder interests, initiatives and deliverables need to be prioritized and prioritization requires a process. We conceptualized an approach that is composed of synergistic review, project components, and resource allocation [Figure 2]b. The process starts with synergizing various stakeholder interests into concise individual projects. An Alliance project may consist of a clinically relevant intended use case, a data set (e.g., pixel and metadata), and an applicable regulatory science pathway [e.g., [Figure 2]b, triangle]. The Alliance membership, composed of subject-matter experts from various domains, will have the opportunity to review, contribute, and potentially modify these projects through free and voluntary feedback to the project owner. Over time, individual effort and maturation of ideas will result in optimized projects (“big ideas”). To help realize the proposed deliverables and/or allocate additional resources, we established the Alliance Steering Committee, a flexible organizational structure, and a code of conduct [Supplemental Table 2 [Additional file 2]]
An example project is illustrated in [Figure 2]c. A subset of members in the Alliance are studying the relevance of tumor-infiltrating lymphocytes (TILs) as a prognostic and predictive biomarker.[76],[77] The interest in this clinical use case led to a collaborative project that includes members from the FDA, academic medical centers (AMCs), and industry. The project, referred to as the high-throughput truthing (HTT) project, aims to demonstrate the collection and use of pathologist annotations for the purpose of evaluating AI/ML algorithms and other digital pathology initiatives. The project also aims to qualify the glass slides, whole-slide images, and pathologist annotations for evaluating AI/ML algorithms through the precompetitive FDA's Medical Device Development Tools (MDDT) program.[78] If qualified, the “ground-truth” materials can serve as a publicly available, standardized evaluation “tool” for algorithm evaluation that can be used in submissions to the FDA.
In relation to the Alliance, the HTT project was submitted to the Alliance and discussed in November 2019. The Alliance can contribute in multiple ways to accelerate the realization of this and similar projects. First, the Alliance confirmed that the aims of the project could benefit many stakeholders.
The discussions provided useful feedback from subject-matter experts regarding the clinical use case, sourcing slides from multiple sites, agreements for sharing materials within the project, and issues related to sharing materials publicly. The discussions also identified future work that could build on the lessons, methods, infrastructure, and relationships created while pursuing the current aims. Important future work identified in the discussions included scaling the effort to address generalizability across sites and generalizability across use cases.
The Alliance has since provided help with the project [Figure 2]b, triangle 01, relevant intended use case; [Figure 2]c, 01] by disseminating the project needs. This networking through the Alliance has yielded volunteers for sourcing and scanning slides, pathologists to annotate slides and images, and opportunities to collect data. Connections have been created that are expected to help in the development of the statistical analyses and the future hosting of slides, images, and annotations. Currently, the project is developing the strategy and materials for the FDA's MDDT program [Figure 2]b, triangle, MDDT; [Figure 2]c, 03]. The development is a learning experience for all involved, with contributions from project and Alliance subject-matter and regulatory affairs experts. The learning experience is expected to continue through official interactions with the FDA related to the MDDT submission. Thus, aside from helping to create the ground-truth data set, the Alliance aims to understand regulatory issues and processes for future streamlining of other projects and submissions. As demonstrated here, a qualified data set may result in time-savings when preparing submissions, generating additional tools, and streamlining regulatory review, resulting in faster time to market and improved patient care.
| Who is the Alliance ? | | |
The Alliance is composed of a diverse and interdisciplinary group of stakeholders who contribute to various aspects of diagnostic pathology, from tissue acquisition to reporting and data analytics. When deconstructing the clinical digital pathology and AI/ML pipeline into its component parts, numerous workflow steps have to function in unison [Figure 3]a. Aside from the modular nature and operational complexity, these components emphasize the importance of involving various stakeholders with each module. Given the novelty of pursuing a collaborative regulatory science effort to solve the challenge of clinical adoption of digital pathology, we noted a lack of coRegulatory Submissions
ncrete data on interested stakeholders and their priorities. In September 2019, we conducted an internal survey [n = 42; [Supplemental Table 3 [Additional file 3]]. At that time, the survey respondents stated that the top 3 deliverables/workflow steps to focus on should be the DICOM standard, AI/ML test validation, and pixel and metadata capture [Figure 3]b. By self-reported primary affiliation, the Alliance encompasses representation from academia (32%), industry (50%), government regulators and nongovernment organizations (12%), and patient advocacy groups (6%) [Figure 3]c. | Figure 3: Workflow steps and Alliance survey results. (a) Digital pathology workflows include preanalytical, retrieval, scan (image acquisition), clinical data, metadata, machine learning algorithm development, clinical integration, clinical utility, and financial sustainability considerations; all dependent on the specific use case/application. These workflow steps correspond to the axis labels in Figure 3b. (b) The Alliance conducted a survey among the members in September 2019. Bar graphs show the workflow steps that survey respondents felt the Alliance should focus on. These steps are reflected in a workflow diagram in Figure 3a. (c) Survey results from September 2019. DICOM: Digital Imaging and Communications in Medicine (here referring to an interoperable file format for digital pathology); EHR: Electronic health record; H&E: Hematoxylin and eosin stain; IHC: Immunohistochemistry; LIMS: Laboratory information management system; MDIC: Medical Device Innovation Consortium
Click here to view |
| Meetings, Growth, and Working Groups | | |
Since its inception in May 2019, the Alliance hosted numerous teleconferences, web meetings, and three, in-person, national meetings [Figure 4]a. Over this period (May 2019–January 2020), the Alliance membership grew from an initial n = 37 (July 2019) to n = 322 individuals [May 2020; [Figure 4]a. Each of these in-person meetings solicited collaborative input from stakeholders toward execution of concrete regulatory science deliverables. [Figure 4]a also includes the number of participants and frequency of steering committee web meetings. By July 2019, it became clear that various stakeholders worked on or had interest in distinct topics that the Alliance subsequently organized into 8 working groups by autumn 2019 [Figure 4]b. These group topics are intended to align stakeholders with subject-matter expertise and interest. Clearly, some functional requirements are relevant for multiple groups. However, we hope to minimize such redundancies by providing clear documentation of projects through appropriate project management and frequent content updates. The names of the founding and current working group leaders are provided in [Figure 4]b. One example of a regulatory science deliverable is also provided per group [Figure 4]b. For further updates or details on the various topics, please visit the Alliance website[8] or to become a member and get involved. | Figure 4: Roadmap and working groups. (a) Roadmap of in-person events (status May 2020). In addition to the date, the roadmap shows hosting organization, key developments, and location of the meetings. The graph shows the membership number over time along with the number and frequency of the steering committee meetings as well as the high-throughput truthing working group. (b) The Alliance proposed to tackle regulatory science deliverables in digital pathology by splitting up the topic into eight distinct working groups. Each workgroup is provided with the steering committee member (s) and at least one key regulatory science deliverable. The steering committee is also responsible for minimizing redundancy between the workgroups. AI: Artificial intelligence; DPA: Digital Pathology Association; FDA: Food and Drug Administration; HTT: High-throughput truthing (an independent workgroup); MDIC: Medical Device Innovation Consortium; ML: Machine learning; USCAP: USCAP stands for United States and Canadian Academy of Pathology
Click here to view |
| The Alliance facilitates Regulatory Submissions | | |
As a first key regulatory science deliverable, in late 2019, members of the Alliance submitted an MDDT proposal to the FDA for review (HTT project described above). The experience gained through this submission will create a starting point and testing ground for the proposed approach of the Alliance. In contrast to the largely confidential submission owned by the submitting entity (typically represented through a consulting firm and/or a regulatory affairs division), gaining and sharing the submission experience may inform subsequent submissions, and Alliance members can draw from the experience of these submissions. This particular concept is new to digital pathology. Similarly, we consider several precompetitive submission programs by the FDA[78],[79] a paradigm shift that enables different ways to engage with regulatory entities. Importantly, the Alliance intends to create a repository of submission documents as a resource to bolster subsequent submissions with the collective experience of previous submitters. We propose that the field, and in particular patients,[80] will ultimately benefit from sharing the experiences of Alliance members who have submitted to regulatory agencies.
| Conclusion | | |
In the current environment of sparse and dispersed regulatory guidance for digital pathology and AI/ML, with siloed pursuits by diverse stakeholders, the Alliance saw an opportunity to establish an important missing element: a precompetitive regulatory science collaboration. We believe that for patients to benefit from highly complex new technologies, benefit and risk assessments are essential.[81],[82] The Alliance helps tackle this daunting task (i. e., benefit and risk assessment for digital pathology and AI/ML) through regulatory sciences with the hope of successful clinical integration and improved patient care. That said, there are numerous issues that we need to address. For example, we want to investigate and develop protocols and definitions for continuous performance assessments of continuously learning ML algorithms. Similarly, approaching financial sustainability will require clear demonstration of clinical utility. However, the fact that numerous unanswered questions persist represents an opportunity for other agencies, regulatory entities, professional groups, and collaborative movements (like the Alliance) to step up and drive developments toward comprehensive risk and safety assessments. It is important to emphasize the crucial importance of funding for regulatory and implementation science projects, in particular those that aim to inform technically appropriate and efficient science-based regulatory decision-making processes. Such funding is needed to advance cutting-edge innovations into clinical practice. In summary, the Alliance aims to advance the field of digital pathology and we hope that synergistic efforts between various stakeholders and regulatory scientists will ultimately speed the improvement of patient care. This begs the question: Who, if not us?
Acknowledgments
The Alliance is supported by the Medical Device Innovation Consortium (Arlington, VA), the Digital Pathology Association and the Digital Pathology Association Foundation (Carmel, IN), and the Center for Integrated Diagnostics, Department of Pathology, Massachusetts General Hospital/Harvard Medical School (Boston, MA). This work is also in part supported by NIH (RO1 CA225655) to J.K.L, and the content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institute of Health or any other organization.
Financial support and sponsorship
Nil.
Conflicts of interest
There are no conflicts of interest.
| References | | |
| 1. | Weinberg AM. Science and trans-science. Science 1972;177:211.  |
| 2. | |
| 3. | Abels E, Pantanowitz L. Current state of the regulatory trajectory for whole slide imaging devices in the USA. J Pathol Inform 2017;8:23. [ PUBMED] [Full text] |
| 4. | Spindler P, Bach KF, Schmiegelow M, Bedlington N, Eichler HG. Innovation of medical products: the evolution of regulatory science, research, and education. Ther Innov Regul Sci 2016;50:44-8. |
| 5. | |
| 6. | FDA. Regulatory Science in FDA's Center for Devices and Radiological Health: A Vital Framework for Protecting and Promoting Public Health; 2018. |
| 7. | |
| 8. | |
| 9. | Abels E, Pantanowitz L, Aeffner F, Zarella MD, van der Laak J, Bui MM, et al. Computational pathology definitions, best practices, and recommendations for regulatory guidance: A white paper from the Digital Pathology Association. J Pathol 2019;249:286-94. |
| 10. | Aeffner F, Zarella MD, Buchbinder N, Bui MM, Goodman MR, Hartman DJ, et al. Introduction to digital image analysis in whole-slide imaging: A white paper from the digital pathology association. J Pathol Inform 2019;10:9. [ PUBMED] [Full text] |
| 11. | Bera K, Schalper KA, Rimm DL, Velcheti V, Madabhushi A. Artificial intelligence in digital pathology New tools for diagnosis and precision oncology. Nat Rev Clin Oncol 2019;16:703-15. |
| 12. | García-Rojo M. International Clinical guidelines for the adoption of digital pathology: A review of technical aspects. Pathobiology 2016;83:99-109. |
| 13. | Guo H, Birsa J, Farahani N, Hartman DJ, Piccoli A, O'Leary M, et al. Digital pathology and anatomic pathology laboratory information system integration to support digital pathology sign-out. J Pathol Inform 2016;7:23. [ PUBMED] [Full text] |
| 14. | Hosseini MS, Brawley-Hayes JA, Zhang Y, Chan L, Plataniotis K, Damaskinos S. Focus quality assessment of high-throughput whole slide imaging in digital pathology. IEEE Trans Med Imaging 2020;39:62-74. |
| 15. | Lange H. Digital Pathology: A regulatory overview. Lab Med 2011;42:587-91. |
| 16. | Mukhopadhyay S, Feldman MD, Abels E, Ashfaq R, Beltaifa S, Cacciabeve NG, et al. Whole slide imaging versus microscopy for primary diagnosis in surgical pathology: A multicenter blinded randomized noninferiority study of 1992 cases (pivotal study). Am J Surg Pathol 2018;42:39-52. |
| 17. | Niazi MKK, Parwani AV, Gurcan MN. Digital pathology and artificial intelligence. Lancet Oncol 2019;20:e253-61. |
| 18. | Williams BJ, Bottoms D, Clark D, Treanor D. Future-proofing pathology part 2: Building a business case for digital pathology. J Clin Pathol 2019;72:198-205. |
| 19. | Azar JC, Busch C, Carlbom IB. Histological stain evaluation for machine learning applications. J Pathol Inform 2013;4:S11. [ PUBMED] [Full text] |
| 20. | Komura D, Ishikawa S. Machine learning methods for histopathological image analysis. Comput Struct Biotechnol J 2018;16:34-42. |
| 21. | Ström P, Kartasalo K, Olsson H, Saolorzano L, Delahunt B, Berney DM, et al. Pathologist-level grading of prostate biopsies with artificial intelligence. arXiv preprint arXiv:1907.01368 (2019). |
| 22. | Rivenson Y, Liu T, Wei Z, Zhang Y, de Haan K, Ozcan A. PhaseStain: The digital staining of label-free quantitative phase microscopy images using deep learning. Light Sci Appl 2019;8:23. |
| 23. | Alba E. Optimization techniques for solving complex problems. In: Hoboken NJ, editor. Wiley: Telecom; 2009. p. 476. |
| 24. | Frensch PA, Funke J. Complex problem solving: The European perspective. In: Hoboken NJ, editor. L. Erlbaum Associates: Psychology Press; 1 edition; 1995. p. 340. |
| 25. | Malakooti B. Operations and Production Systems with Multiple Objectives. Hoboken, New Jersey: John Wiley & Sons Inc.; 2014. p. 1075. |
| 26. | Lyles MA, Marjorie A, Mitroff II. Organizational problem formulation: An empirical study. Administrative Sci Quarterly 1980;25:102-19. |
| 27. | Cucoranu IC, Parwani AV, Vepa S, Weinstein RS, Pantanowitz L. Digital pathology: A systematic evaluation of the patent landscape. J Pathol Inform 2014;5:16. [ PUBMED] [Full text] |
| 28. | Coudray N, Ocampo PS, Sakellaropoulos T, Narula N, Snuderl M, Fenyö D, et al. Classification and mutation prediction from non-small cell lung cancer histopathology images using deep learning. Nat Med 2018;24:1559-67. |
| 29. | Campanella G, Hanna MG, Geneslaw L, Miraflor A, Werneck Krauss Silva V, Busam KJ, et al. Clinical-grade computational pathology using weakly supervised deep learning on whole slide images. Nat Med 2019;25:1301-9. |
| 30. | Campanella G, Rajanna AR, Corsale L, Schüffler PJ, Yagi Y, Fuchs TJ. Towards machine learned quality control: A benchmark for sharpness quantification in digital pathology. Comput Med Imaging Graph 2018;65:142-51. |
| 31. | |
| 32. | |
| 33. | Hamilton PW, Bankhead P, Wang Y, Hutchinson R, Kieran D, McArt DG, et al. Digital pathology and image analysis in tissue biomarker research. Methods 2014;70:59-73. |
| 34. | Ho J, Ahlers SM, Stratman C, Aridor O, Pantanowitz L, Fine JL, et al. Can digital pathology result in cost savings? A financial projection for digital pathology implementation at a large integrated health care organization. J Pathol Inform 2014;5:33. |
| 35. | Isaacs M, Lennerz JK, Yates S, Clermont W, Rossi J, Pfeifer JD. Implementation of whole slide imaging in surgical pathology: A value added approach. J Pathol Inform 2011;2:39. [ PUBMED] [Full text] |
| 36. | Mills AM, Gradecki SE, Horton BJ, Blackwell R, Moskaluk CA, Mandell JW, et al. Diagnostic efficiency in digital pathology: A comparison of optical versus digital assessment in 510 surgical pathology cases. Am J Surg Pathol 2018;42:53-9. |
| 37. | Pantanowitz L, Dickinson K, Evans AJ, Hassell LA, Henricks WH, Lennerz JK, et al. American Telemedicine Association clinical guidelines for telepathology. J Pathol Inform 2014;5:39. [ PUBMED] [Full text] |
| 38. | Williams BJ, Bottoms D, Treanor D. Future-proofing pathology: The case for clinical adoption of digital pathology. J Clin Pathol 2017;70:1010-8. |
| 39. | Baidoshvili A, Bucur A, van Leeuwen J, van der Laak J, Kluin P, van Diest PJ. Evaluating the benefits of digital pathology implementation: Time savings in laboratory logistics. Histopathology 2018;73:784-94. |
| 40. | Fraggetta F, Garozzo S, Zannoni GF, Pantanowitz L, Rossi ED. Routine Digital Pathology Workflow: The Catania Experience. J Pathol Inform 2017;8:51. [ PUBMED] [Full text] |
| 41. | Hanna MG, Reuter VE, Hameed MR, Tan LK, Chiang S, Sigel C, et al. Whole slide imaging equivalency and efficiency study: Experience at a large academic center. Mod Pathol 2019;32:916-28. |
| 42. | Hanna MG, Reuter VE, Samboy J, England C, Corsale L, Fine SW, et al. Implementation of digital pathology offers clinical and operational increase in efficiency and cost savings. Arch Pathol Lab Med 2019;143:1545-55. |
| 43. | Sirintrapun SJ, Lopez AM. Telemedicine in cancer care. Am Soc Clin Oncol Educ Book 2018;38:540-5. |
| 44. | FDA. Laboratory Developed Tests. 2018; |
| 45. | Schreier J, Feeney R, Keeling P. Diagnostics reform and harmonization of clinical laboratory testing. J Mol Diagn 2019;21:737-45. |
| 46. | Zehnbauer B. Integration standardization and diagnostics oversight of laboratory testing. J Mol Diagn 2019;21:735-6. |
| 47. | Halling KC, Schrijver I, Persons DL. Test verification and validation for molecular diagnostic assays. Arch Pathol Lab Med 2012;136:11-3. |
| 48. | Clinical Laboratory Improvement Amendments of 1988. USA: Public Law; 1988. p. 100-578. |
| 49. | |
| 50. | Cucoranu IC, Parwani AV, Pantanowitz L. Digital whole slide imaging in cytology. Arch Pathol Lab Med 2014;138:300. |
| 51. | Zarella MD, Bowman D, Aeffner F, Farahani N, Xthona A, Absar SF, et al. A practical guide to whole slide imaging: A white paper from the digital pathology association. Arch Pathol Lab Med 2019;143:222-34. |
| 52. | Hartman DJ, Ahmad F, Ferris RL, Rimm DL, Pantanowitz L. Utility of CD8 score by automated quantitative image analysis in head and neck squamous cell carcinoma. Oral Oncol 2018;86:278-87. |
| 53. | Rashidi HH, Tran NK, Betts EV, Howell LP, Green R. Artificial intelligence and machine learning in pathology: The present landscape of supervised methods. Acad Pathol 2019;6:2374289519873088. |
| 54. | Herrmann MD, Clunie DA, Fedorov A, Doyle SW, Pieper S, Klepeis V, et al. Implementing the DICOM standard for digital pathology. J Pathol Inform 2018;9:37. [ PUBMED] [Full text] |
| 55. | D'Angelo R, Weiss R, Wolfe D, Chinnam R, Murat A, Gluesing J, et al. Facing the inevitable: being prepared for regulatory requirements for laboratory developed tests. Am J Clin Pathol 2018;149:484-98. |
| 56. | Genzen JR. Regulation of laboratory-developed tests. Am J Clin Pathol 2019;152:122-31. |
| 57. | Genzen JR, Mohlman JS, Lynch JL, Squires MW, Weiss RL. Laboratory-developed tests: A legislative and regulatory review. Clin Chem 2017;63:1575-84. |
| 58. | Kim AS, Bartley AN, Bridge JA, Kamel-Reid S, Lazar AJ, Lindeman NI, et al. Comparison of laboratory-developed tests and FDA-approved assays for BRAF, EGFR, and KRAS Testing. JAMA Oncol 2018;4:838-41. |
| 59. | Walker Q. Designing a standard of proof: The case for professional standards in next-generation sequencing laboratory-developed tests. J Law Biosci 2017;4:216-26. |
| 60. | Khullar D, Ohn JA, Trusheim M, Bach PB. Understanding the Rewards of Successful Drug Development-Thinking Inside the Box. N Engl J Med 2020;382:473-80. |
| 61. | Sim I. Mobile Devices and Health. N Engl J Med 2019;381:956-68. |
| 62. | Clunie D, Hosseinzadeh D, Wintell M, De Mena D, Lajara N, Garcia-Rojo M, et al. Digital imaging and communications in medicine whole slide imaging connectathon at digital pathology association pathology visions 2017. J Pathol Inform 2018;9:6. |
| 63. | |
| 64. | MDIC. Press Release: Patient Preference Collaboration; 2017. |
| 65. | |
| 66. | |
| 67. | |
| 68. | |
| 69. | |
| 70. | FOCR. ctDNA for Monitoring Treatment Response (ctMoniTR) Project; 2019. |
| 71. | Colling R, Pitman H, Oien K, Rajpoot N, Macklin P, CM-Path AI in Histopathology Working Group, et al. Artificial intelligence in digital pathology: A roadmap to routine use in clinical practice. J Pathol 2019;249:143-50. |
| 72. | Turner OC, Aeffner F, Bangari DS, High W, Knight B, Forest T, et al. Society of toxicologic pathology digital pathology and image analysis special interest group article*: Opinion on the application of artificial intelligence and machine learning to digital toxicologic pathology. Toxicol Pathol 2020;48:277-94. |
| 73. | |
| 74. | Research FoC. Pilot Project 2.0: Establishing the Utility of Real-World Endpoints; 2019. Available from: https://www.focr.org/rwe. [Last accessed on 2020 Jul 07]. |
| 75. | DPA. A Brief Explanation of the Differences Between the DPA and the Alliance; 2019. |
| 76. | Denkert C, von Minckwitz G, Brase JC, Sinn BV, Gade S, Kronenwett R, et al. Tumor-infiltrating lymphocytes and response to neoadjuvant chemotherapy with or without carboplatin in human epidermal growth factor receptor 2-positive and triple-negative primary breast cancers. J Clin Oncol 2015;33:983-91. |
| 77. | |
| 78. | |
| 79. | |
| 80. | |
| 81. | |
| 82. | |
| 83. | |
| 84. | |
| 85. | |
| 86. | |
| 87. | |
| 88. | |
| 89. | |
[Figure 1], [Figure 2], [Figure 3], [Figure 4]
[Table 1]
|
