|J Pathol Inform 2015,
Development of a semi-automated method for subspecialty case distribution and prediction of intraoperative consultations in surgical pathology
Raul S Gonzalez1, Daniel Long1, Omar Hameed2
1 Department of Pathology, Microbiology and Immunology, Vanderbilt University Medical Center, Nashville, TN, USA
2 Department of Pathology, Microbiology and Immunology; Department of Surgery; Vanderbilt Ingram Cancer Center, Vanderbilt University Medical Center, Nashville, TN, USA
|Date of Submission||09-Feb-2015|
|Date of Acceptance||22-May-2015|
|Date of Web Publication||29-Jun-2015|
Department of Pathology, Microbiology and Immunology; Department of Surgery; Vanderbilt Ingram Cancer Center, Vanderbilt University Medical Center, Nashville, TN
Source of Support: None, Conflict of Interest: None
| Abstract|| |
Background: In many surgical pathology laboratories, operating room schedules are prospectively reviewed to determine specimen distribution to different subspecialty services and to predict the number and nature of potential intraoperative consultations for which prior medical records and slides require review. At our institution, such schedules were manually converted into easily interpretable, surgical pathology-friendly reports to facilitate these activities. This conversion, however, was time-consuming and arguably a non-value-added activity. Objective: Our goal was to develop a semi-automated method of generating these reports that improved their readability while taking less time to perform than the manual method. Materials and Methods: A dynamic Microsoft Excel workbook was developed to automatically convert published operating room schedules into different tabular formats. Based on the surgical procedure descriptions in the schedule, a list of linked keywords and phrases was utilized to sort cases by subspecialty and to predict potential intraoperative consultations. After two trial-and-optimization cycles, the method was incorporated into standard practice. Results: The workbook distributed cases to appropriate subspecialties and accurately predicted intraoperative requests. Users indicated that they spent 1-2 h fewer per day on this activity than before, and team members preferred the formatting of the newer reports. Comparison of the manual and semi-automatic predictions showed that the mean daily difference in predicted versus actual intraoperative consultations underwent no statistically significant changes before and after implementation for most subspecialties. Conclusions: A well-designed, lean, and simple information technology solution to determine subspecialty case distribution and prediction of intraoperative consultations in surgical pathology is approximately as accurate as the gold standard manual method and requires less time and effort to generate.
Keywords: High reliability, information technology, intraoperative consultation, lean, surgical pathology
|How to cite this article:|
Gonzalez RS, Long D, Hameed O. Development of a semi-automated method for subspecialty case distribution and prediction of intraoperative consultations in surgical pathology. J Pathol Inform 2015;6:40
|How to cite this URL:|
Gonzalez RS, Long D, Hameed O. Development of a semi-automated method for subspecialty case distribution and prediction of intraoperative consultations in surgical pathology. J Pathol Inform [serial online] 2015 [cited 2019 Nov 19];6:40. Available from: http://www.jpathinformatics.org/text.asp?2015/6/1/40/159439
| Introduction|| |
Since the establishment of the frozen section procedure more than 100 years ago, interpretation of intraoperative consultations, including evaluation of gross specimens, touch preparations, and frozen sections, has become a key function of many Surgical Pathology Departments, in both academic centers and community hospitals. , Depending on the size of the department and the daily volume and complexity of the intraoperative consultations requested by surgeons, this service can require a significant amount of manpower and, therefore, demand much of the department's resources. As a result, many departments attempt to predict what surgical cases might result in a request for an intraoperative consultation to better anticipate workload and appropriately allocate resources. This allows prior patient medical history and pathology specimens to be reviewed before and/or concurrently with the intraoperative consultation in order to minimize errors, as well as to help ensure the presence of the appropriate staff for timely and accurate performance and interpretation of these consultations. Depending on the setup, such staffing may include (in addition to attending pathologists) residents, fellows, laboratory technicians and technologists, and pathologists' assistants.
Each day at our institution, a surgical pathology fellow downloads and reviews the next day's operating room schedule, noting which procedures are likely to require intraoperative consultation. Until recently, this task was done manually, with the fellow individually reviewing each case in the schedule and predicting, based on factors such as the type of procedure and operating surgeon, the likelihood of intraoperative consultation. Simultaneously, this fellow would assign each specimen-generating procedure to a surgical pathology subspecialty, with that subspecialty responsible for any material that resulted from the procedure. For example, the genitourinary pathology service would assume ownership of any specimens associated with a prostatectomy, whereas our gynecologic pathology service would manage specimens associated with a hysterectomy.
While some surgical procedures at our institution are virtually guaranteed to require frozen sections (such as margin assessments on a pancreatoduodenectomy), other procedures only variably generate a consultation (such as thyroidectomies for mass lesions), and some frozen sections are, of course, unpredictable (such as incidentally discovered lesions during routine, nononcologic surgeries). In addition, manually compiling the list required approximately 90-150 min/day, depending on the volume of scheduled surgeries: The software at our institution required the fellow to review a long list of daily surgical procedures, manually highlight each one potentially requiring intraoperative consultation and then copy and paste it into a separate document, and finally gather and record relevant clinical information about each patient and procedure. These factors make the action of compiling a list of potential intraoperative consultations burdensome and occasionally unhelpful, potentially painting it as a non-value-added activity.
Given that the operating room schedules at our institution were already computerized, we hypothesized that an automated evaluation of the daily schedule could better sort surgeries into their appropriate pathology subspecialty and, with appropriate programming, help predict which cases would require intraoperative consultation. With proper design and input, we felt such a lean mechanism could perform this task as well, or nearly as well, as the human surgical pathology fellow, in much less time. To that end, we developed, tested, and refined a semi-automated Microsoft ® Excel 2010 (Microsoft Corporation, Redmond, Washington, USA) workbook to perform this task. Details of this process are presented herein.
| Materials and Methods|| |
We developed a dynamic Excel workbook that, upon receiving an itemized list of one day's operating room schedule as input, automatically converts that schedule into different tabular formats. At our institution and others, canned text descriptions are typically used to describe surgical procedures [Figure 1]. As a result, a keyword list can be employed to search the schedule and categorize the surgical procedures, as keywords and phrases differ little from one case of a particular surgical procedure to the next. This keyword list can also be utilized to predict whether a given surgical case may require an intraoperative consultation.
|Figure 1: A screenshot of the daily online operating room schedule provided by the Department of Surgery at our institution. The information necessary to predict what organ system each procedure addresses and whether an intraoperative consultation may be required is present; however, reading a daily list of every procedure, and then making such determinations for each case, can be time - consuming. Sensitive identifying information has been censored|
Click here to view
The Excel workbook functions as follows: First, the user copies to the clipboard the entire daily list of surgical procedures [Figure 1], rather than reading and selecting individual procedures. This list is then pasted into a blank worksheet in the Excel file. Since the format of the surgical schedule is fixed, the workbook then utilizes the INDEX Excel function to isolate pertinent information (i.e., patient identifiers, surgery descriptions, start times, etc.) for each surgical procedure within the pasted text. Next, a series of array formulas based around the FIND Excel function search for specific keywords and phrases within each surgery description. This list of keywords is maintained in a separate worksheet. Each keyword is tied to a pathology subspecialty service and is assigned a weighted score. For example, the term "colectomy" is given a stronger weighted score for the gastrointestinal pathology service than "exploratory laparotomy." For each scheduled surgery, the total weighted scores from this keyword search are calculated, and a series of formula-based logic tests assign each procedure to a pathologic subspecialty based on these scores. A similar keyword search is performed to select cases that have a high probability of intraoperative consultation. Finally, the data is organized into a simple, user-friendly format [Figure 2]. Multiple Visual Basic macros were created that allow the user to interact with a single worksheet user interface, while data is manipulated in separate worksheets within the same workbook. These macros allow the user to seamlessly make various adjustments to the schedule, such as reassigning a case to another pathologic subspecialty or adding a case to the list of predicted intraoperative consultations.
|Figure 2: The surgery schedule in Figure 1, inputted into and then formatted by our Excel workbook. The workbook performs the task of sorting each surgery into an organ system (color - coded by row background), indicating which surgical pathology subspecialty should claim any specimens generated by that surgery. Furthermore, a keyword search - based algorithm automatically predicts whether the procedure may generate an intraoperative consultation (indicated by the "MAYBE" highlighted in the rightmost column). Sensitive identifying information has been censored|
Click here to view
Following two optimization cycles for the intraoperative consultation keyword list (as discussed below), the workbook was formally implemented on July 1, 2013, after incoming surgical pathology fellows were instructed on proper usage of the workbook, thus replacing the manual method of schedule analysis with this semi-automated method.
As part of an ongoing quality initiative, we track how many such consultations per subspecialty are predicted for each day, and how many are actually performed. Utilizing such data from the final 6 months of manual prediction and the first 6 months of semi-automated prediction allowed us to compare the two methods. We calculated the number of cases by which the predicted utilization differed from actuality, for each subspecialty on each day. For instance, predicting two consultations, but receiving four was considered to be inaccurate by two cases, as was predicting four and only receiving two.
Data were available for 118 working days using the manual prediction method and 96 working days using the semi-automated prediction method (i.e., data were not available for each and every working day within either of the two 6-month periods). For each of these two sets, we calculated the mean difference between predicted and actual intraoperative consultations for each subspecialty, as well as the standard deviation of the data distribution. The two data sets were then compared for each subspecialty using an unpaired Student's t-test, also calculated within Excel. Since this approach simply calculated the difference between prediction and reality for each day's number of intraoperative consultations, a distinction between false-negative estimations (underprediction) and false-positive estimations (overprediction) could not be performed.
| Results|| |
In designing the Excel workbook, we first generated two rough sets of keywords and phrases; the first was designed to sort surgical procedures into subspecialties, and the second [Table 1] to predict whether an intraoperative consultation might be requested. For instance, a surgical procedure described as a "laryngectomy" would be sorted into our head/neck/lung subspecialty and marked as likely to require intraoperative consultation, as these procedures typically generate multiple frozen sections to evaluate resection margins for in situ or invasive carcinoma.
|Table 1: Keyword list for semi-automated prediction of intraoperative consultations|
Click here to view
The workbook was designed so that, after interpreting and formatting the input, the user could tabulate its results in many ways: The user can view every surgery in order, color-coded by specialty; the user can view and print lists for each subspecialty's surgery load for the day; and the user can print a list of the cases predicted to require intraoperative consultation for the day. Additional functionalities were also designed; for instance, since the published daily operating room schedule lists a presumed start and end time for all procedures, a graph plotting the relative utilization of the operating rooms over the course of the day can be generated.
After confirming that the workbook properly accepted its input and processed it into interpretable output, we implemented two optimization cycles. In each, the daily intraoperative consultation predictions were generated both manually (the method in place at the time) and using the workbook (the new method) for 1 week. After the week was finished, the two methods were compared. Each incorrect judgment made by the workbook (both false-positive predictions and false-negative omissions) was individually scrutinized to determine the reason for the error, and the keyword list was then amended to include keywords and phrases often seen in descriptions of "overlooked" surgeries and to remove keywords and phrases that routinely generated false-positive predictions.
For optimization of the prediction algorithm, we decided to focus more on reducing the number of false-negative cases than false-positive ones. In other words, sensitivity was given more weight than specificity. This decision was based on our experience that underestimation of workload strained resources more significantly than overestimation.
In essence, the workbook has three tasks: Convert the provided operating room schedule into a format easily interpreted and navigated by a surgical pathologist; sort procedure-generated specimens by subspecialty; and predict the need for intraoperative consultation. As discussed above, the workbook performs its first task perfectly. It also performs its second task extremely well; given the discrete and unequivocal nature of most surgical procedures, the workbook almost always sorted each operation into the surgical pathology subspecialty that was most appropriate to receive any resulting specimens. Errors in this process usually occurred only when an operating room visit was intended to excise tissue typically assigned to more than one subspecialty - for example, a nephrectomy combined with a cholecystectomy. This task is not truly critical, as "misassigning" a specimen can easily be noticed and corrected by a human processing the specimen upon its receipt in surgical pathology. Still, we found that having a general sense of how busy each subspecialty would be for intraoperative consultation and specimen grossing purposes was welcome information to the surgical pathology team.
The workbook's third task is its most difficult, yet also it is most beneficial. By relying on the carefully optimized list of linked keywords and phrases, the workbook performed well in predicting what surgical procedures might result in intraoperative consultations. Comparison of 6 months' worth of data for the manually and semi-automatically compiled predictions showed the mean daily difference in predicted versus actual intraoperative consultations performed underwent no statistically significant changes before and after implementation for most of the subspecialties [Table 2]. For instance, the manual method's prediction for the number of gastrointestinal intraoperative consultations differed from the actual number of consultations performed by an average of 0.74 cases/day, whereas the semi-automated method's average difference was 0.82 cases/day (P = 0.43). Of note, the head/neck/lung and breast/endocrine subspecialties were combined in this analysis because intraoperative consultations for thyroid and parathyroid specimens switched from the former subspecialty to the latter during the time period, meaning that analyzing the two subspecialties separately would introduce a confounding variable into both results. Since the goal for each subspecialty was to have the semi-automated method perform similarly to the gold standard of manual prediction, a high P value (indicating no significant difference between the two values) was desired. As stated above, the workbook was designed for sensitivity rather than specificity, making it unsurprising that the daily average "inaccuracy" was larger for the semi-automated method than for the manual method.
|Table 2: Comparison of manual and semi - automated methods of intraoperative consultation prediction|
Click here to view
Discussions with the surgical pathology fellows who helped test the in-development workbook while performing manual prediction indicated that the workbook method was strongly preferred for three main reasons. First, while the manual method had typically required from 90 to 150 min to generate per day, depending on surgery load and fellow speed, the semi-automated method only took approximately 30-45 min/day, which primarily involved gathering clinical history and pulling slides for the cases predicted to require intraoperative consultation. Second, the manual method generated a several-page document, printed in the format seen in [Figure 1]; copies of this document were circulated among the surgical pathology team with handwritten notes indicating which procedures might require an intraoperative consultation. The semi-automated method, in contrast, generated a succinct and cleanly formatted list of the day's presumed intraoperative consultations on one or two sheets of paper, facilitating improved dissemination of this information. Third, the workbook method was less mentally taxing, allowing the fellow to concentrate on the cases predicted to generate a consultation rather than evaluating every single scheduled procedure each day.
| Conclusions|| |
The task of sorting through a daily list of surgical procedures in order to predict which might require a frozen section generates results that are important and useful, as relevant patient data, including prior specimen slides, can be obtained prior to surgery, and the surgical pathology team as a whole can gain a concept of how the day's workload may progress. However, the process of compiling this information can be onerous and often time-consuming.
Our project shows that subspecialty case distribution and prediction of intraoperative consultations in surgical pathology can be performed utilizing a well-designed, lean, and simple information technology solution, such as a programmed Excel workbook. In addition, we have shown that it is roughly as accurate as the "gold standard" manual method and requires less time to generate.
The key determinant in designing a successful prediction algorithm was choosing which search terms would indicate that a surgical procedure might require an intraoperative consultation. Some terms merited obvious inclusion; for instance, a "Whipple procedure" (pancreatoduodenectomy) always generates frozen sections for surgical margins, and at our institution, a mastectomy with sentinel lymph node sampling (detected by the linked keywords "mastectomy" and "lymph") always requires touch-prep analysis of the sentinel nodes for metastatic tumor. We also decided to include cases that variably resulted in intraoperative consultation. One example is a "parathyroidectomy," which would usually, but not always result in a frozen section to confirm that parathyroid tissue was excised. Still other terms were murky; a surgeon performing a "radical resection, tumor, soft tissue" would sometimes request an intraoperative evaluation of surgical margins, but just as often would not.
The final list [Table 1] of search terms underwent two rounds of trial-and-error revision before being ready for official implementation. During this process, we discovered several interesting quirks in the language the surgery-generated operating room schedule employed to describe procedures. For example, a "cystectomy" could describe resection of a urinary bladder (which usually underwent frozen section evaluation of surgical margins) or of an ovarian cyst (which only sometimes required an intraoperative consultation, based on the preoperative diagnosis). To circumvent such situations, our final list of search words employs several linked terms, such as "cystectomy" AND "bladder," to decrease the flagging of surgeries for benign lesions. We initially made attempts to include operating surgeon in the keywords, assuming that some surgeons were more likely than other to request an intraoperative consultation during a given procedure, but our preliminary results did not warrant implementing this approach in our final list.
Both the manual and the semi-automated methods of generating predictive lists resulted in both false-positive and false-negative results. Unfortunately, these are simply unavoidable. As mentioned above, some surgical procedures (such as parathyroidectomies) can be assumed to usually require intraoperative consultation, but this occasionally does not occur, resulting in a false-positive prediction. In addition, unexpected intraoperative findings, such as an unsuspected mass, can generate an unpredicted, false-negative frozen section request. False-negative predictions can also result from the unfortunate, but well-known phenomenon of inappropriate frozen section requests, like those wherein the pathologic finding will not alter the course of the surgery. ,
As with any software-based technology solution, updates are required as external factors change. As time progresses, the subset of procedures at our institution that yield intraoperative consultations will likely change, as the surgeon population changes and standard operating procedures are altered. Other external factors may affect the accuracy of the workbook as well. In fact, such an event has already occurred during the relatively short lifespan of our workbook. An initial comparison between the manual and semi-automated methods was performed 2 months after the workbook was formally implemented, and at this time, the workbook's daily average prediction error for our gynecologic pathology subspecialty was 0.39, compared to the manual prediction error of 0.38 (P = 0.94). After 6 months, however, the workbook's average error was 0.66 [Table 2] (P = 0.0059). Analysis of the situation determined that the canned language used by the surgery department to describe each gynecologic surgical procedure underwent revision during this time period, rendering the keyword list nonoptimal. This underscores the fact that periodic reevaluation and revision of the workbook's keyword list will need to continue indefinitely in order to maintain accuracy.
Even including the problematic gynecologic subspecialty data, the manual and semi-automated methods had average error rates that differed by less than one case per day (2.69 vs. 3.21), and this difference showed a P = 0.098, which was not quite statistically significant compared to the conventional cutoff value of <0.05.
Our particular solution is highly customized to our institution, especially considering that the software used to display the daily list of surgical procedures is homegrown, and the workbook would not work in its current form at another institution, given the differences in each institution's information technology infrastructure, language for describing surgical procedures, and patterns of intraoperative consultation utilization. The workbook would have to be redesigned to accept, parse, and search the local operating room schedule, and the keyword lists would need to be modified for accurate prediction of intraoperative consultations and selection of pathology subspecialty assignment. In addition, depending on an institution's method of manually generating a surgical procedure list (which is dependent on the software in place), the amount of time saved daily by the semi-automated method may vary.
Despite these limitations, our Excel workbook concept offers a high-tech yet low-cost solution ideally situated to interact and fit in with the growing use of electronic information to manage workflow in surgical pathology departments. It predicts intraoperative consultation requests with reasonable accuracy, allowing for relatively rapid determination of the next day's workload and workflow within the frozen section suite. In addition, the workbook might also facilitate such aims as better meeting the goal of returning frozen section results within 20 min.  Finally, better prediction of cases requiring frozen section may also be of additional value in facilities requiring the use of telepathology services to assist in accurate interpretation of difficult cases. 
In summary, we have shown that a well-designed, lean, and simple information technology solution to determine subspecialty case distribution and prediction of intraoperative consultations in surgical pathology is approximately as accurate as the gold standard manual method and requires less time and effort to generate.
Financial Support and Sponsorship
Conflict of Interest
There are no conflict of interest.
| References|| |
Gal AA. The centennial anniversary of the frozen section technique at the Mayo Clinic. Arch Pathol Lab Med 2005;129:1532-5.
Taxy JB. Frozen section and the surgical pathologist: A point of view. Arch Pathol Lab Med 2009;133:1135-8.
Weiss SW, Willis J, Jansen J, Goldblum J, Greenfield L. Frozen section consultation. Utilization patterns and knowledge base of surgical faculty at a university hospital. Am J Clin Pathol 1995;104:294-8.
Jaafar H. Intra-operative frozen section consultation: Concepts, applications and limitations. Malays J Med Sci 2006;13:4-12.
Novis DA, Zarbo RJ. Interinstitutional comparison of frozen section turnaround time. A College of American Pathologists Q-Probes study of 32868 frozen sections in 700 hospitals. Arch Pathol Lab Med 1997;121:559-67.
Winokur TS, McClellan S, Siegal GP, Redden D, Gore P, Lazenby A, et al.
A prospective trial of telepathology for intraoperative consultation (frozen sections). Hum Pathol 2000;31:781-5.
[Figure 1], [Figure 2]
[Table 1], [Table 2]