Journal of Pathology Informatics Journal of Pathology Informatics
Contact us | Home | Login   |  Users Online: 443  Print this pageEmail this pageSmall font sizeDefault font sizeIncrease font size 




 
Table of Contents    
RESEARCH ARTICLE
J Pathol Inform 2012,  3:26

Utilization and utility of clinical laboratory reports with graphical elements


1 Department of Pathology, University of Utah School of Medicine, Salt Lake City, Utah, USA
2 ARUP Institute for Clinical and Experimental Pathology, Salt Lake City, Utah, USA
3 Department of Pathology and Biomedical Informatics, University of Utah School of Medicine; ARUP Institute for Clinical and Experimental Pathology, Salt Lake City, Utah, USA

Date of Submission26-Apr-2012
Date of Acceptance21-May-2012
Date of Web Publication25-Aug-2012

Correspondence Address:
Brian H Shirts
Department of Pathology, University of Utah School of Medicine, Salt Lake City, Utah
USA
Login to access the Email id

Source of Support: None, Conflict of Interest: None


DOI: 10.4103/2153-3539.100145

Rights and Permissions
   Abstract 

Background: Graphical reports that contain charts, images, and tables have potential to convey information more effectively than text-based reports; however, studies have not measured how much clinicians value such features. We sought to identify factors that might influence the utilization of reports with graphical elements postulating that this is a surrogate for relative clinical utility of these graphical elements. Materials and Methods: We implemented a pilot project at ARUP laboratories to develop online enhanced laboratory test reports that contained graphical elements. We monitored on-demand clinician access to reports generated for 48 reportable tests over 22 months. We evaluated utilization of reports with graphical elements by clinicians at all institutions that use ARUP as a reference laboratory using descriptive statistics, regression, and meta-analysis tools to evaluate groups of similar test reports. Results: Median download rate by test was 8.6% with high heterogeneity in download rates between tests. Test reports with additional graphical elements were not necessarily downloaded more often than reports without these elements. Recently implemented tests and tests reporting abnormal results were associated with higher download rates (P < 0.01). Higher volume tests were associated with lower download rates (P = 0.03). Conclusions: In select cases graphical information may be clinically useful, particularly for less frequently ordered tests and in on reports of abnormal results. The utilization data presented could be used as a reference point for other laboratories planning on implementing graphical reporting. However, between-test heterogeneity was high and in many cases graphical elements may add little clinical utility, particularly if these merely reinforce information already contained in text based reports.

Keywords: Clinical laboratory information systems utilization, enhanced reports, figures, LIS interface, multimedia utilization, pictures, tables


How to cite this article:
Shirts BH, Larsen N, Jackson BR. Utilization and utility of clinical laboratory reports with graphical elements. J Pathol Inform 2012;3:26

How to cite this URL:
Shirts BH, Larsen N, Jackson BR. Utilization and utility of clinical laboratory reports with graphical elements. J Pathol Inform [serial online] 2012 [cited 2019 Dec 10];3:26. Available from: http://www.jpathinformatics.org/text.asp?2012/3/1/26/100145


   Introduction Top


Several studies have shown that formatting can improve the clarity and clinical utility of pathology reports. [1],[2],[3] As the complexity of laboratory testing increases there is a growing need for well designed, intuitive reports that convey complex information to clinicians. Graphical trending of serial laboratory measures is well established in laboratory information systems and electronic medical records and its utility has been widely acknowledged; [4],[5],[6] however, additional graphical elements are being incorporated into both paper and electronic test result reports such as specimen photographs, karyotype images, flow cytometry plots, gel images, charts of ancillary information, links to outside web-based information, and even interactive calculators. [7],[8],[9] Although graphical reports have potential to convey information more effectively than standard text-based reports, studies have not measured how much clinicians value novel graphical features in laboratory reports. [9],[10] Despite the lack of data on their clinical utility; color, charts, images, and tables have become standard for reports provided by small specialty laboratories that circumvent standard interfaces between laboratory information systems and electronic medical records. [11] High-quality, color reports are often used as a selling point, although in some cases reports used as marketing tools overstate the clinical utility of novel testing modalities and actual test performance may be equivalent to or worse than standard testing. [12],[13] Local and regional reference laboratories may need to implement higher quality reporting to compete; however, it is not clear if extra images and charts add clinical value, or if they are merely distractions that look nice but contribute little to clinical care. Understanding factors that influence utilization of clinical information is important to laboratories because developing and implementing each individual test report requires time and effort from laboratory directors, managers, and informatics personal in addition to infrastructure development to produce and maintain reporting systems. We sought to identify factors that might influence the utilization of reports with graphical elements beyond simple trending information postulating that this may be a surrogate for relative clinical utility of these graphical elements.


   Materials and Methods Top


ARUP laboratories, a national reference laboratory affiliated with the University of Utah, established pilot projects to test an enhanced electronic laboratory reporting system (EELRS) in November 2009. This was implemented as a method of providing clinicians graphical elements that cannot be transmitted or displayed with current interface systems, as a means to integrate information from multiple test reports, and as a location for links to sources of additional information. [8] This system supplements and does not replace the standard mechanisms for delivering test results. Each time a test result is verified within the laboratory information system (LIS), a pdf-format chart is generated and stored on a server. A URL (http://www.aruplab.com/Lab-Reports) along with chart id number and password are provided as a footnote to the LIS-version test result. This allows all individuals with access to standard test reports to access enhanced versions containing the same test results with graphical elements via web download. Protected health information contained within reports is stored on secure servers, and can only be accessed by those that have access to the password transmitted with reports via current interfaces. Login information linked to specific test reports allowed us to track how often these reports were accessed. We evaluated utilization of enhanced electronic laboratory reports from institutions across the United States. We evaluated instances of access to enhanced reports viewed between November 2009 and September 2011 to determine if there were characteristics of the result presentation or result content that determined how frequently clinicians accessed enhanced reports. This study was reviewed and deemed human subjects exempt by University of Utah Institutional Review Board.

Enhanced reports delivered for EELRS have been created and implemented for 62 different tests. New reports were implemented throughout the study period with 28 reports implemented in 2009, 15 implemented in 2010, and 19 implemented in 2011. For analysis, we only included tests which had been ordered at least 50 times in the study period, leaving 48 test reports: 27 implemented in 2009, 13 implemented in 2010, and 8 implemented in 2011 [Table 1]. We categorized these into groups of reports on similar tests. There were five genomic microarray tests, six maternal screening tests, and 24 cytogenetic/FISH tests. We further divided the cytogenetic/FISH tests into four subgroups: seven constitutional cytogenetic tests, four constitutional FISH panels, four oncology cytogenetic tests, and nine oncology FISH panels. 13 individual tests did not lend themselves to any obvious groupings. Beyond enabling complex reports that are not interface compatible, electronic delivery of report documents can eliminate the need to mail hard-copy forms to clients. For several tests, specifically cytogenetics tests, enhanced reports eliminated hard-copy mailing. Other enhanced reports were created specifically for online graphical presentation.
Table 1: Overall download rates and download rates of tests with abnormal results for enhanced reports generated at ARUP laboratories

Click here to view


We tracked the total number of enhanced reports generated and the reports accessed at the individual report level for all institutions that ordered relevant tests throughout the study period. We only included the initial access of a report in our analysis, so multiple visits to the same report were not part of our analysis. We initially used descriptive statistical methods (median download rates for individual tests and groups) to explore the proportion of generated reports accessed for each individual test and to determine the average access rate for each test group.

It appeared that tests brought online more recently were downloaded more frequently than older tests and that higher volume tests were downloaded less often than low volume tests, so we used linear regression to test if these factors influenced download rates using months from implementation as one variable and the natural log of the test volume as another continuous variable. We then used the Freeman-Tukey double arcsine transformation method for meta-analysis of proportional data as implemented by the metaprop function in the R-meta package to synthesize data, determine average download rates within group, and evaluate within-group heterogeneity. Data on chart content that could be classified as normal or abnormal was available for 31 of the 48 tests, so we also evaluated whether report content influenced download rate by comparing download rates for reports with abnormal results to overall download rates for these tests.

Enhanced reports for the 13 FISH tests were initially implemented with the intention of adding images of cells with representative marker patterns on the enhanced reports; however, due to technical issues these images were not added to the online reports during the study period. Thus, these reports presented the same information presented in traditional reports transmitted over electronic interfaces with only minor differences in formatting. So, we used two groups of FISH test reports (oncology FISH and constitutional FISH) as controls and compared these groups with other groups of tests. We used 95% confidence intervals from meta-analysis to ascertain whether download rate for groups differed significantly and differed from "negative control" FISH testing download rates.

We evaluated categories of graphics presented in enhanced reports in order to better understand between-test heterogeneity in download rates. We thought that different types of graphics might explain why certain reports were downloaded more than others. We present several examples of enhanced reports to illustrate examples with high and low download rates.


   Results Top


During the study period starting at the implementation of EELRS on November 9, 2009 and ending September 30, 2011 around 174,170 enhanced reports were generated for the 48 reportable tests evaluated [Table 1]. The majority of enhanced reports generated were maternal screening reports (66.4%), after this the next most represented group was oncology cytogenetics (6.2% of reports), followed by constitutional cytogenetics reports (5.6%), constitutional fish panel reports (2.5%), oncology fish (2.1%), and genomic microarray reports (1.4%). 15.8% of generated reports were for the 13 individual tests which not lend themselves to any obvious groupings [Table 1]. For all generated reports 8,947 were downloaded for a total download rate of 5.1%. However, the highest volume tests had low download rates, so the median test download rate was 8.6% and several tests had much higher download rates so the estimated average test download rate was 13.3% [Table 2].
Table 2: Summary statistics for enhanced report download rates by category of laboratory tests

Click here to view


Both time from implementation and number of total tests ordered were significantly associated with download rate. More recently implemented tests were associated with higher download rates, with download rates declining by 5% to 10% in the first six months after a new report is implemented (P < 0.001 for trend). Higher volume tests were downloaded at a rate lower than low volume tests, with download rates declining by about 2% per order of magnitude of test volume (P = 0.03). Reports indicating abnormal test results were also downloaded on average 8% more often than reports indicating normal results (P < 0.01).

All groups of tests except the Maternal Serum Screening group had significantly higher download rates than the Oncology FISH control group (P < 0.05). However, only the "other" group had a significantly higher download rate than the Constitutional FISH control group (P < 0.05). There was striking overall heterogeneity in download rates between tests (P < 0.001 for heterogeneity). Even when we examined tests in the same group, there was substantial within group heterogeneity (P < 0.001 for within group analysis of heterogeneity for all groups except Oncology Cytogenetics, P = 0.198, see [Table 2]). This high level of within group heterogeneity tempers any conclusions that might be drawn from between group comparisons. Although download rates differed from test to test and group to group, there were no clear factors beyond those noted above that correlated with download rate. We present and discuss selected examples of representative reports below.

Specific Examples of Report Groups

Maternal serum screening reports


Maternal serum screening tests were the highest volume tests with enhanced reports. The enhanced report graphically displays likelihood ratios compared to decision cutoffs [Figure 1]. This group of reports had among the lowest download rates (4.7% average by meta-analysis, [Table 2]). A positive or abnormal serum screen was associated with a substantially higher report download rate (13.2%). The most frequently ordered test was the maternal serum screen with AFP, hCG, EST, INH (quad screen) which had an overall download rate of 1.1%, and a download rate of 5.0% for tests reporting positive screening.
Figure 1: Example enhanced report for maternal serum screening

Click here to view


Fish reports

PDF reports for FISH testing displayed the standard information that would be transmitted over an electronic interface with standard formatting and a color logo [Figure 2]. Despite this lack of additional information, there was substantial heterogeneity in download rates for FISH reports (P for heterogeneity < 0.001); with rates varying between 23.4% and 0.5% [Table 1] and [Table 2]. Reports on FISH for suspected malignancies were downloaded less than FISH reports for constitutional abnormalities (4.6% and 13.8% respectively). Overall oncology FISH reports with abnormal results were downloaded only slightly more often than those with normal results (7.1% and 4.6% respectively), and reports with abnormal constitutional FISH results were downloaded more often than reports with normal constitutional FISH findings (19.2% and 13.8% respectively), but neither of these differences were significant.
Figure 2: Example enhanced report for FISH testing

Click here to view


Cytogenetic test reports

In the past, images of karyotypes were mailed or faxed to clinicians. Enhanced cytogenetics test reports replaced paper images of karyotypes with PDF reports accessible online [Figure 3]. Cytogenetic test results for constitutional chromosomal abnormalities were downloaded as often as FISH reports for constitutional abnormalities (14.5% and 13.8% respectively). Although cytogenetic reports with karyotypes for oncology testing were downloaded more often than similar FISH reports for oncology testing (14.7% and 4.6% respectively), download rates for abnormal tests were not significantly higher than overall download rates in this group [Table 2].
Figure 3: Example enhanced report for cytogenetic testing

Click here to view


Drug screening reports

Several ungrouped tests were ordered most frequently [Table 1]. Among those with the highest download rates were drug screening test reports that presented information on metabolic pathways [Figure 4], BRC-ABL testing reports showing historical trend information, and calculi analysis reports with photographs of the calculi.
Figure 4: Example enhanced report for opiate confirmatory testing

Click here to view


Additional example reports for several tests evaluated in this manuscript can be found at http://www.aruplab.com/AboutARUP/PressRoom/Articles_LandingPages/enhanced_laboratory_reporting.jsp.


   Discussion Top


Understanding factors that influence utilization of graphical elements on laboratory reports can facilitate efficient and effective communication of information. [10] Our study attempts to determine the potential clinical importance of graphical information by using report access as a surrogate for report utility. We believe utilization is a reasonable, objective surrogate for utility; however, we acknowledge that there are major limitations to this assertion. Our assumption could be confounded by several common practices such as office staff downloading all enhanced reports or instructions for access being lost, hidden, or difficult to find because of interface formatting. Furthermore, our study evaluated utilization across many institutions that use AURP as a reference laboratory so much of the heterogeneity in utilization may be due to institution specific practices. As with any measure of subjective value, utility is context dependent and difficult to measure accurately. However, to our knowledge, there are no other published studies that examine the utility or utilization of graphical report elements in the clinical laboratory setting, beyond reports that evaluate the utility of trending information for numeric tests, suggesting that these measures of utility and utilization could serve as a useful baseline for other laboratories planning on implementing graphical reporting. We have received comments from clinicians that they would like to know what additional benefit is provided in the enhanced report so they can determine whether it is worthwhile to download the report, suggesting that clinicians are thinking about laboratory information in a way that is consistent with utilization being a surrogate for clinical utility.

Our experience with 48 custom build reports, as presented here, shows that custom report access rates were often low, but varied widely from test to test. Despite this heterogeneity, we observed several significant trends in download rates that might shed light on clinician behavior and help laboratory medical directors evaluate how often the addition of a graphical element might be clinically useful. Recently implemented reports were downloaded more often than older reports, reports for more frequently ordered tests were downloaded less than reports for more frequently ordered tests, and reports with abnormal results were downloaded more often than reports with normal results. Each of these observations may illustrate clinical learning patterns. Many clinicians are curious about new information, and once they have ascertained what is in a report they do not take the time to access it again if it is unlikely to add value. However, knowing that the information is available, clinicians may then access it again when they come across an abnormal or unexpected result. This suggests that in most cases implementing graphical reports for novel tests may be better received than adding graphical elements to existing test reports.

Download rates for the majority of individual reports generated were below 10% suggesting that for most clinicians formatting and color images are not important enough to justify the time necessary to download the reports. On the other hand, several reports were accessed at much higher rates suggesting that there are instances where graphical elements on reports present valuable information that may be impossible to convey over current interfaces. Also, the fact that reports associated with positive or abnormal tests are accessed more than negative reports suggests that the need for added information may be situational and case specific.

In conclusion, our analysis suggests that color images and other graphical elements in many cases do not enhance the utility of clinical laboratory reports. However, in some cases additional graphical information may be useful to clinicians, particularly for less frequently ordered tests where the interpretation of text results may not be as apparent from text base reports. The utilization data presented could be used as a reference point for estimating future utilization rates for laboratories planning on implementing graphical reporting. However, substantial observed heterogeneity suggests laboratory directors should not be surprised when graphical elements are not received as they expected. Laboratory directors should also consider carefully the expected clinical benefit of information display methods before redesigning reports in the face of presumed market pressure. In addition, non-laboratory clinicians could be more aware of what information from laboratory testing they are using for clinical decisions and base their choice of laboratory testing primarily on the quality of the information rather than the presentation of this information.


   Acknowledgements Top


The authors would like to thank Becky Chestnut for her assistance in technical writing and editing this manuscript.

 
   References Top

1.Powsner SM, Costa J, Homer RJ. Clinicians are from mars and pathologists are from Venus. Arch Pathol Lab Med 2000;124:1040-6.  Back to cited text no. 1
[PUBMED]    
2.Aumann K, Amann D, Gumpp V, Hauschke D, Kayser G, May AM, et al. Template-based synoptic reports improve the quality of pathology reports of prostatectomy specimens. Histopathology 2012;60:634-44.  Back to cited text no. 2
    
3.Valenstein PN. Formatting pathology reports: Applying four design principles to improve communication and patient safety. Arch Pathol Lab Med 2008;132:84-94.  Back to cited text no. 3
[PUBMED]    
4.Connelly DP, Lasky LC, Keller RM, Morrison DS. A system for graphical display of clinical laboratory data. Am J Clin Pathol 1982;78:729-37.  Back to cited text no. 4
[PUBMED]    
5.Keller D, Schaller WJ, Wong JS, de Groen PC. World-Wide Web-based graphical user interfaces for laboratory data. Methods Inf Med 2002;41:411-3.  Back to cited text no. 5
[PUBMED]    
6.Henry JB, Kelly KC. Comprehensive graphic-based display of clinical pathology laboratory data. Am J Clin Pathol 2003;119:330-6.  Back to cited text no. 6
[PUBMED]    
7.Lowe HJ, Buchanan BG, Cooper GF, Kaplan B, Vries JK. Image engine: an integrated multimedia clinical information system. Medinfo 1995;8:421-5.  Back to cited text no. 7
[PUBMED]    
8.Shirts BH, Gundlapalli AV, Jackson B. Pilot study of linking Web-based supplemental interpretive information to laboratory test reports. Am J Clin Pathol 2009;132:818-23.  Back to cited text no. 8
[PUBMED]    
9.Bauer DT, Guerlain S, Brown PJ. The design and evaluation of a graphical display for laboratory data. J Am Med Inform Assoc 2010;17:416-24.  Back to cited text no. 9
[PUBMED]    
10.Tufte ER. The visual display of quantitative information. Cheshire, Conneticut: Graphics Press; 2001.  Back to cited text no. 10
    
11.Myers ER, Havrilesky LJ, Kulasingam SL, Sanders GD, Cline KE, Gray RN, et al. Genomic tests for ovarian cancer detection and management. Evid Rep Technol Assess (Full Rep) 2006:1-100.  Back to cited text no. 11
    
12.Benor S, Russell GH, Silver M, Israel EJ, Yuan Q, Winter HS. Shortcomings of the inflammatory bowel disease Serology 7 panel. Pediatrics 2010;125:1230-6.  Back to cited text no. 12
[PUBMED]    
13.Lippi G, Favaloro EJ, Plebani M. Direct-to-consumer testing: More risks than opportunities. Int J Clin Pract 2011;65:1221-9.  Back to cited text no. 13
[PUBMED]    


    Figures

  [Figure 1], [Figure 2], [Figure 3], [Figure 4]
 
 
    Tables

  [Table 1], [Table 2]


This article has been cited by
1 Hereditary cancer gene panel test reports: wide heterogeneity suggests need for standardization
Sukh Makhnoon,Brian H Shirts,Deborah J Bowen,Stephanie M Fullerton
GENETICS in MEDICINE. 2018;
[Pubmed] | [DOI]
2 Specialized Laboratory Information Systems
Bryan Dangott
Clinics in Laboratory Medicine. 2016; 36(1): 41
[Pubmed] | [DOI]
3 Promoting improved utilization of laboratory testing through changes in an electronic medical record: experience at an academic medical center
Matthew D Krasowski,Deborah Chudzik,Anna Dolezal,Bryan Steussy,Michael P Gailey,Benjamin Koch,Sara B Kilborn,Benjamin W Darbro,Carolyn D Rysgaard,Julia A Klesney-Tait
BMC Medical Informatics and Decision Making. 2015; 15(1): 11
[Pubmed] | [DOI]
4 Stand-alone Laboratory Information Systems Versus Laboratory Modules Incorporated in the Electronic Health Record
John H. Sinard,William J. Castellani,Myra L. Wilkerson,Walter H. Henricks
Archives of Pathology & Laboratory Medicine. 2015; 139(3): 311
[Pubmed] | [DOI]
5 Specialized Laboratory Information Systems
Bryan Dangott
Surgical Pathology Clinics. 2015; 8(2): 145
[Pubmed] | [DOI]
6 Developing patient-friendly genetic and genomic test reports: formats to promote patient engagement and understanding
Susanne B Haga,Rachel Mills,Kathryn I Pollak,Catherine Rehder,Adam H Buchanan,Isaac M Lipkus,Jennifer H Crow,Michael Datto
Genome Medicine. 2014; 6(7)
[Pubmed] | [DOI]



 

 
Top
  

    

 
  Search
 
   Browse articles
  
    Similar in PUBMED
   Search Pubmed for
   Search in Google Scholar for
 Related articles
    Access Statistics
    Email Alert *
    Add to My List *
* Registration required (free)  

 
  In this article
    Abstract
   Introduction
    Materials and Me...
   Results
   Discussion
   Acknowledgements
    References
    Article Figures
    Article Tables

 Article Access Statistics
    Viewed10298    
    Printed115    
    Emailed1    
    PDF Downloaded730    
    Comments [Add]    
    Cited by others 6    

Recommend this journal