Journal of Pathology Informatics Journal of Pathology Informatics
Contact us | Home | Login   |  Users Online: 203  Print this pageEmail this pageSmall font sizeDefault font sizeIncrease font size 




 
Table of Contents    
ORIGINAL ARTICLE
J Pathol Inform 2012,  3:3

A novel strategy for evaluating the effects of an electronic test ordering alert message: Optimizing cardiac marker use


1 Department of Pathology, Massachusetts General Hospital, Harvard Medical School, 55 Fruit Street, Boston, Massachusetts, USA
2 Information Systems, Partners Health Care System, Inc, One Constitution Center, Charlestown, MA, USA

Date of Submission10-Oct-2011
Date of Acceptance16-Dec-2011
Date of Web Publication29-Feb-2012

Correspondence Address:
Anand S Dighe
Department of Pathology, Massachusetts General Hospital, Harvard Medical School, 55 Fruit Street, Boston, Massachusetts
USA
Login to access the Email id

Source of Support: None, Conflict of Interest: None


DOI: 10.4103/2153-3539.93400

Rights and Permissions
   Abstract 

Background: Laboratory ordering functions within computerized provider order entry (CPOE) systems typically support the display of electronic alert messages to improve test utilization or implement new ordering policies. However, alert strategies have been shown to vary considerably in their success and the characteristics contributing to an alert's success are poorly understood. Improved methodologies are needed to evaluate alerts and their mechanisms of action. Materials and Methods: Clinicians order inpatient and emergency department laboratory tests using our institutional CPOE system. We analyzed user interaction data captured by our CPOE system to evaluate how clinicians responded to an alert. We evaluated an alert designed to implement an institutional policy restricting the indications for ordering creatine kinase-MB (CKMB). Results: Within 2 months of alert implementation, CKMB-associated searches declined by 79% with a corresponding decline in CKMB orders. Furthermore, while prior to alert implementation, clinicians searching for CKMB ultimately ordered this test 99% of the time, following implementation, only 60% of CKMB searches ultimately led to CKMB test orders. This difference presumably represents clinicians who reconsidered the need for CKMB in response to the alert, demonstrating the alert's just-in-time advisory capability. In addition, as clinicians repeatedly viewed the alert, there was a "dose-dependant" decrease in the fraction of searches without orders. This presumably reflects the alerting strategy's long-term educational component, as clinicians aware of the new policy will not search for CKMB when not indicated. Conclusions: Our analytic approach provides insight into the mechanism of a CPOE alert and demonstrates that alerts may act through a combination of just-in-time advice and longer term education. Use of this approach when implementing alerts may prove useful to improve the success of a given alerting strategy.

Keywords: Cardiac marker, ck-mb, computerized provider order entry, laboratory utilization, ordering alert, POE


How to cite this article:
Baron JM, Lewandrowski KB, Kamis IK, Singh B, Belkziz SM, Dighe AS. A novel strategy for evaluating the effects of an electronic test ordering alert message: Optimizing cardiac marker use. J Pathol Inform 2012;3:3

How to cite this URL:
Baron JM, Lewandrowski KB, Kamis IK, Singh B, Belkziz SM, Dighe AS. A novel strategy for evaluating the effects of an electronic test ordering alert message: Optimizing cardiac marker use. J Pathol Inform [serial online] 2012 [cited 2019 Dec 10];3:3. Available from: http://www.jpathinformatics.org/text.asp?2012/3/1/3/93400


   Introduction Top


Optimal patient care requires clinicians to modify their laboratory test ordering practices in response to advances in medical knowledge and technology. However, despite convincing evidence or guideline changes, some clinicians may continue ordering tests according to historic practices. Among the resources commonly leveraged by institutions to influence test ordering practices is the computerized provider order entry (CPOE) system. [1],[2],[3],[4],[5]

CPOE systems allow clinicians to directly input orders for diagnostic and therapeutic measures into computers. CPOE systems often include a laboratory test ordering module, which may be configured to permit the display of test-specific ordering alert messages or alerts. [1] Ordering alert messages may contain information such as testing indications, alternative tests to consider, cost, turnaround time, and assay limitations. [1],[5],[6],[7] Sophisticated systems can offer more advanced decision support by customizing alerts to remind clinicians of patient-specific issues such as redundant test orders or suggested corollary orders. [1],[2],[3],[8],[9],[10],[11],[12],[13] Further, alerts can be noninterruptive or interruptive. [1],[12] Noninterruptive alerts are passively displayed during the ordering process and do not require action on the part of the ordering provider. Interruptive alerts stop the workflow, requiring the provider to acknowledge the alert, and may additionally require a specific action such as entering an indication for testing.

Laboratories and institutions may design alerts to change test ordering practices in response to inappropriate test utilization or to implement new testing guidelines. [1],[2],[3],[14] One of the likely benefits of such alerts is that they provide just-in-time advice. As clinicians have limited time to reference textbooks, medical literature, and other traditional sources of knowledge during routine patient care, provision of just-in-time knowledge has been suggested as a strategy with great potential for improving clinical decision making. [4],[15] Alerts, in addition to their immediate effects on workflow and decision making, may also provide a longer term educational benefit with utility beyond the current ordering session. Despite the potential of a well-designed alert strategy, ordering alerts have been shown to vary considerably in efficacy [3],[14] and the precise characteristics and mechanisms contributing to an alert's success are not well understood.

In this report, we perform a detailed analysis of an alert used to implement an institutional decision to change the acceptable indications for creatine kinase-MB (CKMB) testing. Based on recent clinical guidelines, [16] our institution restricted CKMB testing for all indications except postpercutaneous coronary intervention (post-PCI). This represented a significant change, as CKMB was previously used in combination with troponin to diagnose or exclude routine acute myocardial infarction, assess infarct size, and monitor reinfarction. The new policy stated that troponin measurement alone should be used for these purposes. The challenge in substantially changing CKMB indications without banning the test altogether necessitated a well-designed CPOE alert strategy. We developed a novel alert evaluation methodology and applied it to the alert for CKMB. In creating this methodology, our aim was to develop a framework to better study and quantify the mechanisms and effects of a CPOE alert with the ultimate goal of improving alert design.


   Materials and Methods Top


Setting

The Massachusetts General Hospital (MGH) is an 898-bed tertiary care academic hospital in Boston, MA. The MGH Clinical Laboratories in the Department of Pathology support the inpatient medical, surgical, pediatric, and obstetric services of the hospital, as well as primary care and specialty outpatient practices in the greater Boston area. The clinical laboratories include core laboratory (chemistry, hematology, and immunology), microbiology, blood transfusion services, and various specialty laboratories. This study was conducted with approval of our hospital's Institutional Review Board (IRB).

Provider Order Entry System and Knowledge Management Middleware

The MGH Computerized Provider Order Entry system (CPOE) is an internally developed application. The CPOE system is used by inpatient and emergency room providers to place all patient orders including laboratory tests. The primary users of our CPOE system are residents in training, clinical fellows, attending clinicians, medical students, nurse practitioners, and physician assistants. Trainees comprise the largest group of ordering providers, accounting for over 90% of CPOE orders.

As described previously, the laboratory order entry module of the CPOE system utilizes a search methodology that accepts a search string from the user and returns a list of tests matching the search string. [17] The system maintains a list of search terms (test synonyms, common misspellings, and associated clinical conditions) used to match available tests to a given search string. Users select desired tests from the search results. Representative screen shots from the CPOE system are shown in [Figure 1]a and b. The system tracks user interactions including searches entered and tests ordered. Each event is time stamped and includes information about the patient, provider, search terms used, and tests ordered.
Figure 1: Screenshots of the laboratory test search module are shown. Note the ordering message (noninterruptive alert) displayed for CK isoenzymes (CKMB + CPK) shown in (a). Clinicians selecting CK isoenzymes from the search screen (by clicking "Add"), encounter the interruptive alert screen (b). The interruptive alert requires that the user enter an indication for testing or cancel the testing

Click here to view


As previously described, a novel middleware system interacts with the CPOE system and is used to provide real-time updates to test ordering messages, search terms and other test parameters. [17] The middleware can also be used to design interruptive alert windows to be displayed during the test ordering process and request CPOE user information regarding the test. Representative screen images of the middleware are shown in [Figure 2]a and b.
Figure 2: (a) A representative screen shot of the middleware knowledge management system. Ordering messages (noninterruptive alerts) can be added in a matter of minutes by changing the "SQL Additional Message" field and transmitting the update to the live system. Interruptive alerts can be built with using interface shown in (b)

Click here to view


CKMB Ordering Guidelines Changes

Effective 3/21/11, hospital guidelines were changed to restrict the ordering of CKMB solely to patients following percutaneous coronary intervention. This change was made in consultation with representative clinical staff including cardiology, medicine, and emergency medicine. In support of this change, a noninterruptive ordering message describing the new policy was added to the CKMB search results screen [Figure 1]a. Clinicians continuing past this noninterruptive message (by clicking "Add" to order CKMB) encounter an interruptive CPOE alert [Figure 1]b, requiring them to withdraw the CKMB order or enter an indication for testing. The guideline change was also communicated through several other means including email announcements.

Collection of CPOE User Interaction Data

Data for the study were compiled from CPOE searches and orders from February 1, 2011 to June 1, 2011. CKMB orders placed using free text, add-on test orders, or templates were not included in our analysis since these would not trigger the alert. CKMB orders placed using free text and add-on testing accounted for less than 1% of total orders. In addition, template orders, which accounted for less than 10% of total CKMB orders both pre- and postintervention, were presumably appropriate, as templates were updated to conform to the policy change.

Analysis of CPOE User Interaction Data

The captured data did not directly link searches and orders. To retrospectively link searches and orders, an order was deemed associated with a search if the search and order were for the same patient and provider and the order followed the search within 72 minutes (one twentieth of a day). While most often searches and associated orders occurred within a much shorter period, we used 72 minutes to account for the likelihood that providers may occasionally be interrupted during the ordering process.

We analyzed user data to compile a list of terms used to search for and order CKMB. Common CKMB-associated search terms included "cardiac markers," "CKMB," "MB" and "ck". In order to assess the user response to the alert we needed to determine the probable user intent when users searched using a CKMB-associated search term, as many search terms could be used to order CKMB or alternative tests. For example, searching for "creatine kinase" would result in both CKMB and CPK being retrieved and therefore the clinician could order either CPK or CKMB. In determining whether a search using a CKMB-associated term was intended to create a CKMB order, we applied the following criteria:

  1. If the search was associated with an order for CKMB, it was presumed to be intended for CKMB and classified as "CKMB order generating."
  2. If the search was only associated with orders for alternative tests retrievable using the search term, it was presumed to be intended for an alternative test and classified as "alternative test order generating."
  3. If a search using a CKMB-associated term was associated with neither a CKMB order nor an order for an alternative test the search was classified as "nonorder generating." To determine baseline provider intentions when using each term and thus how to count nonorder-generating searches, we examined data from the preintervention period (February 2011). We calculated a ratio (W) for each search term according to the following formula: Wi=Ci÷(Ci+Ai) where Wi equals the weighting factor for term i, Ci equals the number of searches in the preintervention period using term i that were CKMB-order-generating (criterion a, see above) and Ai equals the number of searches in the preintervention period using term i that were alternative test order generating (criterion b). A list of search terms used to order CKMB, weighting factors and associated data is shown in [Table 1].
    Table 1: Weighting factors used in analysis

    Click here to view
In the tabulation of searches using CKMB-associated search terms across the entire study period, CKMB-order-generating searches (criterion a) were counted as one search. Likewise, alternative-test-order-generating searches (criterion b) were counted as zero CKMB searches. Finally, nonorder-generating searches (criterion c) were counted as Wi fraction of a search. Except as otherwise noted, data were compiled and analyzed using Microsoft Access 2003 and Microsoft Excel 2003, including built-in statistical functions.


   Results Top


CKMB orders decreased by approximately 80% within weeks of alert implementation [Figure 3]. The total number of searches using CKMB-associated search terms decreased 79%, from 1,120 per month in the preintervention period (February 2011) to 233 per month within 2 months following implementation (P<0.01). Order-associated searches (searches for CKMB followed by an order for CKMB) decreased 87%, from 1,106 to 139 per month (P<0.01).
Figure 3: Searches using CKMB-associated search terms. Shown are 7-day rolling averages for CKMB searches per day, CKMB searches associated with CKMB orders, and the proportion of CKMB searches not associated with CKMB orders with the values for the dotted and dashed lines on the left vertical axis and the solid line on the right vertical axis. The alert was implemented on March 21, 2011 as indicated by the black triangle

Click here to view


Further, as shown in [Figure 3], the proportion of searches not resulting in orders markedly increased following the introduction of the alert, from 1% in the preintervention period (February 2011) to 40% postintervention (April 2011, P < 0.01). Since clinicians searching for CKMB and then deciding not to order the test in response to an alert would log a search without an associated order, the proportion of searches not resulting in orders provides an indication of the just-in-time advisory effect of the alert. To confirm of our strategy of weighting nonorder-generating (ambiguous) search terms, we performed a second analysis of search data, analogous to [Figure 3], but confining searches to the unambiguous terms "CK MB," "CKMB ," "CK-MB," "CKM," and "MB." The results of this analysis including only unambiguous terms are consistent with the results of our primary analysis (data not shown).

To investigate the long-term educational component of the alert, we hypothesized that the proportion of searches not resulting in orders would decrease as clinicians are increasingly exposed to the alert. Thus, clinicians who had previously seen the alert would be more likely to be aware of the new guidelines and would be less likely in the future to search for CKMB when not indicated. [Figure 4] shows the proportion of CKMB-associated searches not resulting in orders as a function of the number of times that provider had previously searched using a CKMB-associated search term and thus likely viewed the alert. As shown, in a "dose-dependant" fashion, providers who had not previously seen the alert (i.e., 0 prior searches) were more likely to place a search without an associated order compared to providers that had seen the alert one or multiple times (P<0.01).
Figure 4: Analysis of provider actions grouped by the number of times the provider had previously encountered the alert. Shown are the total numbers of CKMB searches, CKMB searches associated with orders (both on the left vertical axis), and the proportion of searches not associated with orders (right vertical axis), grouped by the number of times that provider had previously searched for CKMB following alert implementation (and thus likely saw the alert). CKMB searches were rounded to the nearest integer

Click here to view



   Discussion Top


We demonstrate a strategy for tracking and analyzing CPOE user interaction data to evaluate the effectiveness of a test ordering alert and to provide insight into the mechanism of an alert's effect. In particular, we show that our CKMB alert was highly effective in reducing inappropriate test orders and that it accomplished this goal through a combination of just-in-time advice and longer term clinician education. Many prior studies have demonstrated the overall efficacy of various alerts and alerting strategies. [1],[2],[3],[14],[18] However, to our knowledge, no prior studies have attempted to quantify the distinct advisory and educational effects of an alert. Thus, we considered the quantification of these effects to be an important aim for our study.

Our data have several implications for alert design and implementation. For example, the continued volume of CKMB-related searches as well as searches not resulting in orders weeks following the intervention highlights the need to communicate and advise clinicians on a policy changes over an extended time period. Reminders, either as CPOE alerts or other announcements, in the period immediately before and after a change, would likely be inadequate. In our study the need for an extended or even an indefinite alerting period may be in part a result of high provider turnover, particularly residents and fellows in multiinstitutional programs. While the issue of provider turnover may disproportionately affect hospitals where residents place many of the orders, even institutions with primarily full-time, permanent clinical staff may find value in a prolonged alerting period. In particular, our data demonstrating that even clinicians who had previously seen the alert still on occasion searched for CKMB without ordering it suggest the need for continued reinforcement. Nonetheless, one of the characteristics of an effective alert is that it should provide education and be triggered less frequently over time.

Several characteristics and features of our CPOE system and middleware were important contributors to the success of our alert and analysis strategy. Foremost, our analysis strategy would not have been possible without the CPOE capacity to track searches and orders along with associated information, such as providers, patients, and specific search terms used. Further, a well-designed and appropriately used CPOE system was essential. As previously described, efforts were made to develop an easy-to-use laboratory CPOE system that had robust search functionality. [17] The system design and implementation has created a culture and infrastructure such that free text orders are very rarely used. Since free text orders would not trigger the alert, frequent use of free text would have undermined our alerting strategy.

In addition, as previously described, a novel pathology middleware system interacts with our CPOE system, providing an interface that authorized pathology staff use to create and modify test ordering messages, alerts, and search parameters. [17] Using the middleware, development, and implementation of the alert was under the complete control of laboratory staff and took less than 10 minutes to create and test. Many alternative systems may utilize enterprise-wide information technology teams (central IT) to develop and implement CPOE alerts. [1],[19] Central IT teams are outside the domain of pathology and typically have many competing priorities. Had our alert required development by central IT, the alert building process could have taken months (rather than minutes) and may have required approval by an "alerts committee," despite CKMB guidelines having already been approved by the medical community. In addition, had our monitoring revealed that a change in the alert was indicated, or that expanded search parameters were needed to prevent free text, rapid revisions may not have been possible with a central IT approach.

Our analysis is subject to several limitations. As described in the methods, certain search terms were ambiguous, for example, a search for "cardiac markers" would most often be intended for multiple markers (i.e., CKMB and troponin), but occasionally, troponin alone might be desired. Our weighting system based upon preintervention search-order associations attempts to control for this uncertainty. Nonetheless, it is inexact and could overestimate both the postintervention CKMB search rate and the proportion of CKMB-associated searches not resulting in orders. For example, clinicians searching for "cardiac markers" may have more often intended troponin T without CKMB in the postintervention period than in the preintervention baseline period on which the weighting data are based. However, our data demonstrate a similar pattern when the analysis is confined to searches using terms unambiguously intended for CKMB (e.g., a search for "ckmb0"). In addition, the data demonstrating that the rate of searches not associated with orders declines with each instance a provider sees the alert offer internal validation of our approach.

Another potential limitation stems from the use of pre- and postintervention comparisons without randomization. While randomization of the providers to either receive the alert or not receive it may have offered a better control group, doing so would have been impractical within the confines of our technology and institutional structure. Methods of policy communication besides the alert (e.g., email announcements and word of mouth) likely contributed significantly to the initial decrease in searches and orders. However, trends beyond this initial postintervention period were presumably primarily alert mediated. Furthermore, the proportion of searches not resulting in orders likely reflects mainly alert effects. We are not aware of any other changes between the pre- and postintervention period likely to confound our results.

In this report we apply this approach to only a single alert. However, the methodology described here is straightforward and should permit this analysis to be extended to a variety of alerts to better understand the characteristics important for success. In particular, our methodology of weighting search inputs permits a linkage to be established between searches and orders. This is an important aspect of the analysis since search is a central feature of many CPOE systems and without a systematic approach to determine user intent the connection between searches, alerts, and orders cannot be readily determined. The evaluation of a larger cohort of alerts may help more precisely define the alert characteristics most effective in offering advice and those most effective in educating.

 
   References Top

1.Baron JM, Dighe AS. Computerized provider order entry in the clinical laboratory. J Pathol Inform 2011;2:35.  Back to cited text no. 1
[PUBMED]  Medknow Journal  
2.Georgiou A, Williamson M, Westbrook JI, Ray S. The impact of computerised physician order entry systems on pathology services: A systematic review. Int J Med Inform 2007;76:514-29.  Back to cited text no. 2
[PUBMED]  [FULLTEXT]  
3.Garg AX, Adhikari NK, McDonald H, Rosas-Arellano MP, Devereaux PJ, Beyene J, et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA 2005;293:1223-38.  Back to cited text no. 3
[PUBMED]  [FULLTEXT]  
4.Davenport TH, Glaser J. Just-in-time delivery comes to knowledge management. Harv Bus Rev 2002;80:107-11, 26.  Back to cited text no. 4
[PUBMED]    
5.Kim JY, Dzik WH, Dighe AS, Lewandrowski KB. Utilization management in a large urban academic medical center: A 10-year experience. Am J Clin Pathol 2011;135:108-18.  Back to cited text no. 5
[PUBMED]  [FULLTEXT]  
6.Tierney WM, Miller ME, McDonald CJ. The effect on test ordering of informing physicians of the charges for outpatient diagnostic tests. N Engl J Med 1990;322:1499-504.  Back to cited text no. 6
[PUBMED]  [FULLTEXT]  
7.Bates DW, Kuperman GJ, Jha A, Teich JM, Orav EJ, Ma'luf N, et al. Does the computerized display of charges affect inpatient ancillary test utilization? Arch Intern Med 1997;157:2501-8.  Back to cited text no. 7
[PUBMED]  [FULLTEXT]  
8.Tierney WM, McDonald CJ, Martin DK, Rogers MP. Computerized display of past test results. Effect on outpatient testing. Ann Intern Med 1987;107:569-74.  Back to cited text no. 8
[PUBMED]    
9.Bates DW, Kuperman GJ, Rittenberg E, Teich JM, Fiskio J, Ma'luf N, et al. A randomized trial of a computer-based intervention to reduce utilization of redundant laboratory tests. Am J Med 1999;106:144-50.  Back to cited text no. 9
[PUBMED]  [FULLTEXT]  
10.Chen P, Tanasijevic MJ, Schoenenberger RA, Fiskio J, Kuperman GJ, Bates DW. A computer-based intervention for improving the appropriateness of antiepileptic drug level monitoring. Am J Clin Pathol 2003;119:432-8.  Back to cited text no. 10
[PUBMED]  [FULLTEXT]  
11.Overhage JM, Tierney WM, Zhou XH, McDonald CJ. A randomized trial of "corollary orders" to prevent errors of omission. J Am Med Inform Assoc 1997;4:364-75.  Back to cited text no. 11
[PUBMED]  [FULLTEXT]  
12.Shah NR, Seger AC, Seger DL, Fiskio JM, Kuperman GJ, Blumenfeld B, et al. Improving acceptance of computerized prescribing alerts in ambulatory care. J Am Med Inform Assoc 2006;13:5-11.  Back to cited text no. 12
[PUBMED]  [FULLTEXT]  
13.Lo HG, Matheny ME, Seger DL, Bates DW, Gandhi TK. Impact of non-interruptive medication laboratory monitoring alerts in ambulatory care. J Am Med Inform Assoc 2009;16:66-71.  Back to cited text no. 13
[PUBMED]  [FULLTEXT]  
14.Hunt DL, Haynes RB, Hanna SE, Smith K. Effects of computer-based clinical decision support systems on physician performance and patient outcomes: A systematic review. JAMA 1998;280:1339-46.  Back to cited text no. 14
[PUBMED]  [FULLTEXT]  
15.Laposata ME, Laposata M, van Cott EM, Buchner DS, Kashalo MS, Dighe AS. Physician survey of a laboratory medicine interpretive service and evaluation of the influence of interpretations on laboratory test ordering. Arch Pathol Lab Med 2004;128:1424-7.  Back to cited text no. 15
[PUBMED]  [FULLTEXT]  
16.Saenger AK, Jaffe AS. Requiem for a heavyweight: The demise of creatine kinase-mb. Circulation 2008;118:2200-6.  Back to cited text no. 16
[PUBMED]  [FULLTEXT]  
17.Grisson R, Kim JY, Brodsky V, Kamis IK, Singh B, Belkziz SM, et al. A novel class of laboratory middleware. Promoting information flow and improving computerized provider order entry. Am J Clin Pathol 2010;133:860-9.  Back to cited text no. 17
[PUBMED]  [FULLTEXT]  
18.Kuperman GJ, Bobb A, Payne TH, Avery AJ, Gandhi TK, Burns G, et al. Medication-related clinical decision support in computerized provider order entry systems: A review. J Am Med Inform Assoc 2007;14:29-40.  Back to cited text no. 18
[PUBMED]  [FULLTEXT]  
19.Kuperman GJ, Diamente R, Khatu V, Chan-Kraushar T, Stetson P, Boyer A, et al. Managing the alert process at newyork-presbyterian hospital. AMIA Annu Symp Proc 2005:415-9.  Back to cited text no. 19
[PUBMED]  [FULLTEXT]  


    Figures

  [Figure 1], [Figure 2], [Figure 3], [Figure 4]
 
 
    Tables

  [Table 1]


This article has been cited by
1 Effectively Reducing CK-MB Utilization Using Computer Order Entry in the Emergency Department
Paul Sullivan,James Waymack,David Griffen,Cassie Jaeger
American Journal of Medical Quality. 2017; 32(1): 107
[Pubmed] | [DOI]
2 Health Informatics Tools to Improve Utilization of Laboratory Tests
Hassan A. Aziz,Hafsa M. Alshekhabobakr
Laboratory Medicine. 2017; : lmw066
[Pubmed] | [DOI]
3 Analysis of Daily Laboratory Orders at a Large Urban Academic Center
Joseph W. Rudolf,Anand S. Dighe,Christopher M. Coley,Irina K. Kamis,Bradley M. Wertheim,Douglas E. Wright,Kent B. Lewandrowski,Jason M. Baron
American Journal of Clinical Pathology. 2017;
[Pubmed] | [DOI]
4 Utilization of cardiac troponin assays in adult and pediatric populations: Guideline recommendations vs. reality
Amy K. Saenger,Shannon Haymond
Clinical Biochemistry. 2015; 48(18): 1213
[Pubmed] | [DOI]
5 Electronic Alerts for Triage Protocol Compliance Among Emergency Department Triage Nurses
James F. Holmes,Joshua Freilich,Sandra L. Taylor,David Buettner
Nursing Research. 2015; 64(3): 226
[Pubmed] | [DOI]
6 Cost and turn-around time display decreases inpatient ordering of reference laboratory tests: a time series
D. Z. Fang,G. Sran,D. Gessner,P. D. Loftus,A. Folkins,J. Y. Christopher,L. Shieh
BMJ Quality & Safety. 2014; 23(12): 994
[Pubmed] | [DOI]
7 Clinical and financial impact of removing creatine kinase-MB from the routine testing menu in the emergency setting
Rachel D. Le,Joshua M. Kosowsky,Adam B. Landman,Ida Bixho,Stacy E.F. Melanson,Milenko J. Tanasijevic
The American Journal of Emergency Medicine. 2014;
[Pubmed] | [DOI]
8 The role of informatics and decision support in utilization management
Jason M. Baron,Anand S. Dighe
Clinica Chimica Acta. 2013;
[Pubmed] | [DOI]
9 Reference laboratory utilization management
Patrick M. Sluss
Clinica Chimica Acta. 2013;
[Pubmed] | [DOI]



 

 
Top
  

    

 
  Search
 
   Browse articles
  
    Similar in PUBMED
   Search Pubmed for
   Search in Google Scholar for
 Related articles
    Access Statistics
    Email Alert *
    Add to My List *
* Registration required (free)  

 
  In this article
    Abstract
   Introduction
    Materials and Me...
   Results
   Discussion
    References
    Article Figures
    Article Tables

 Article Access Statistics
    Viewed2861    
    Printed136    
    Emailed1    
    PDF Downloaded419    
    Comments [Add]    
    Cited by others 9    

Recommend this journal