|J Pathol Inform 2014,
Can automated alerts within computerized physician order entry improve compliance with laboratory practice guidelines for ordering Pap tests?
Lydia Pleotis Howell1, Scott MacDonald2, Jacqueline Jones3, Daniel J Tancredi4, Joy Melnikow5
1 Department of Pathology and Laboratory Medicine, University of California Davis Health System, Sacramento, California, USA
2 Department of Internal Medicine, University of California Davis Health System, Sacramento, California, USA
3 Department of Clinical Information Systems/Knowledge Management, University of California Davis Health System, Sacramento, California, USA
4 Department of Pediatrics, University of California Davis Health System, Sacramento, California, USA
5 Department of Family and Community Medicine, University of California Davis Health System, Sacramento, California, USA
|Date of Submission||14-Jul-2014|
|Date of Acceptance||28-Aug-2014|
|Date of Web Publication||30-Sep-2014|
Lydia Pleotis Howell
Department of Pathology and Laboratory Medicine, University of California Davis Health System, Sacramento, California
Source of Support: None, Conflict of Interest: None
| Abstract|| |
Background: The electronic health record (EHR) provides opportunity to improve health and enhance appropriate test utilization through decision support. Electronic alerts in the order entry system can guide test use. Few published reports have assessed the impact of automated alerts on compliance of Pap ordering with published screening guidelines. Methods: Programming rules for Pap test ordering were developed within the EHR (Epic, Madison, WI) of the University of California, Davis Health System using American College of Obstetrics and Gynecology's 2009 guidelines and implemented in primary care clinics in 2010. Alerts discouraged Pap orders in women <21 and >71 years and displayed when an order was initiated. Providers were not prevented from placing an order. Results were measured during four calendar periods: (1) pre-alert (baseline) (July 2010 to June 2011), (2) post alert (alerts on) (July 2011 to December 2011), (3) inadvertent alert turn-off ("glitch") (January 2012 to December 2012), (5) post-glitch (alerts re-instated) (1/2013-7/2013). Metrics used to measure alert impact were between time and period seasonally adjusted relative frequency ratios. Results: Alerts were most effective in the <21 year old age group. During the baseline period 2.7 Pap tests were order in patients less than age 21 for every 100 Paps in those 21-71 years of age. This relative frequency decreased to 1.7 in the post-alert period and 1.4 during the glitch, with an even greater decline to 0.8 post-glitch when alerts were reinstated. Less impact was observed in the >70 year old group where the baseline relative frequency was 2.4 and declined to 2.1 post-alert, remained stable at 2.0 during the glitch period, and declined again to 1.7 post-glitch when alerts were reinstated. This likely reflects inclusion of women with a history of abnormal Pap tests for whom continued Pap testing is indicated, as well as reluctance by providers and patients to accept discontinuation of Pap testing for women with a history of normal Pap results. In both age groups, decreases in ordering were greatest when the alerts were functioning, indicating that the alerts had an effect beyond the influences of the environment. Conclusions: Discouraging alerts can impact ordering of Pap tests and improve compliance with established guidelines, thus avoiding unnecessary follow-up tests that can create potential patient harm and unnecessary expense. Alerts represent a potential model to address utilization of other lab tests. Longer study intervals are necessary to determine if provider compliance is maintained.
Keywords: Computerized order entry, decision-support, lab test utilization, Pap testing
|How to cite this article:|
Howell LP, MacDonald S, Jones J, Tancredi DJ, Melnikow J. Can automated alerts within computerized physician order entry improve compliance with laboratory practice guidelines for ordering Pap tests?. J Pathol Inform 2014;5:37
|How to cite this URL:|
Howell LP, MacDonald S, Jones J, Tancredi DJ, Melnikow J. Can automated alerts within computerized physician order entry improve compliance with laboratory practice guidelines for ordering Pap tests?. J Pathol Inform [serial online] 2014 [cited 2021 Sep 26];5:37. Available from: https://www.jpathinformatics.org/text.asp?2014/5/1/37/141994
| Introduction|| |
There is a growing expectation that health care providers will control utilization of laboratory testing in order to improve quality of care and reduce the costs of unnecessary testing. In 1998, the Institute of Medicine's (IOM) National Roundtable on Health Care Quality documented three types of quality problems: Overuse, underuse, and misuse.  These issues were re-emphasized in the IOM's 2001 report Crossing the Quality Chasm which stated that "Many patients, doctors, nurses, and health care leaders are concerned that the care delivered is not, essentially, the care we should receive. " Use of laboratory testing is included in these concerns. The IOM's statement can easily be rephrased as "… the lab tests delivered are not, essentially, the tests we should receive." Inappropriate use of testing can often lead to false positive results and incidental findings which prompt additional diagnostic studies, invasive procedures, and even unnecessary treatment. Each of these is associated with potential morbidity, mortality, anxiety, and expense, all of which can be harmful to the patient. Laboratory directors, including directors of cytopathology laboratories, therefore have a responsibility to address test utilization as part of their quality programs to ensure that testing is delivered appropriately and only to those who need it. Insurers are also embracing utilization as a measure of quality and changing their payment models to incentivize quality of care, including efficient use of laboratory services. The Federal Government is also discouraging test overuse through the quality measures included in the health information technology (IT) meaningful use program.  New payment models are emerging which forego fee for service for new models, such as a bundled payment structure, which incentivize less testing.
Over the years, many professional organizations and other groups have created laboratory practice guidelines (LPGs) to aid providers in appropriate selection of laboratory tests for their patients. LPGs for Pap testing and other cervical cancer screening tests have been developed by several professional societies, and were sometimes in conflict. In 2012, guidelines for cervical cancer screening were developed jointly by the American Cancer Society, American Society for Cervical and Colposcopic Pathology, and the American Society of Clinical Pathology with the participation of many other organizations. These guidelines include no Pap testing in women under age 21 years or over age 65 if the latter has a history of normal Pap tests,  and are consistent with guidelines on cervical cancer screening published by the US Preventive Services Task Force in 2012.  Both the American College of Obstetricians and Gynecologists (ACOG) and the American Association of Family Physicians included these guidelines for Pap testing in their "choosing wisely" list of evidence-based recommendations that physicians and patients should discuss in order to assist in wise decisions about the most appropriate care. The Choosing Wisely campaign is an initiative of the American Board of Internal Medicine Foundation "to encourage physicians, patients and other stakeholders to think and talk about medical tests and procedures that may be unnecessary or cause harm.  Leading specialty societies have been invited to participate in this initiative and contribute a list of tests that physicians and patients should question. To date, over 50 organizations have joined this effort which began in 2012, and the recommendations regarding testing that each organization has provided appear on a posted list on their website. 
A recent study funded by the Centers for Disease Control and Prevention found many impediments to provider compliance with LPGs, including lack of awareness, unfamiliarity and perceived difficulty in application, inconvenience, cumbersomeness, confusion, lack of a reminder system, lack of time, and fear of delayed or misdiagnosis leading to lawsuits. They suggest computerized physician order entry (CPOE) as a method to address these impediments.  Our study describes a decision-support method within the CPOE of our electronic medical record (EMR) which automates discouraging alerts to guide Pap test ordering for women under age 21 and over age 70 (the ages within the published ACOG guidelines at the time this study was undertaken).  We share pilot findings regarding the alerts' effectiveness in discouraging Pap tests orders in these age groups.
| Materials and methods|| |
This project was reviewed and approved by our university's institutional review board. Decision-support rules for interruptive alerts to discourage inappropriate Pap test orders were developed by an outside contractor and the EMR team for our health system. Our health system uses the Epic EMR (Epic, Verona, Wisconsin) which was first implemented in 2002. The Epic EMR is an integrated suite of health care software which combines chart review, order management and documentation for ambulatory and in-patient care, registration and scheduling, as well as systems for laboratory professionals, pharmacists, and radiologists; and billing systems for insurers.
Ages chosen for the Pap test alerts were under age 21 years under and over age 70 years, in accordance with ACOG 2009 cervical cancer screening guidelines when the project was initiated in 2010. These alerts were implemented within the primary care clinics, and displayed when an order was initiated for women < 21 years of age and over age 70 [Figure 1]. There was no "hard stop;" in other words, providers were not prevented from placing an order. Designated physician champions served as liaisons to the providers in the primary care clinics to share development of the alerts and receive feedback from users after implementation.
|Figure 1: Screen shot of interruptive alert discouraging Pap testing in women under age 21 years|
Click here to view
Results were measured during four calendar periods: Baseline (prealerts) (July 2010 to June 2011), postalert (alerts on): (July 2011 to December 2011), inadvertent alert turn-off ("glitch") (January 2012 to December 2012), postglitch (alerts re-instated) (January 2013 to July 2013). For each month during these periods. The number of total Pap tests ordered for each age group, is expressed relative to the number ordered in the reference age group (21-71 year olds). The metrics used to measure impact of the alert were between time and period seasonally adjusted relative frequency ratios (ARFR), estimated using multinomial logistic regression models that included fixed effects for month (11 df) to adjust for seasonality. ARFR < 1 are consistent with a reduction in the relative frequency of Pap tests for inappropriate age groups, a beneficial result. Model fit was examined graphically and residuals (ln [observed relative frequency] - ln [fitted relative frequency]) were tested for autocorrelation up to the fourth order using Durbin-Watson statistics. For each age group, pairwise ARFR and 95% confidence intervals were constructed among the following periods: (1) The baseline period (reference category) before alerts were initiated (July, 2010 to June, 2011), (2) A postalert period following initiation of alerts (July, 2011 to December, 2011), (3) A period in which alerts inadvertently stopped functioning (January, 2012 to December, 2012), and (4) A period in which the alerts were re-instated (January, 2013 to July, 2013).
| Results|| |
[Table 1] summarizes the data from the 4 time periods. The observed and fitted monthly relative frequencies of Pap orders in both age groups during the 4 time periods are illustrated in [Figure 2]. The multinomial logistic regression model fit the data well and the residuals did not exhibit statistically significant autocorrelation (all four Durbin-Watson P values exceeded 0.20). During the baseline period, 2.7 Pap tests were ordered in patients less than age 21 for every 100 Pap tests ordered for patients ages 21-71. In the postalert period, the relative frequency decreased to 1.7, a large reduction in seasonally adjusted relative frequency (ARFR = 0.58, 95% CI: 0.47, 70). During the period in which the alert inadvertently stopped functioning, the relative frequency in this age group was 1.4 (ARFR = 0.51, 95% CI: 0.43-0.59). When the alert was re-instated, an even greater decline was observed, with the relative frequency reduced to 0.8 (ARFR = 0.28, 95% CI: 0.21, 0.36), a 72% relative reduction in seasonally adjusted monthly relative frequencies compared to the baseline period.
|Table 1: Absolute and standardized relative frequencies of Pap tests by age group and time period |
Click here to view
|Figure 2: (a) Observed and fitted monthly relative frequencies of Pap tests ordered in women under age 21 during 4 time periods. (b) Observed and fitted monthly relative frequencies of Pap tests ordered in women over age 70 during 4 time periods|
Click here to view
For the over 70-year-old age group, the relative frequency of Pap test ordered during the baseline period was 2.4. In the postalert period, the relative frequency was 2.1 (ARFR = 0.95, 95% CI: 0.79, 1.15). During the period in which the alert inadvertently stopped functioning, the relative frequency of orders in this age group remained stable at 2.0 (ARFF = 0.85, 95% CI: 0.74, 0.98) When the alerts were re-instated, the relative frequency of Pap tests ordered was 1.7 (ARFR = 0.69, 95% CI: 0.57, 0.83), a large reduction from the baseline period.
| Discussion|| |
In crossing the quality chasm, the IOM suggests that quality of care could be improved through the creation of an infrastructure to support evidence-based practice, and by facilitating the use of IT.  The interruptive alerts that we created to discourage Pap test ordering in women under 21 and over 70 years old are an example of such an IT intervention that supports evidence-based practice. Our study suggests that these alerts are effective in discouraging Pap test ordering in these age groups since substantial decreases in the relative frequency of tests ordered occurred when the alerts were functioning, and a lesser decrease was observed during the interval with nonfunctioning alerts. Though it is possible that growing provider awareness of LPGs contributed at least in part to the decrease in Pap test ordering, the period in which the alerts inadvertently ceased to function provided a fortuitous opportunity to consider this potentially confounding variable. During the nonfunctioning alert period, Pap test orders relative frequencies in the under 21-year-old age group were slightly but not statistically significantly lower than in the previous functioning alert period (ARFR = 0.88, 95% CI: 0.71, 1.10), a reduction which was much less than the decline in orders before or after the nonfunctioning alert period. In the over age 70 age group, there was also no significant decline in Pap test ordering during the nonfunctioning alert period compared to the previous period (ARFR = 0.89, 95% CI: 0.73, 1.08), but monthly percentage of Pap tests ordered in this age group declined when alerts were first initiated and) and once again after the alerts were instated re-instated (alert-re-instated vs. alert-off ARFR = 0.89, 95% CI: 0.66, 0.99). We therefore believe that these interruptive discouraging alerts influenced compliance with guidelines beyond general awareness of LPGs for Pap testing alone.
It is interesting to note that our alerts had a greater influence on Pap test ordering in women under age 21 than in those over age 70. We speculate that providers may be more comfortable with LPGs that delay the start of cervical cancer screening to age 21 than they are with discontinuing screening in older women. Additionally, older women may not be very accepting of discontinuing Pap testing since the importance of this test has been strongly emphasized throughout their lifetime. Older patients may therefore pressure their providers to perform this test, making it difficult for providers to deny this test. A recent survey of obstetrician-gynecologists showed that only about half followed the 2009 ACOG recommendations to begin Pap testing at age 21 and discontinue at age 70. Physicians surveyed expressed concern that patients wouldn't come for regular exams without receiving a Pap test.  The ability of providers to effectively communicate with patients about changes in screening recommendations as well as the importance of having health care visits for other reasons may therefore limit the impact of an alert.
Alerts, as a form of decision-support in CPOE, have been shown to be effective in influencing a variety of physician ordering patterns. There are many published studies on prescribing alerts. These include reports of alert's effectiveness in reducing inappropriate medications for elderly patients in the emergency department and other settings, ,, reducing prescribing of heavily marketed hypnotic medications in ambulatory care settings,  and reducing overall prescribing of antimicrobials in addition to improving the appropriateness of antimicrobial prescribing. ,, According to a review of the literature on prescribing alerts in the elderly, a wide variety of CPOE interventions show clear potential to reduce inappropriate prescribing and polypharmacy in this population, but none of these have yet been widely adopted.  CPOE with decision-support has also shown effectiveness in reducing inappropriate transfusions. , Relatively few studies, however, have addressed alerts for orders for laboratory tests. Levick et al. showed a 21% decrease in repeat and serial orders in brain natriuretic peptide in intensive care patients after a "soft-stop" e-advisory to the ordering MD.  Similarly, unnecessary repeat viral serology tests decreased following implementation of alerts in a cardiovascular surgery clinic.  In contrast, alerts within the CPOE system that reminded providers about ordering tests for drug monitoring were not shown to be effective, according to a randomized controlled trial by Palen et al.  The literature on decision-support has to be interpreted with caution as many of the interventions described, including our study depend on institution-specific workflows and culture. Furthermore a disproportionate number of reports come from a small group of HIT focused academic institutions. 
Alerts as part of CPOE decision-support must be implemented thoughtfully. Alerts can be irritating since they interrupt workflow, create extra steps, and lead to "alert fatigue" in which individuals pay less attention to all alerts due to habituation.  Pearson et al. found that the most effective alerts were those that occurred after order selection, as were the alerts in our study.  Physicians and nurses responding to alerts as part of CPOE report that they prefer nonintrusive alerts, and that interruptive alerts should be selectively used for high-severity events. , We were fortunate to have strong clinical champions in the implementation of our project who consulted with clinical users to identify their issues and achieve buy-in as the project developed.
Dedicated support from the EMR team is also important to the success of this type of project. We are fortunate to have a strong health system IT team with a culture of innovation that we could leverage for this project. Our hospital is one of 118 hospitals nationally designated as a "Most Connected Hospital" by US News, and one of 11 hospitals in California designated as "Most Wired" by Hospitals and Health Networks magazine. , Our health system is a recent recipient of the Nicholas Davies Award which recognizes health care organizations who demonstrate the value of the EMR in supporting patient care, improved patient outcomes, addressing challenges and describing solutions that can be replicated by others.  Nonetheless, IT resources are often stretched thin within the institution, and this project was initiated using a contracted outside programmer. This made it difficult for our in-house EMR team to re-trace steps and resolve issues later.
Limitations to this study include the limited sample size and time frame, and the fact that this is a single-site study. We cannot fully assess the influence of national LPG education campaigns on ordering patterns or other system changes on the study effects. The observational nature and single-site of our study limit conclusions that can be drawn relative to a randomized controlled trial. While needed to confirm our findings, such a trial may be difficult to accomplish due to many logistic and economic challenges, including difficulty implementing alerts within different EMR systems at different institutions. As mentioned earlier, we believe that the period in which the alerts ceased functioning provided a comparison period to evaluate the influence of growing provider awareness of LPGs related to Pap testing.
In summary, our study suggests that interruptive alerts can effectively discourage ordering of Pap tests and improve compliance with established LPGs, thus providing the opportunity to avoid unnecessary follow-up tests that can create potential patient harm and unnecessary expense. Longer study intervals are necessary to determine if provider compliance is maintained. Additionally, our experience with Pap test alerts within CPOE provides an effective model worthy of consideration for controlling utilization of other lab tests. We encourage other laboratories and healthcare systems to consider this important method as a part of laboratory quality programs.
| Acknowledgments|| |
This work received the Future of the Profession Prize Paper Award when it was presented in part at the 61 st Annual Scientific Meeting of the American Society of Cytopathology in Orlando, Florida, November, 2013. Funding to support this project was provided by the UC Davis Health System Practice Management Board's Pay for Performance program.
| References|| |
|1.||Chassin MR, Galvin RW. The urgent need to improve health care quality. Institute of medicine national roundtable on health care quality. JAMA 1998;280:1000-5. |
|2.||Committee on Quality of Healthcare in America, Institute of Medicine. Crossing the Quality Chasm. National Academies Press, March 2001. Available from: http://www.nap.edu/html/quality_chasm/reportbrief.pdf. [Last accessed on 2014 Aug 09]. |
|3.||Available from: http://www.cms.gov/Regulations-and-Guidance/Legislation/EHRIncentivePrograms/Downloads/CQM_EPs_2012_02_02.pdf. [Last accessed on 2014 Jul 04]. |
|4.||Saslow D, Solomon D, Lawson HW, Killackey M, Kulasingam SL, Cain J, et al. American Cancer Society, American Society for Colposcopy and Cervical Pathology, and American Society for Clinical Pathology screening guidelines for the prevention and early detection of cervical cancer. CA Cancer J Clin 2012;62:147-72. |
|5.||Moyer VA, U.S. Preventive services task force. Screening for cervical cancer: U.S. Preventive services task force recommendation statement. Ann Intern Med 2012;156:880-91, W312. |
|6.||American Board of Internal Medicine. Choosing Wisely. Available from: http://www.choosingwisely.org/. [Last accessed on 2014 Aug 09]. |
|7.||Available from: http://www.choosingwisely.org/doctor-patient-lists/.[Last accessed on 2014 Aug 09]. |
|8.||Counts JM, Astles JR, Lipman HB. Assessing physician utilization of laboratory practice guidelines: Barriers and opportunities for improvement. Clin Biochem 2013;46:1554-60. |
|9.||ACOG Committee on practice bulletins - gynecology. ACOG practice bulletin no 109: Cervical cytology screening. Obstet Gynecol 2009;114:1409-20. |
|10.||Perkins RB, Anderson BL, Gorin SS, Schulkin JA. Challenges in cervical cancer prevention: A survey of U.S. obstetrician-gynecologists. Am J Prev Med 2013;45:175-81. |
|11.||Terrell KM, Perkins AJ, Dexter PR, Hui SL, Callahan CM, Miller DK. Computerized decision support to reduce potentially inappropriate prescribing to older emergency department patients: A randomized, controlled trial. J Am Geriatr Soc 2009;57:1388-94. |
|12.||Smith DH, Perrin N, Feldstein A, Yang X, Kuang D, Simon SR, et al. The impact of prescribing safety alerts for elderly persons in an electronic medical record: An interrupted time series evaluation. Arch Intern Med 2006;166:1098-104. |
|13.||Simon SR, Smith DH, Feldstein AC, Perrin N, Yang X, Zhou Y, et al. Computerized prescribing alerts and group academic detailing to reduce the use of potentially inappropriate medications in older people. J Am Geriatr Soc 2006;54:963-8. |
|14.||Fortuna RJ, Zhang F, Ross-Degnan D, Campion FX, Finkelstein JA, Kotch JB, et al. Reducing the prescribing of heavily marketed medications: A randomized controlled trial. J Gen Intern Med 2009;24:897-903. |
|15.||Clyne B, Bradley MC, Hughes C, Fahey T, Lapane KL. Electronic prescribing and other forms of technology to reduce inappropriate medication use and polypharmacy in older people: A review of current evidence. Clin Geriatr Med 2012;28:301-22. |
|16.||Gulliford MC, van Staa T, McDermott L, Dregan A, McCann G, Ashworth M, et al. Cluster randomised trial in the general practice research database: 1. Electronic decision support to reduce antibiotic prescribing in primary care (eCRT study). Trials 2011;12:115. |
|17.||Samore MH, Bateman K, Alder SC, Hannah E, Donnelly S, Stoddard GJ, et al. Clinical decision support and appropriateness of antimicrobial prescribing: A randomized trial. JAMA 2005;294:2305-14. |
|18.||Yazer MH, Triulzi DJ, Reddy V, Waters JH. Effectiveness of a real-time clinical decision support system for computerized physician order entry of plasma orders. Transfusion 2013;53:3120-7. |
|19.||Rothschild JM, McGurk S, Honour M, Lu L, McClendon AA, Srivastava P, et al. Assessment of education and computerized decision support interventions for improving transfusion practice. Transfusion 2007;47:228-39. |
|20.||Levick DL, Stern G, Meyerhoefer CD, Levick A, Pucklavage D. Reducing unnecessary testing in a CPOE system through implementation of a targeted CDS intervention. BMC Med Inform Decis Mak 2013;13:43. |
|21.||Niès J, Colombet I, Zapletal E, Gillaizeau F, Chevalier P, Durieux P. Effects of automated alerts on unnecessarily repeated serology tests in a cardiovascular surgery department: A time series analysis. BMC Health Serv Res 2010;10:70. |
|22.||Palen TE, Raebel M, Lyons E, Magid DM. Evaluation of laboratory monitoring alerts within a computerized physician order entry system for medication orders. Am J Manag Care 2006;12:389-95. |
|23.||Bright TJ, Wong A, Dhurjati R, Bristow E, Bastian L, Coeytaux RR, et al. Effect of clinical decision-support systems: A systematic review. Ann Intern Med 2012;157:29-43. |
|24.||Kudler NR, Pantanowitz L. Overview of laboratory data tools available in a single electronic medical record. J Pathol Inform 2010;1:3. |
|25.||Pearson SA, Moxey A, Robertson J, Hains I, Williamson M, Reeve J, et al. Do computerised clinical decision support systems for prescribing change practice? A systematic review of the literature (1990-2007). BMC Health Serv Res 2009;9:154. |
|26.||Handler JA, Feied CF, Coonan K, Vozenilek J, Gillam M, Peacock PR Jr, et al. Computerized physician order entry and online decision support. Acad Emerg Med 2004;11:1135-41. |
|27.||Sidebottom AC, Collins B, Winden TJ, Knutson A, Britt HR. Reactions of nurses to the use of electronic health record alert features in an inpatient setting. Comput Inform Nurs 2012;30:218-26. |
|28.||Available from: http://www.health.usnews.com/health-news/most- connected-hospitals/articles/2012/07/16/most-connected-hospitals- the-list. [Last accessed on 2014 Aug 09]. |
|29.||Available from: http://www.hhnmostwired.com/winners/PDFs/2013PDFs/2013MostWiredSurveyWinnersbyState.pdf. [Last accessed on 2014 Aug 09]. |
|30.||Available from: http://www.himss.org/News/NewsDetail.aspx?ItemNumber=22354. [Last accessed on 2014 Aug 09]. |
[Figure 1], [Figure 2]
|This article has been cited by|
||Effectiveness of Practices to Support Appropriate Laboratory Test Utilization
| ||Matthew Rubinstein,Robert Hirsch,Kakali Bandyopadhyay,Bereneice Madison,Thomas Taylor,Anne Ranne,Millie Linville,Keri Donaldson,Felicitas Lacbawan,Nancy Cornish |
| ||American Journal of Clinical Pathology. 2018; 149(3): 197 |
|[Pubmed] | [DOI]|
||The role of informatics in patient-centered care and personalized medicine
| ||Matthew G. Hanna,Liron Pantanowitz |
| ||Cancer Cytopathology. 2017; 125(S6): 494 |
|[Pubmed] | [DOI]|
||Roles of Health Care Providers and Patients in Initiation of Unnecessary Papanicolaou Testing After Total Hysterectomy
| ||Fangjian Guo,Yong-Fang Kuo |
| ||American Journal of Public Health. 2016; : e1 |
|[Pubmed] | [DOI]|
||Unintended adverse consequences of electronic health record introduction to a mature universal HIV screening program
| ||Laura N. Medford-Davis,Katharine Yang,Siavash Pasalar,M. Tyson Pillow,Nancy P. Miertschin,William F. Peacock,Thomas P. Giordano,Shkelzen Hoxhaj |
| ||AIDS Care. 2016; : 1 |
|[Pubmed] | [DOI]|
||Effectiveness of a computerized alert system based on re-testing intervals for limiting the inappropriateness of laboratory test requests
| ||Giuseppe Lippi,Marco Brambilla,Patrizia Bonelli,Rosalia Aloe,Antonio Balestrino,Anna Nardelli,Gian Paolo Ceda,Massimo Fabi |
| ||Clinical Biochemistry. 2015; 48(16-17): 1174 |
|[Pubmed] | [DOI]|