Nutrition Calculator information, tips and advice

Process

Residents received a brief 10-minute training before the initial use of the TPNCalculator. Through a public workstation or a desktop computer, the user (resident, nurse practitioner, pharmacist, nutritionist) accesses the TPNCalculator using individual log-in identification and password. All interactions between the TPNCalculator Server and the user's browser are 128-bit encrypted to be compliant with the Health Insurance Portability and Accountability Act.

The user may select to enter a TPN Order for a new patient or to modify/renew an old order for an existing patient. TPNCalculator performs all necessary fluid and component calculations on the basis of provider input via an Internet browser. It contains nutritional guidelines, an osmolality calculator, and 62 rule-based alerts and reminders (Fig 1). These alerts and reminders include a check for age/weight ratio (if the child is outside the normal weight range for the calculated age, then the user is prompted), dose range alerts based on total dose, dose per weight, concentration (as indicated), and percentage of total calories (as indicated) for the included components such as glucose, lipids, protein, electrolytes, heparin, and ranitidine. Other features include automatic selection of the appropriate protein additive based on age and an osmolality alert, which is triggered only in patients without a central line. Users are reminded to add phosphorus and calcium if they have not done so and to add heparin for central line TPN. Reminders can be complex based on previous user selections; for example, if a user chooses not to order trace elements, then not only is he or she prompted to add selenium, zinc, and chromium, but also the dose ranges of these additives will now be checked. Users are prevented to submit an order unless it contains patient name and medical record number, physician name, and identification number. Other data entered by the user are validated against a possible range (eg, infusion time cannot exceed 24 hours). Patient data are automatically populated using a link to the hospital admission, discharge, and transfer system.

[FIGURE 1 OMITTED]

Data collected during a designated quality improvement project included the number of parenteral nutrition orders and the frequency and type of errors as identified by the pharmacist. Data were collected during a control period (October 2, 2000, to November 14, 2000) immediately before and during 2 intervention periods after (intervention 1: November 15, 2000, to December 31, 2000; intervention 2: August 27, 2002, to October 13, 2002) the implementation of TPNCalculator. Data analysis was done with [chi square] test. Prescribers, pharmacists, and nurses were surveyed online on their experiences and opinions in the course of using TPNCalculator.

RESULTS

During the control period, average patient census was . During this time, 557 TPN orders were written compared with 471 orders and 656 orders in the 2 intervention periods (average census: and ). The number of TPN orders per patient per day was not significantly different between the time periods (control: ; intervention 1: ; intervention 2: orders per patient per day): During the control and the first intervention periods, only half of the orders were available to the pharmacy by order deadline at noon (% control, % intervention). Prescribers most commonly cited lack of time as the cause for the delay (86%). During the first intervention period, lack of time (54%) and reduced staffing during the holidays (31%) were given as a frequent cause of delays.

During the control period, a total of 60 errors ( errors per 100 orders) that required the pharmacist to contact the provider were detected compared with only 20 errors ( errors per 100 orders) in the first and 8 errors ( errors per 100 orders) in the second intervention period. This represents a 61% reduction in error rate (P < .01) in the first and an 89% reduction in error rate (P < .001) in the second intervention period.

Error type distribution was significantly different among the 3 groups (Fig 2). During the control period use of the paper form, prescribers were significantly more likely to order fluids that exceeded the osmolality guidelines, make calculation errors, and generate orders that demonstrated other knowledge deficiencies (eg, no heparin ordered for central line TPN). No statistical difference was noted in the number of orders with insufficient fluid amounts to allow all additives to be dissolved between the control period and both intervention periods, although there was a trend toward a reduction in intervention 2. There was an increase from incomplete orders per 100 TPN orders in the control period to in the intervention period (not significant). Omission of the page number on the order form (order identifier) contributed to 92% of all incomplete orders in the intervention period (Fig 3). In the second intervention period, there was a 100% reduction of omissions (P < .001).

[FIGURES 2-3 OMITTED]

After a 6-month period, when all stakeholders had an opportunity to interact with the TPNCalculator, 84 providers, nurses, and pharmacists were surveyed using an online questionnaire. Twenty-eight prescribers responded, including 22 postgraduate year 1 residents (100% of interns) and 6 neonatal nurse practitioners (100% of neonatal nurse practitioners). Seven percent had used TPNCalculator <5 times per week, 21% used it 5 to 9 times, 36% used it 10 to 19 times, and 36% used it >20 times per week. On a scale from 1 (very easy) to 5 (very difficult), TPNCalculator was rated . On a Likert scale (1 = strongly agree, 3 = neutral, 5 = strongly disagree), prescribers were asked to compare the TPNCalculator with the paper form (Table 1).

In comparison with the paper form, TPNCalculator was found to be easier to learn and to use. Prescribers believed that it protected against errors, saved time, was helpful, and constituted an improvement. Users were neutral to the statement that the TPNCalculator provided a better learning experience.

As part of the survey, prescribers were asked about the series of potential problems associated with an ordering tool. Prescribers were neutral toward the statements that TPNCalculator carried the risk of inaccurate programming (), had limited scope and was not designed to handle all clinical cases (), had potentially incomplete information/ missing data (), redirected provider priorities (), and generated false expectation such as being alerted to all problems (). Prescribers disagreed that TPNCalculator caused data overload in the user ().

Using separate online questionnaires, we also surveyed 8 pediatric pharmacists (73% of pharmacists) and 43 neonatal nurses (42% of nurses). Comparing TPNCalculator and the paper form on a Likert scale from 1 (strongly agree) to 5 (strongly disagree), pharmacists and nurses agreed that TPNCalculator was easier to read, was easier to learn, and resulted in less need for order clarification. Pharmacists reported that TPNCalculator resulted in fewer errors, saved time, and was helpful for data entry (Table 2).

DISCUSSION

ADEs are common, affecting as many as 1 in every 25 hospitalized patients. (12) The ADE Prevention Study Group reported that few risk factors are patient related, suggesting that rather than target ADE-prone individuals, prevention strategies should focus on improving medication systems. (13)

After identifying a medication ordering process with significant risk for ADEs, we designed, implemented, and evaluated a web-based medical information system (TPNCalculator) to improve the ordering process and reduce ADEs. We hypothesized that underlying system failures could be corrected with inexpensive, pragmatic problem-solving approaches. A deliberate attempt was made to address only the "broken" link, thereby preserving the majority of the existing ordering process. Furthermore, we based the interface design on existing paper forms to provide users with a familiar format and maximum comfort level. This minimized required training (usually <10 minutes) and permitted the existing order-processing flow to remain intact. The use of participant/observers (prescribers and pharmacists) as programmers, designers, and testers eliminated the need for extensive life cycle assessment (systematic set of procedures for compiling and examining the inputs and outputs of materials and energy and the associated environmental impacts directly attributable to the functioning of a product or service system throughout its life cycle), including evaluation and analysis of usage, resource utilization, patient care implication, and failure modes. (11) Using this approach, the complete design process (1 Cold Fusion programmer [.L.]) and quality assurance (1 nutritionist [.C.] and 1 pharmacist [.C.]) took 3 weeks (~200 man-hours), which starkly contrasts with other ongoing clinical medical informatics projects at JHH. Modifications of TPNCalculator require minimal intervention, because any change made on the web server is immediately available to all users throughout the system.

In this study, we were able to use a close-held, multitalented, small development group to develop a rapid solution in cooperation with local information services. This model was highly effective in creating utilities for in-house support of clinicians and has in the meantime been duplicated in several other JHH projects. It must be pointed out that a development model such as ours is frequently not available to those outside the clinical process. Reasons for this may include Health Insurance Portability and Accountability Act concerns, institutional policies and interests, lack of access to data from legacy systems, and the inability to assemble a small team with all of the required skills.

Evaluation of error rates and user surveys revealed an overwhelming success. We reduced the total number of errors by 89%. Calculation errors were eliminated through TPNCalculator, and knowledge problems drastically reduced. Users of the system (including those who were involved only peripherally, eg, nurses) were enthusiastic and supportive and compared it favorably with the previous paper-based system. We experienced significant pressure to make TPNCalculator available to all units in the children's hospital and to an affiliated hospital NICU once residents had used it during their NICU rotation. As a result, 6 months after the development of TPNCalculator, it was deployed throughout the JHH Children's Medical & Surgical Center.

In our survey, users did not estimate the risks of TPNCalculator to be higher than in the paper system; however, experience (14) has shown that the deployment of a new software tool may introduce a new type of failure mode in the form of programming/ software errors. An analysis of 3140 medical device recalls conducted by the Food and Drug Administration between 1992 and 1998 revealed that 242 (%) were attributable to software failures. (15) Of those, 192 (79%) were caused by software defects that were introduced when changes were made to the software after its initial production and distribution. Although TPNCalculator corrected a number of system problems, in its initial version, it had its own system failure: a drastic increase in the TPN orders without order identification (page number). Once this (small and insignificant for patient safety) problem was identified, TPNCalculator was first modified to prompt the user for a page number, and in a later version, an automatic page number creation rule was added. Data from the second intervention period demonstrated that omissions had been eliminated without adding new errors. The example of the "missing page number" emphasizes that rapid cycle evaluation is able to handle software design problems through frequent series of design, evaluation, and redesign. An attempt to design a "perfect" and completely error-free application from the start would have required considerable planning, consulting, and review before implementation, leading to long cycle times, which might have derailed the process long before completion. In addition, the desire to design the "perfect" system might have resulted in an overengineered product with additional features, complicating its use and making it unattractive to users.

Federal guidelines require that medical software systems become subject to validation requirements (Title 21 Code of Federal Regulations [section] [a]). Such systems must be validated to ensure accuracy, reliability, and consistent intended performance. Reducing total development time to 3 weeks inherently means that validation features such as requirement and feature lists are mainly implicit in the design and not necessarily expressed in detail. In our case, this was possible because the software was modeled on an existing paper form in use at JHH. The paper form implicitly determined system inputs and outputs, functionality, interface definition, user interactions, safe ranges, limits, and default values. Using domain experts for the testing allowed for testing of all potential clinical scenarios using patient data available during the development period. The robustness of the system was tested in a clinical environment by allowing users to use TPNCalculator while for a period of 3 months all calculations were duplicated in the pharmacy to ensure safety. During that period, users generated >1800 orders, which spanned the complete range of possible inputs. The existing paper order form served as a safety back-up but was never used.

Developing TPNCalculator would not have been possible without certain information infrastructure already in place at JHH. We relied on the availability of public workstations in all clinical areas where providers might order TPN. If access to computers would have required traveling (even a short distance) or a waiting period, then we suspect that users' enthusiasm for this application would have been drastically diminished. Furthermore, the decision by JHH management to provide an Internet browser on all workstations was crucial to implementation. Allowing Internet browsing on public clinical workstations is not a universal practice, as demonstrated by differences even among the hospitals within the Johns Hopkins Health System.

We have demonstrated that pragmatic approaches to ADEs involving simple medical information systems can be successful in both reducing the number of errors and increasing user satisfaction. The approach used in the development of TPNCalculator, which could be called "find it, fix it, and forget it," identifies a specific problem and uses existing infrastructure to develop a pragmatic medical informatics solution to the problem while leaving the remaining system intact. By limiting the number of people involved in the process of problem identification, development, testing, and deployment, the used resources such as time and manpower can be drastically reduced without losing effectiveness. By shortening the time interval between the birth of an idea and its implementation, we increased the speed of innovation. Although this approach carries a greater risk for software design flaws, in our opinion, this risk can be minimized by using participant/ observers in the development and is further offset by the significant gains through early implementation and cost reduction.

CONCLUSION

Low-cost, pragmatic approaches using Internet technology in the design of medical information systems might pose a viable alternative for the prevention of ADEs.

TABLE 1. Prescriber Survey (N = 28)

 Average Rating Range

Easy to use 1-3
Easy to learn 1-2
Protects against errors 1-2
Saves time 1-2
Helpful 1-2
Improvement 1-2
Better learning experience 1-4

Comparing TPNCalculator with paper from on a 5-point Likert
scale (1 = strongly agree, 5 = strongly disagree).

TABLE 2. Nurse /Pharmacist Survey

 Nurses Pharmacists
 (n = 43) (n = 8)

Easier to read 
Easy to learn 
Reduced transcribing errors 
Saved time 
Helpful for data entry 
Significant improvement 
Less need for clarification 
Contained less errors N/A 
Improved order timeliness N/A 

Comparing TPNCalculator with paper from on a 5-point Likert
scale (1 = strongly agree, 5 = strongly disagree).

ABBREVIATIONS. ADE, adverse drug event; NICU, neonatal intensive care unit; TPN, total parenteral nutrition; JHH, Johns Hopkins Hospital.

ACKNOWLEDGMENTS

This work was supported by the Center for Innovation and Quality Patient Care at the Johns Hopkins Hospital.

We are grateful to Harold P. Lehmann, MD, PhD, for critical review of this manuscript.

REFERENCES

(1.) Kohn LT, Corrigan JM, Donaldson MS, eds. To Err Is Human: Building a Safer Health System. Washington, DC: National Academy Press; 1999

(2.) Leape LL, Bates DW, Cullen DJ, et al. Systems analysis of adverse drug events. ADE Prevention Study Group. JAMA. 1995;274:35-43

(3.) Leape LL, Kabcenell AI, Gandhi TK, Carver P, Nolan TW, Berwick DM. Reducing adverse drug events: lessons from a breakthrough series collaborative. Jt Comm J Qual Improv. 2000;26;321-331

(4.) Leape LL. Error in medicine. JAMA. 1994;272:1851-1857

(5.) Kaushal R, Bates DW, Landrigan C, et al. Medication errors and adverse drug events in pediatric inpatients. JAMA. 2001;285:2114-2120

(6.) Horn W, Popow C, Miksch S, Kirchner L, Seyfang A. Development and evaluation of VIE-PNN, a knowledge-based system for calculating the parenteral nutrition of newborn infants. Artif Intell Med. 2002;24:217-228

(7.) Hermansen MC, Kahler R, Kahler B. Data entry errors in computerized nutritional calculations. J Pediatr. 1986;109:91-93

(8.) Legler JD. Physicians' accuracy in manual computation. MD Comput. 1990;7:155-159

(9.) Tepas JJ 3rd, Mollitt DL, String DL, Pieper P. Error in fluid and calorie calculation in the surgical neonate. J Pediatr Surg. 1991;26:132-134

(10.) Vincer MJ, Murray JM, Yuill A, Allen AC, Evans JR, Stinson DA. Drug errors and incidents in a neonatal intensive care unit. A quality assurance activity. Am J Dis Child. 1989;143:737-740

(11.) Cohen MR, Senders J, Davis NM. Failure mode and effects analysis: a novel approach to avoiding dangerous medication errors and accidents. Hosp Pharm. 1994;29:319-330

(12.) Brennan TA, Leape LL, Laird NM, et al. Incidence of adverse events and negligence in hospitalized patients. Results of the Harvard Medical Practice Study I. N Engl J Med. 1991;324:370-376

(13.) Bates DW, Miller EB, Cullen DJ, et al. Patient risk factors for adverse drug events in hospitalized patients. ADE Prevention Study Group. Arch Intern Med. 1999;159:2553-2560

(14.) McDaniel JG. Improving system quality through software evaluation. Comput Biol Med. 2002;32:127-140

(15.) US Department Of Health and Human Services Food and Drug Administration. General Principles of Software Validation; Final Guidance for Industry and FDA Staff. Available at: .gov/cdrh/comp/ guidance/

Christoph U. Lehmann, MD *; Kim G. Conner, RPh, Mac ([double dagger]) ([section]); and Jeanne M. Cox, MS, RD ([section])

From the * Division of Neonatology, Department of Pediatrics, and Division of Health Sciences Informatics, Johns Hopkins University, Baltimore, Maryland; ([double dagger]) Division of Pediatric Pharmacy, Johns Hopkins Hospital, Baltimore, Maryland; and ([section]) Division of Pediatric Gastroenterology/Nutrition, Johns Hopkins Hospital, Baltimore, Maryland.

Received for publication Jan 21, 2003; accepted Jul 25, 2003.

Reprint requests to (.L.) Eudowood Neonatal Pulmonary Division & Division of Health Sciences Informatics, Johns Hopkins University, 600 N Wolfe St, CMSC 210, Baltimore, MD 21287-3200. E-mail: clehmann@