Data Collection: WHAT you measure and HOW you measure it

Data collection is one of the fundamental principles of systems-thinking and quality improvement. Being evidence-based is the buzz phrase in most quality and safety initiatives, and reams of data are collected on many things. There is often the notion that the more data we have, the better the improvement will be. This mindset frequently results in less than optimal results since quality is much more important than quantity. In data collection, the most important consideration is the identify What needs to be measured and How this will be measured. The “what” and the “how” can only be identified if a system or process is first made fully visible and understood. Then we need to identify the critical process elements (the critical 20% creating 80% of the results) and go to the source, with frontline care providers who actually do the work, to capture the information. Distinguishing between the formal from the informal process at work rears its head again.

What follows is an example of data collection that tracks the formal process rather than the informal process. In this hospital OR, efficiency and improvement focuses on time intervals manually entered into a computer, ideally in real time. Multiple time intervals are measured and the only one that is focused on at the quality improvement level is the time that the patient is in the room; for the first case of the day, 7:30 is the goal. As you read this, ask yourself whether the collected computer data reflects the “what” and the “how” to improve the OR environment.

Formal System: Computer Recorded Case Times

Set-up Begin: 7:18
Room Ready: 7:20
Patient In: 7:23*
Induction Complete: 8:22
Incision Made: 8:51
Ready to Close: 12:49
Surgery End: 13:40
Ready to Transfer: 13:40
Time Out: 13:42

*This is the only time deemed important: Patient In after 7:30 is documented as a late start

Informal System: Nursing Account of Same Case

This is not a litany of complaints. This is not one OR person “blaming” others, this is merely a commentary on one case, that unfortunately is very representative of what we are doing every single case.

Room 9, a relatively simple Dr. “X” spine procedure. 0730 on Tuesday.

We did not have a 6AM person setting up our rooms. The room did have a Jackson table with a Wilson frame on it. There were no dressings for the Wilson frame, and no face pillow. There was no “crank” for the Wilson, and the only one in the drawer in Room 10 was the wrong one.

Our float person was covering another room until 8AM.

The regular computer was not working. I had phoned the help desk two weeks earlier about the problem, it was never fixed. Today it was a problem because our films were from an outside hospital, and they were on a disc. Three phone calls to the help desk eventually brought a technician to the room. NIC notified.

Case was not booked with a C-arm. Dr. “Y” notified Dr. “X”’s office about the importance of doing this. Two phone calls to X-ray. The radiology tech was very gracious and did manage to secure a C-arm for an hour or so.

No microscope in Room 9. We claimed one that “J” and “B” were with (labeled Room 11) and eventually were able to locate the matching monitor, not in our area.

Two individual lights in the OR lights were out…call Bio Med.

Dr. “X”’s spine set contained two rongeurs that were caked with old bone. I bagged and tagged them, notified the NIC, she notified “K” from Quality Control and I ordered a Laminectomy I and II from Central Processing. I visited clean holding twice to collect them. The needed instruments in that kit were also dirty.

“K” came asking for the “slips” from the offending kits.

My room had one step stool. I borrowed another from Room 10, which was very soiled with blood and bone. I borrowed another. The care assistants did clean up the first one.

The arm board attached to the Jackson table broke as anesthesia attending slid it higher on the table. The patient’s arm did not fall. Located a care assistant who was dispatched to find another arm board.

My room (cabinets) were poorly stocked and in disarray. There were three incomplete Neuro prep kits, no body part for the hair clipper, no 1” paper tape, no gelfoam, 15 boxes of random sutures on the shelves, the Neuro sutures were depleted.
I had one ring stand only, and a broken lid on one of the trash containers. At 12 noon there were no towels on the exchange cart, nor were there any 60cc syringes, or 1000 drapes. Those were just the items I went looking for.

When my case finished, I took the patient to the recovery room, went to preop to see the next patient, and returned to the OR to start setting up…I was not able to clear the old sutures out of the cabinet, or gather supplies to re-stock the rooms. I did ask the supply room to come and make a list, but I never saw that happen. My apologies to the person who relieved me at three o’clock… and I guess that’s where we’re at… it’s a mess… I’m sorry… but it just isn’t getting done… run in and out of the adjoining rooms… scramble for what you need… borrow one of these… and lend something in return.

I know you can read between the lines… everyone of these simple “items” represents one more phone call… more waiting… time out of the OR… time away from the surgery… distractions and a hurriedness.

Help.

Questions: (What you measure and how you measure it)
1. Does the formal system reporting that quality improvement is based on reflect the informal system at work?
2. By computer, the patient was in the room before 7:30 and will be recorded as an on-time start – is this measure indicative of a quality process where no improvements are necessary?
3. With all of the running around that has to occur to get the patient in the room, do you think that the room was actually ready at 7:23?
4. Do any of the other recorded times suggest possible areas for improvement?

This example has a lot of lessons in it. Next time we’ll look at the implications of this work environment on staff satisfaction and patient safety.