Welcome back to the fourth in a six-part series of short, actionable articles about critical issues in healthcare quality improvement. This week, I’m addressing a challenge not often discussed but which has profound impacts on Quality Improvement (QI): data collection strategies.
The Issue: Appropriate Data Collection Strategies
Research is a data-hungry enterprise.
While a lot of thought may go into designing the implementation plan, data collection and evaluation are often secondary discussions.
Unfortunately, I have encountered many healthcare professionals who assume that the means to collect and analyze data are universal. They also assume that any implementation plan can be evaluated the same as any other.
This has led to many QI projects being implemented without appropriate data collection and evaluation strategies in place. Program Directors failing to develop appropriate data collection strategies will, at best, cause delays and increase the resources required to complete the project. In worst-case scenarios, these problems can prevent a project from being completed and evaluated entirely.
An Example: The Impact of Poor Data Collection Strategies
Hospital at Home (HaH) programs have demonstrated success in allowing a non-trivial portion of patients to be treated at home rather than in the hospital.
I had the opportunity to participate in an evaluation of one of these innovative programs. I was excited at the time because the program offered an opportunity to compare patients in the program to those treated in the traditional hospital setting, because the program targeted a specific subset of disease conditions, and because the processes and procedures were well-defined in the implementation. In short, the project had all the hallmarks of a very interesting evaluation.
We met with the program directors to discuss the requirements of the evaluation, the data sources needed, and timelines for completion.
As we got into the discussion of data sources, I asked a clarifying question to make sure we had all of the information we needed: “How do you identify the patients who are participating in the program?”
The pause on the other end of the conference call quickly went from uncomfortable to concerning.
“Wouldn’t that be captured in the EHR system?”
We had to explain that because the EHR system was not designed with this program in mind, it was unlikely to include a field indicating participation in the HaH program. Perhaps there might be a way to identify patients based on having a discharge to home and subsequent daily physician and nursing visits – if those costs were still being borne by the hospital.
Fixing the Problem
The program directors agreed to find out how that was being captured in the system and follow up with us. Ultimately, they acknowledged that the program had been operating for several months with no solid record of the participating patients.
They asked our team to examine claims data to try to identify patients who were likely to have been in the program. They asked the hospital facilities to revisit their own records and staff memories to identify patients.
After a considerable time and expense for this exploration, program directors were instructed to implement a system for identifying patients in the program.
The program timeline was extended, in part to allow for sufficient data collection for analysis. The delay eventually stretched over a year before enough data was available for the evaluation.
Building In Data Collection Strategies
The moral of the story is: don’t implement your project until you have an appropriate data collection strategy.
If you begin thinking, “We’ll figure that out as we go,” that’s a sign NOT to implement the project.
At a minimum, you need to be able to identify the following:
- Who are the project population (i.e., participants)
- Metrics for key outcomes and inputs
- The data collection systems and processes
- How to link data from different systems
If you don’t have the expertise, bring in staff experienced in research design and evaluation methods. Ask them to help design these elements of the project in parallel with the program implementation plan.
I have seen many QI projects where designers planned the evaluation after the implementation had already begun. It’s often too late for you to make changes to the implementation after the fact if needed. And typically, those evaluations are weaker than if you designed the evaluation in parallel.
You will also want to confirm the method you will use to collect data. Will you use a stand-alone tool like an Excel workbook or a paper form? Or will you use a survey tool or custom fields in your EHR system?
Regardless of the data collection method chosen, you’ll want to consider its impact on staff workflows in the setting. After all, you want to minimize the total impact of the QI project on the delivery of care to your patients.
Conclusion
It would be great if you could design a perfect QI project and evaluation every time. However, time, resources, and systems constraints often make this challenging.
Therefore, your goal is to have a workable solution rather than a perfect one before implementation.
You will facilitate team efficiency by designing the data collection and evaluation strategy during the project planning stages. You will also have a greater opportunity to identify successes or potential problems in the implementation along the way.
Other Articles in This Series
If you found this brief article useful, see the other articles in the series here:
