This is the sixth part in a six-part series about critical issues in healthcare Quality Improvement (QI) analytics. My goal was to highlight persistent issues observed over the last decade in the QI space. For this last article, I want to focus on using the correct analytic methods and interpretations.
Here we go.
The Issue: Incorrect Analytic Methods and Interpretations
The data analysis community has no shortage of methods available. It seems like researchers release papers describing new methods nearly every week. Often, developers create new methods to handle specialized and nuanced analytic scenarios.
Yet, for all the advancements data scientists are making, most data analytic projects require relatively basic methods. With well-thought-out research designs and basic analytic methods, QI staff can answer a lot of questions.
Incorrect Analytic Methods
That being said, it always surprises me when I see QI staff using the wrong analytic method for a project. Often the problem is that the analyst is using a statistic intended for a different type of outcome. For example, the analyst uses a model for continuous outcomes when their data are categorical.
Incorrect Interpretations
A related issue arises when staff don’t understand the correct interpretation of the results. The analyst creates an interpretation that sounds good based on their limited knowledge of the analysis.
The critical problem is that statisticians and data scientists create analytic methods to be used with particular types of outcome variables to answer specific types of questions. Additionally, each method includes a specific interpretation of the results.
Ultimately, analysts using incorrect analytic methods and interpretations make poor decisions at best and wrong decisions at worst.
Identifying the Correct Analytic Methods
You can often identify the correct analytic method to use if you have three pieces of information about your data:
- What type of question are you answering? Are you trying to describe something, compare groups, identify relationships, detect similar groups of observations, or predict an outcome? The answer to this question will help you determine, broadly, the type of analysis you need.
- What type of variable is your outcome? This refers to the level of measurement of your outcome. Is it a categorical variable with discrete values or a continuous variable with granular detail? Often, the level of measurement for your outcome will influence the specific flavor of analysis you use.
- Do your data follow a specific type of distribution (e.g, a bell curve)? Some statistical tests require your data to have a specific distribution or general shape when graphed as a histogram. If your data meet this requirement, you can use a parametric test, which assumes a specific distribution. Otherwise, you may need a non-parametric method which relaxes the distributional assumption.
To be certain, there are other factors that go into picking the correct analytic method. However, the three factors listed above will get you very close in most circumstances.
To refine your selection process, you’ll want to learn more about the specific circumstances and assumptions required for various analyses.
Interpreting Analytic Results Correctly
Unlike selecting the correct analytical method, learning how to interpret the results requires learning the specifics of each method.
You will need to know that a test statistic is the calculated value of the analysis. Sometimes referred to as a point estimate, the test statistic is what gets reported most often as substantive information.
Unfortunately, the interpretation of the test statistics changes from one analytic method to another. Sometimes, the differences in interpretation are dramatic. Always look up the interpretation for the specific method you are using.
You will also need to know that most test statistics also have an estimate of potential error, called the standard error. The standard error tells you how much precision you have in your analysis, with smaller values being more precise.
Again, the interpretation of the standard error is based on the meaning of the test statistic. Therefore, it is doubly important for you to understand how to interpret the test statistic before interpreting the standard error.
Conclusion
When you are designing an analytic QI project, selecting the correct analytic methods does not need to cause stress. Identify your situation in terms of the type of question, level of measurement for the outcome, and distributional assumptions.
With these pieces of information, most introductory resources for data analytics should provide you with a methodology that will work. To be certain, I recommend you consult with an experienced analyst in your network. Alternatively, you can reach out to an experienced consultant like F1 Analytics.
Once you identify the best analytic methodology, you will be perfectly positioned to research how to interpret the results correctly. Here again, doing your homework or consulting with an expert will help solidify the information clearly.
Following these recommendations will help you plan and execute your next analytic project with the correct analytic methods and interpretation
Other Articles in This Series
If you found this brief article helpful, see the other articles in the series here:
- Small Sample Sizes
- Lack of Comparison Groups
- Sufficient Time Frames
- Poor Data Collection Strategies
- Reliance on Metrics Intended for Other Uses
- Misuse and Misinterpretation of Statistical Tools and Results
