It’s a familiar story, and probably one that you know. A manager meets with the analyst team and gives them parameters for a project. “We need to know if our marketing improved sales with our target market.”
The analytic team probably had a bunch of questions following a project statement like the one above….and rightly so. If they didn’t have a bunch of questions, then I am guessing the manager didn’t get back exactly what they wanted and there was substantial rework required on the project.
To get the best performance out of your analytic team, you want to be prepared with information that will help clarify your request and give the analytic team stronger direction. Here are five (5) things you should be able to discuss with your analysts before they begin working on your next project
What Is Your Research Question?
The question you want answered sets the stage for the entire project. The best answer to the wrong question won’t give you the information you are looking for.
Provide your analysts with both the question and the reason why you want to know the answer. It might be tempting to only give them the information they need to do their jobs, but explaining why you want the answer gives the analysts an opportunity to consider whether the data they’ll be using can provide an answer that will meet your needs.
In the example above, asking if marketing improves sales with the target customer segment might seem to carry an explicit “why” within the question. But if your reason for asking the question is to determine the ROI associated with the latest social media campaign, then you’ll want the analysts to identify the increase in sales attributable to the social media campaign and separate it from other marketing channels.
Make sure that the concepts captured in your research question are all clearly defined, especially the outcome. You might be surprised at how often we take conceptual shortcuts when describing complex thoughts in the workplace. It’s easy to assume that everyone in the discussion is on the same page or has all the necessary background information. In some cases, this might be true, but the more siloed your teams become, the less likely they are to have the same shared understanding as others in the organization.
As described above, if your interest is in calculating ROI, then asking if your marketing improved sales might not get you what you need. You would want to clarify that you need to know how much sales increased over what was expected before the marketing campaign went into place. Additionally, if your marketing is referring to a specific campaign, delivered through a specific medium, then that needs to be clarified as well.
Finally, if your marketing team developed multiple campaigns to target different segments within the same medium (e.g., by age groups using Facebook Ads), then you need to specify which target customer segment you are interested in. It may be that you want several separate answers to be able to calculate ROI on each campaign, for each customer segment, and within each medium, and that would need to be clear in the ask.
Finally, if you are looking for a specific type of relationship between concepts in your research question, then the type of relationship needs to be clear. In the example above, there doesn’t appear to be a specific relationship implied by the question. However, you might have a sense that the new marketing campaign would work better among your 45+ age demographic and be less effective at driving sales among those under 25. If that were the case, then you want to know more than just the incremental change in sales by age group. You want to know if the incremental change in sales is different across the demographic groups. This would require additional statistical testing after identifying the sales attributable to the new campaign.
What Is Your Population of Interest?
Identifying the specific population you want to study seems like a no-brainer. Adding clarity to this fundamental idea, however, even if only to state it out loud for everyone to hear, can save headaches down the road when getting your results back.
In our marketing example, it’s easy to assume that when we asked about improved sales, we meant sales volume or dollars. If calculating ROI is your intent, then it is a perfectly reasonable assumption. But what if our interest was in customer acquisition instead? Then, we would be more interested in the change in first-time customers during the period of the marketing campaign as compared to before the campaign. In this scenario, your population of observations would change from all customers to first-time customers, a subtle but important distinction.
When defining your population of interest, you’ll want to consider all of the defining characteristics of interest such as demographics (e.g., women, ages 25 – 34, BMW owners, etc.). Also be sure to consider any behavior-related characteristics such as purchases or opt-ins, and life events such as getting married, graduating from college, or having a child. Finally, while I’ve been discussing person-based populations, the same kinds of considerations are needed when dealing with event- or object-based populations such as production line defects, distribution interruptions, or customer service wait times.
Who Will Be the Final Consumer of the Results?
The ultimate audience for the analysis frequently has an obvious impact on how the results of analyses are presented. Internal audiences often do not require the same degree of polish and visualization that you might otherwise want for a public- or client-facing audience. Yet the impact of the final audience can go deeper than this.
Internal clients may only need rough results to have a quick answer to a question that is not mission critical. In contrast, for public-facing documents, a higher degree of sensitivity and robustness checking may be needed, requiring additional supporting analyses to dot all the I’s and cross all the T’s.
Understanding the wants and needs of the final analytic consumer will help your analysts draw lines around the extent of the analysis that needs to be completed to achieve success.
In What Format Do You Need the Results?
The way results are presented has an important impact on both the degree of polish in the final presentation, as well as the time required to produce results. Most analytic software will produce simple text-based results quickly and easily; however, the formatting of text results is often lacking and unfit for professional presentations. This is why they are often referred to as “raw” results.
If results need to be presented in professionally formatted tables or figures, then additional programming code will be needed to accomplish the task. Furthermore, because polished results are often specific to the analysis, the creation of formatted output may require customized code for each piece of output created.
In addition to tables and figures, your analytic team will need to know if they will be expected to provide any written interpretations of the results. This is an activity that the analytic team should be involved in, or at least provide input to, because they are the team closest to the analysis and with the greatest knowledge of any nuances or caveats that need to be acknowledged. To prepare the analysts for any writing components, be sure to let them know if they need to create write-ups that are in plain language for a non-technical audience, or if they need to provide a technical summary for a research-oriented audience.
What Is the Time Frame for Completion?
I may be dating myself, but do you remember the Staples Easy ButtonTM commercials? In 2005, the big red button was introduced to show how easy office work could be if you used Staples supplies and services. Wouldn’t it be great if we all had one of these for our analytic teams?
Unfortunately, analytics are not an easy button. Most analyses require time to develop plans, write code, test methodologies, and produce results. For those who do not write the code to execute the analyses, it is often surprising how long it can take to complete what sounds like a relatively simple analysis in a planning meeting. What takes 30 seconds to ask for might take a lot longer to complete.
Work with your analysts to scope out a realistic timeline for the completion of the analysis. While a complete and detailed plan is not needed to begin thinking about the timeline, set aside some time to develop a rough plan of what is needed in order to identify how long the analysis might take. Discuss any competing priorities that the analytic team might have coming up. Consider the degree of detail and rigor that is needed for the analysis and make any compromises that are needed if you need results sooner rather than later.
But please keep in mind that doing an analysis halfway up front does not necessarily mean the remainder of the full analysis can be completed in a shorter time frame on the back end. Sometimes it is best to take the time to complete the analysis you want at the outset instead of making deep compromises for speed that require additional rework down the road.
Conclusion
Your data analysis team can provide you with powerful insights to make decisions and drive strategic plans forward. There is, however, a finite limit to the power of your team. Manpower, computing resources, data availability, and competing priorities all act as limiting factors to constrain the amount of work your team can complete. Data requests that are poorly thought out, or conceived of as knee-jerk reactions to current circumstances, can place an additional burden on analytic teams. The resulting analyses are more likely to be poorly executed and require adjustments and rework to address overlooked considerations. By working through this list of analytic elements to consider before making analytic requests, you will be better positioned to get the results you want efficiently and well-executed.
