I was sitting in a meeting with another manager. She gave her analytic team a set of instructions, some detailed and others somewhat vague. The young analysts across the table nodded and took notes. Then the manager asked the question, “How long will it take you to do this?” Both analysts looked like deer caught in headlights, and neither said a word. The analysts were too inexperienced to have a good sense of how long it would take to complete the project. The manager didn’t know all of the analytic nuances that would determine the timeline. To avoid this kind of awkward exchange on your team, here are five factors to consider in your analytic timelines.
How Clearly Defined Is the Question?
If you have been reading my posts for a while, you know that I believe a clearly defined question to be the most important piece of any analysis. How much clarity you put into your question dictates how much work needs to be done before the analysis can proceed. (See my post on 5 Things You Need to Tell Your Analysts Before They Begin Working)
Did you define all the concepts clearly, or will your team need to spend time figuring out which metrics make the most sense? Having a clearly defined outcome is particularly important because it sets the stage for how you interpret all the results. Additionally, if you test multiple metrics for a concept, then the time required for analysis will expand substantially compared to using a single metric.
You should include statements about any specific relationships you want to examine as part of your question. Perhaps you want to break out the analysis across different subgroups, like testing marketing effectiveness across customer age groups. Alternatively, you might want to know if the likelihood of a customer purchase changes at the same rate across the lifespan (i.e., a linear relationship), or does it accelerate or slow down (i.e., a non-linear relationship).
Finally, when you define your question, you also need to consider any underlying assumptions in the analysis that need to be tested for the results to be valid. Testing assumptions and checking the sensitivity of analyses to changes in specification can expand your timeline. When you consider the tests you need to perform in advance, you will also clarify the decision-making processes for interpreting the results. You will know which caveats about underlying assumptions you might need to consider.
Are the Data Available?
You need to determine where the data will be coming from as an important element in setting timelines. If you have the data in-house, will they be accessible to your analysts, or do you need to work with another team to gain access? You also need to consider whether the data are well-formed and clean, or will your analysts need to do a lot of work to prepare the data for the analysis?
If your organization does not already hold the data, are there external data sources that you need to obtain? If there are, then you need to consider the lead time for gaining access to those data.
Do you need to collect the data yourself, through a survey, interviews, document abstraction, or some other method? If your project requires collecting the data, you will need to think about how long that process will take.
Finally, when it comes to using external data, your analysts will need time to review the data files you receive, ensure that the data are what you expected to receive, and get familiar with the characteristics of the data. Your analysts may need a substantial amount of time, depending on the amount and complexity of the data.
Is There a Clearly Defined Methodology?
If you used an analytic project design process with your team (download our free guide for an example), you should have an idea of the methodology that you will be using for the primary analysis. As part of defining the methodology, you will need to discuss with your analysts the time necessary to complete each piece.
You may also have alternative methods to confirm that your results are not changed by the choice of methodology. The thoroughness of your analytic validation strategy will impact the overall timeline for completing the analysis.
Finally, you need to consider your selection of metrics for the analysis. Are your metrics already calculated and available as part of your data sources? If they aren’t, then consider how long it will take to calculate the metrics. Working through each of the variables you plan to use with your analytic team will help you estimate a timeline for execution.
Does Code Already Exist to Perform the Analysis?
Two of the most time-consuming tasks for your analysts are writing and debugging programming code. If you are lucky, then your team may have done the same or similar analyses in the past. Your analysts will have already written the code, or at the very least portions of the code will be available.
Depending on how mature the code is (i.e., how streamlined and automated), your analysts may only need to make updates to use new data files and not overwrite results from previous analyses. In these situations, the timeline for your current analysis should be substantially less than writing the code from scratch.
If your team has done a similar analysis in the past, then there should be code available to adapt as the base for your new project. The time needed for your analysts to adapt the code will depend on how extensive the edits are, and how much new code needs to be written.
If your analysts need to write the code entirely from scratch, this will typically be the longest development time. Ask your analysts to work through the process and outline of the code that needs to be written. The outline does not need to be highly detailed. The analysts should include all the key steps needed for cleaning, prepping, analyzing the data, and generating outputs. With an outline in hand, your analyst should be able to provide you with an estimated time for development.
Do Your Analysts Have the Necessary Skill Sets?
It might seem obvious that your analysts need to have the necessary skill sets to perform the analysis. It is not always easy to assess your analysts’ skill, however, and requires you to think beyond their programming knowledge.
If your analytic team has done this type of work before, this should shorten the development time. Even if only one analyst in the group has experience, they can serve as a mentor to the others. Regardless, you need to consider how long it will take your analysts to learn the necessary skills.
Do your analysts have competing priorities that will distract them from developing your new project and lengthen the timeline? It can be easy for managers to overlook the overall workload and timelines across projects for their analysts. If this happens, your analysts will experience additional stress and the chance for mistakes will increase.
You will need to consider the complexity of the proposed data and analysis. Your analysts will need more time if they must clean messy data, create a bunch of new variables, and estimate many statistical models.
The type of analytic validation your analysts use also plays a role in the analytic timeline. If you have two analysts working independently, your timeline will need to be longer. In contrast, a single analyst will typically complete the work faster, but with a higher chance of errors.
Finally, you must consider whether your analysts have the subject matter expertise necessary for the project. Are there nuances of the data that require the analysts to understand the processes that created the data in the first place? The more complicated the underlying processes are, the longer your analysts need to account for these subtleties.
Conclusion
Like any project, your analysis needs a timeline that is realistic. When you know the key factors influencing analytic timelines, you’ll have a solid foundation to start with. Additionally, when you discuss these factors with your analytic team you can better identify and mitigate challenges. With a thoughtful plan in place, your team will be set up to produce successful results.
