Making analytics work through practical project management
Projects are the primary vehicles through which we deliver value, and consistently delivering value with analytics requires effective project management.
By Erick Wikum
Where does your organization fit into the project management spectrum shown in Figure 1? On one end of the spectrum, labeled “Do Stuff,” organizations focus on action and take a rather laissez-faire approach to project management, with little documentation and only loosely defined deliverables, timelines and budgets. On the other end of the spectrum, labeled “Buttoned Up,” organizations take a disciplined approach to planning, monitoring and executing projects, with liberal documentation and carefully defined deliverables, schedules and budgets. This approach is common among external consulting groups by necessity, since clients insist on having answers to the three key questions of project management: what is to be delivered, when and at what cost? The former approach is more common among internal consulting groups, especially those with corporate funding, which provide internal clients with what appear to be “free” resources.
Why should analytics professionals care about project management? After all, we are trained problem-solvers and when we “do stuff,” good things happen, right? On the contrary, projects are the primary vehicles through which we deliver value, and consistently delivering value with analytics requires effective project management. When it comes to delivering analytics, what matters is not only what you do, but also how you do it.
A project can be defined as “a piece of planned work or activity that is completed over a period of time and intended to achieve a particular aim” . Analytics projects begin with a purpose or aim – for example to describe, to predict or to prescribe. Achieving that purpose requires activity or work, work that takes time and consumes scarce resources. The word “intended” is significant; projects fail to achieve desired aims more frequently than we would like to admit. Project management provides an important mitigation strategy to improve the odds for success. What is your level of familiarity with the following basic project management concepts?
- Project phases
- Work breakdown structure (WBS)
- Issue management
- Risk management
- Communication plan
In case your project management IQ could use a boost, take advantage of the valuable resources – reference materials, training, peer groups and conferences – available through the Project Management Institute (PMI), the world’s premiere project management organization . In addition, befriend experienced project managers (PMs) who can provide valuable advice. One PM taught me to use discrete percentages when reporting progress against project activities: 0 percent if not started, 20 percent when started, 80 percent when complete except for “loose” ends, and 100 percent when completely finished. Another suggested that when planning a project, focus less on task duration and more on when tasks can be completed, treating completion dates as contractual and allowing individual team members to manage their own schedules and responsibilities.
Analytics projects are just that, projects, so general project management principles apply. And yet, analytics projects include unique aspects that require tailored approaches. For one, analytics projects involve specialized subject matter. Choosing an appropriate technique, defining corresponding tasks, estimating level of effort for those tasks and executing the project require analytics expertise. For another, analytics projects include particular types of uncertainty. Data availability and quality is frequently an issue.
Knowing what will happen when specific data meets math is unpredictable. Textbook techniques seldom apply directly to real-world problems, giving rise to the need for invention within analytics projects. Finally, change management can be especially challenging given the general level of discomfort among people with mathematics and math-based solutions.
A project management methodology provides a generic set of process steps that can be customized for a specific project. Leveraging a methodology eliminates the need to reinvent the wheel for each new project. Two methodologies are especially relevant for analytics projects – CRISP-DM and Scrum. While CRISP-DM or Cross Industry Standard Process-Data Mining was developed by an industry consortium to support data mining projects, the methodology readily translates to any analytics project. Scrum, a form of agile, was originally developed to support software development projects, but it also applies well to analytics projects.
As shown in Figure 2, CRISP-DM includes six phases – Business Understanding, Data Understanding, Data Preparation, Modeling, Evaluation and Deployment . While these phases may be executed one after another in a linear manner, the intent of the methodology is to support iteration. For example, after evaluating a model, a modeler might conclude that additional data is needed, giving rise to the need for additional business understanding, data understanding and data preparation. Deployment can involve implementing a model and/or publishing findings and recommendations depending on the nature of the project. The outer ring of the figure represents the cyclical nature of the lifecycle of analytics models .
Data Mining Project
The author applied CRISP-DM in a data mining project to develop anomaly detection models for mining machine sensor data (i.e., data mining on mining data). With a staff of about 15 people, analytics was a relatively small part of the overall project, which included more than 100 people. While the analytics “swim lane” was planned using CRISP-DM, the overall project was planned using a software development methodology. The analytics swim lane was synchronized with other pertinent swim lanes (primarily data and system) according to what was required and when with respect to required inputs and model outputs.
CRISP-DM worked well as an approach within the analytics team. With its iterative nature, though, the methodology posed challenges to track and report progress within the overall project. For example, the project manager found it confusing to learn that the analytics team had looped back to data understanding. In retrospect, conducting an initial round of business understanding and data understanding would have made sense, followed by time-boxed modeling iterations which encompass business understanding, data understanding, data prep, modeling and evaluation as needed.
Scrum is a form of agile that addresses complexity and risk by creating a product (e.g., an analytics model) incrementally through a series of short iterations known as sprints and teasing out requirements based on feedback from concrete deliverables . Requirements are captured as stories of the form “a user with a certain role wants to use the system to …” in a list known as the project backlog.
At the beginning of each sprint, the backlog is prepped and a sprint planning meeting conducted to select which stories will be pursued during the sprint based on priority, required level of effort and available development resources. During the sprint, the development team conducts a daily Scrum to review work completed the previous day and to be completed during the current day. At the tail end of each sprint, a tested, “shippable” version of the product is demonstrated during a sprint review meeting. The project backlog is updated based on feedback captured during the review meeting to initiate the next sprint (see Figure 3).
The author applied Scrum to an analytics project to develop integrated simulation and optimization models for planning oil pipeline terminal infrastructure. A team of four executed four three-week sprints. Despite lack of experience with Scrum, the team caught on quickly. The value of Scrum emerged during the first two sprint review meetings. The team expected and received feedback on its partially complete models. Unexpectedly, the team witnessed sidebar conversations among client employees, an airing of disagreements that arose in response to the demonstration of tangible models.
Resolving these disagreements contributed both to better models and to customer buy-in. The team found that three weeks was long enough to deliver significant new capabilities but short enough to provide little room for delays. By reviewing required weekly progress at the beginning of each week, the team was able to stay on track during each sprint. Having ready access to subject matter experts was critical to avoid delays.
The team adapted Scrum to its needs, for example, by using only one simple metric (stories completed) to track progress. Being flexible with project management is critical to success. Adjust the approach based on the size and complexity of a project and the trade-off between the cost and benefit of various practices.
Monitoring and Controlling
In the realm of project management, project plans receive an outsize amount of attention. Planning a project is certainly important, but monitoring and controlling during execution is equally important for success. At sufficiently frequent (e.g., weekly) review intervals, the team reports progress as well as issues. Successful organizations create an environment in which issues can be raised without risk of judgment or sanction. The sooner an issue is surfaced, the sooner that issue can be addressed, often by tapping into higher-level powers through escalation. Adherence to schedule, scope and budget are tracked carefully during project execution, and non-minor adjustments are made through a formal project change process.
Accurately estimating the number of resources and length of time to accomplish tasks is critical to project success. Underestimating required level of effort may result in project delays or costly overruns. Overestimating may result in a plan being rejected by the client since the project takes too long or costs too much.
When it comes to estimation, experience matters, so shadowing a seasoned analytics professional is helpful. So, too, is iteratively breaking down tasks into more easily estimated tasks. For example, to estimate how long “business understanding” will take, ask how this phase will be accomplished. Suppose that one of the business understanding approaches is to conduct a workshop. Now, conducting a workshop involves preparation and follow-up in addition to the workshop itself. Preparation might require two days, the workshop one day and follow-up two days for a total of five. Accurately estimating the time to conduct a workshop is relatively easy, while estimating the time to conduct the high-level, intangible activity of Business Understanding is not. In some cases, conducting a small pilot to understand the level of effort required for a task makes sense. For example, if custom analytic models are to be created for each of 100 manufacturing plants globally, then the model might first be created for one or two (perhaps the most complex) in order to understand overall effort needed.
The INFORMS Analytics Maturity Model assesses an organization’s analytics maturity in terms of three groups of four factors in the areas of organization, analytics capability and data & infrastructure . While project management is not specifically referenced in that model, a disciplined approach to project management is most certainly necessary for organizations to achieve analytics maturity, enabling the consistent delivery of quality work products. Adopting a more buttoned-up approach to project management rather than simply doing stuff can help your organization mature and achieve maximum return on its analytics investment.
Erick Wikum (firstname.lastname@example.org) is treasurer of the Analytics Society of INFORMS. For nearly 25 years, he has planned, conducted, monitored and led analytics projects to improve decision-making as both an internal and external consultant.
- 80FEATURES Putin vs. Western analysts Russia’s new approach to extending its influence necessitates new approaches to assessment. By Douglas Samuelson Making analytics work through practical project management Making analytics work: Why consistently delivering value requires effective project management. By Erick Wikum Crowdsourcing – Using the crowd: curated vs. unknown Using…
- 58Microsoft has awarded Accenture a seven-year contract extension to continue providing finance and accounting (F&A) and procurement business process services. Accenture will leverage artificial intelligence, automation and analytics capabilities to deliver better business insights and help Microsoft increase productivity and efficiency.
- 58A quick quiz: What is a good nine- or 10-letter description of the emerging interest in business analytics and big data that ends in “-al”? A choice that may come to mind for many is “hysterical.” This choice reflects frenzied excitement about opportunities for business analytics to solve problems often…
- 57Business intelligence (BI) and analytics programs are among the “hottest” curricula being developed at universities and colleges worldwide. Definitions vary on what entails business intelligence or analytics, and there doesn’t seem to be a universal BI/analytics conceptual framework that is being used by organizations and universities to develop their BI/analytics…
- 56November/December 2010 In their 2007 book, “Competing on Analytics: The New Science of Winning,” Tom Davenport and Jeanne Harris captured for many the powerful potential of analytics to provide organizations with a competitive advantage. The book’s title called analytics a “new” science, but concepts and terms such as “business analytics”…