Predictive analytics in the cloud
Ability to deliver ROI solutions more cost-effectively is driving cloud deployment.
BY JAMES TAYLOR
Decision Management Solutions (DMS) recently conducted research into predictive analytics in the cloud. Sponsored by FICO, Lityx and SAP, the research has at its core a survey of more than 350 respondents from a wide range of industries. Following up on a 2011 survey, the 2013 results make it clear that predictive analytics in the cloud is becoming increasingly mainstream, with broader and accelerating adoption.
The most striking result is that the number of companies reporting a positive impact from predictive analytics has risen dramatically since 2011. More than two-thirds of this year’s respondents have seen a positive impact from using predictive analytics in their business. It is also noticeable how much greater the reported impact is in 2013 relative to 2011. In 2013, many more companies reported transformative or significant impact than in 2011, while far fewer reported no usage or no plans as shown in Figure 1.
Figure 1: Increasing impact from predictive analytics.
Bucking the trend, 10 percent of the respondents said they still have no plans to implement analytics, and nearly a third have yet to put predictive analytics into production. As one respondent said, “[There is] still much user resistance to using [the] results of analytics. People still believe in the superiority of human judgment.”
Matching this rise in overall impact from predictive analytics is a similar rise in both current and planned deployment of predictive analytics in the cloud since 2011. The research divided predictive analytics in the cloud into three use cases:
1. Pre-packaged, cloud-based decision-making solutions that embed predictive analytics.
2. Cloud-based predictive modeling – building models in the cloud.
3. Cloud-based deployment of predictive analytics – scoring in the cloud.
These three scenarios leverage the scalability and pervasiveness of the cloud as well as the growing use of the cloud to deliver data. More than 60 percent of survey respondents said they had deployed at least one of these predictive analytics in the cloud use cases – a significant increase over 2011. As Figure 2 shows, an astonishing 90 percent said it was likely they would have at least one class of solution widely deployed in the next few years. Predictive analytics in the cloud is going mainstream and may, in fact, already be there.
Figure 2: Broad adoption of predictive analytics in the cloud.
The primary driver for the use of cloud-based solutions was reduced cost. Advanced analytic applications have historically been both very high ROI and very high cost. There has been constant pressure on the market to deliver ROI solutions more cost-effectively, and this is clearly driving cloud deployments of predictive analytics. The typical obstacles to predictive analytics also came through in the survey: data security and privacy, along with regulatory and compliance concerns, remain the primary obstacles reported. As one respondent said, “Cloud based solutions mean either storing or transmitting our proprietary data to the cloud. Although there are safe ways to do this, our management is not convinced.”
Predictive analytics has a strong history in credit risk and fraud detection. Recently, much of the market’s energy has been directed toward the use of predictive analytics for maximizing the opportunity from customer interactions, often positioned as cross-sell/up-sell. The big focus area for predictive analytics among respondents is in customer interaction; however, the particular focus of respondents was on customer satisfaction, customer retention and customer management rather than on increased sales. Many respondents use predictive analytics in marketing and cross-sell/up-sell, but the number one focus is using predictive analytics to improve customer engagement.
Given the importance of the cloud to big data, with so many new data sources being cloud based, it seemed appropriate to investigate the impact of big data on predictive analytics in the cloud. In particular, the survey explored the degree to which new data types (the variety aspect of big data) and “recent-cy” (the velocity aspect of big data) were impacting respondents.
When asked what data matters most to predictive analytic models, the vast majority of most respondents indicated it was what you might call traditional data types, and structured data from their own internal systems was by far the most important. The survey also revealed a definite sense that unstructured data from internal systems was becoming mainstream, while no other data types were deemed particularly important.
When more experienced analytic teams were separated out, however, and only those with existing deployments or significant impact were considered, the picture was quite different. These more experienced teams show much higher usage of new data types than in 2011. Social media, sensor, weblog, audio and image data types are all rated as much more important in analytic models among those with successful analytic deployments as shown in Figure 3. This probably reflects the use of new data types to improve the predictive power of existing models. Teams are still beginning largely with traditional data types, but they see increased value from new data types once they have some success. With more successful, more established teams using big data more broadly, it seems likely that there will soon be a rapid and significant growth in the use of new data types in building predictive analytics. More traditional structured data will likely remain broadly central to effective predictive analytic models. One survey respondent put it this way: “Big data is a misnomer as data has always been big. The challenge is making use of semi-structured and unstructured data in solutions. This will be the next giant leap forward in using data.”
Figure 3: More experienced practitioners use more data types.
The velocity of data also matters. Predictive analytics is increasingly focused on near real-time, operational data. This kind of data grew the most in importance between 2011 and 2013. This corresponds to the general shift in predictive analytics from batch scoring where results are stored back into a database for later use to real-time scoring. This shift is reflected in the increased use of intra-day and real-time data in predictive models. As one survey respondent put it: “Intra-day data will be the most valuable to our company since we are open 24 hours.”
Scoring streaming data is not yet a mainstream use case though it seems likely that the general shift to a more event-centric, real-time world will bring it squarely into the mainstream before too long.
Back in 2011 it was clear that early adopters were going to get an edge. They were more likely to have plans for broader deployment and saw predictive analytics in the cloud as more valuable. This trend strongly repeated in 2013. Once again, early adopters with one or more use cases deployed were significantly more likely to have plans to expand deployment. Similarly those with experience were likely to rate the value of each scenario more highly. Exposure to predictive analytics in the cloud breeds enthusiasm – those who buy into the promise of predictive analytics and get started like the results and want to do more. Therefore, organizations that get started quickly with predictive analytics in the cloud have the opportunity to create differentiation from slower-moving competitors.
One last result deserves a special call out. Recent years have seen increased interest in decision management or prescriptive analytics—the embedding of predictive analytics into operational decision-making systems. The importance and value of this trend was shown clearly in the survey results. More than 95 percent of survey respondents who adopted this approach (tightly integrating predictive analytics into operations) reported transformative or significant impact. Putting predictive analytics to work in operations is strongly correlated with the most impressive results as shown in Figure 4.
Figure 4: Decision management transforms results.
While a similar result was found in 2011, the percentage reporting the use of this approach has risen significantly since 2011 as shown in Figure 5. Decision management (i.e., prescriptive analytics, with its systematic embedding of predictions into automated decision-making systems) is an effective approach to maximize the transformative power of predictive analytics.
Figure 5: Decision management on the rise.
James Taylor (firstname.lastname@example.org) is CEO of Decision Management Solutions (http://www.decisionmanagementsolutions.com/). This article summarizes the key results of the study. For a full report, as well as a recording of a webinar summary, click here. He is a member of INFORMS.