Share with your friends


Analytics Magazine

Predictive analytics: Harnessing the power of big data

July/August 2013

Eric SiegelBy Eric Siegel

Every day’s a struggle. I’ve faced some tough challenges such as which surgery to get, how to invest for my business and even how to deal with identify theft. With so much stuff coming at me from all angles, daily prosperity relies on spam filters, Internet search engines, and personalized music and movie recommendations. My mailbox wonders why companies still don’t know me well enough to send less junk mail.

These predicaments matter. They can make or break your day, year or life. But what do they all have in common?

These challenges – and many others like them – are best addressed with prediction. Will the patient’s outcome from surgery be positive? Will the credit applicant turn out to be a fraudster? Will the investment fail? Will the customer respond if mailed a brochure?

There’s another angle. Beyond benefiting you and I as individuals, prediction bestows power upon an organization: Big business secures a competitive stronghold by predicting the future destiny and value of individual assets.

For example, in the mid-1990s, Chase Bank witnessed a windfall predicting mortgage outcome. By driving millions of transactional decisions with predictions about the future payment behavior of homeowners, Chase bolstered mortgage portfolio management, curtailing risk and boosting profit.

Introducing … the Clairvoyant Computer

Making such predictions poses a tough challenge. Each prediction depends on multiple factors: the various characteristics known about each patient, each homeowner and each e-mail that may be spam. How shall we attack the intricate problem of putting all these pieces together for each prediction?

The solution is machine learning; computers automatically discovering patterns and developing new knowledge by furiously feeding on modern society’s greatest and most potent unnatural resource: data.

Figure 1: The learning process.
Figure 1: The learning process.

Data can seem like such dry, uninteresting stuff. It’s a vast, endless regiment of recorded facts and figures. It’s the unsalted, flavorless residue deposited en masse as businesses churn away.

But the truth is that today’s big data embodies a priceless collection of experience from which to learn. Every medical procedure, credit application, Facebook post, movie recommendation, fraudulent act, spammy e-mail and purchase of any kind is encoded as data and warehoused. This veritable Big Bang delivers a plethora of examples so great in number only a computer could manage to learn from them.

This learning process discovers and builds on insightful gems such as:

  • Early retirement decreases your life expectancy.
  • Online daters more consistently rated as attractive receive less interest.
  • Vegetarians miss fewer flights.
  • Local crime increases after public sporting events.

Machine learning develops predictive capabilities with a form of number-crunching, a trial-and-error learning process that builds upon statistics and computer science. In commercial, industrial and government applications – in the real-world usage of machine learning to predict – it’s known as:
Predictive analytics — Technology that learns from experience (data) to predict the future behavior of individuals in order to drive better decisions.

Applied Prediction

“The powerhouse organizations of the Internet era, which include Google and Amazon … have business models that hinge on predictive models based on machine learning.”

– Professor Vasant Dhar, Stern School of Business, NYU

Every important thing a person does is valuable to predict, including: consume, work, love, procreate, vote, mess up, commit a crime and die. Here are some examples:

  • Prediction drives the coupons you get at the grocery cash register. U.K. grocery giant Tesco predicts which discounts will be redeemed in order to target more than 100 million personalized coupons annually at cash registers across 13 countries. This increased coupon redemption rates by a factor of 3.6 over previous methods.
  • Predicting mouse clicks pays off massively: websites predict which ad you’ll click in order to instantly choose which ad to show, driving millions in new-found revenue.
  • Netflix awarded $1 million to a team of scientists who best improved their recommendation system’s ability to predict which movies you will like.
  • Obama was re-elected in 2012 with the help of voter prediction. The campaign predicted which voters would be positively persuaded by campaign contact, and which would be inadvertently influenced to vote adversely. Acting on these predictions was shown to successfully convince more voters to choose Obama than traditional campaign targeting.
  • The leading career-focused social network, LinkedIn, predicts your job skills.
  • Online dating leaders, OkCupid and eHarmony predict which prospect on your screen would be the best bet at your side.
  • Target predicts customer pregnancy in order to market relevant products accordingly. Nothing foretells consumer need like predicting the birth of a new consumer.
  • Student essay grade prediction has been developed for automatic grading. The system grades as accurately as human graders.
  • Wireless carriers predict how likely it is you will cancel and defect to a competitor – possibly before you have even conceived a plan to do so – based on factors such as dropped calls, your phone usage, billing information and whether your contacts have already defected.
  • Wikipedia predicts which of its editors, who work for free to keep this priceless asset alive, are going to discontinue their valuable service.
  • Allstate Insurance tripled the accuracy of predicting bodily injury liability from car crashes based on the characteristics of the insured vehicle. This could be worth an estimated $40 million annually to the company.
  • At Stanford University, a machine learned to diagnose breast cancer better than human doctors by discovering an innovative method that considers a greater number of factors in a tissue sample.
  • Researchers predict your risk of death in surgery based on aspects of you and your condition in order to inform medical decisions.
  • Crime-predicting computers help decide who belongs in prison. To assist with parole and sentencing decisions, officials in states such as Oregon and Pennsylvania consult prognostic machines that assess the risk a convict will offend again.

Organizations of all kinds benefit by applying predictive analytics, since there’s ample room for operational improvement; organizations are intrinsically inefficient and wasteful on a grand scale. Marketing casts a wide net; “junk mail” is marketing money wasted and trees felled to print unread brochures. An estimated 80 percent of all e-mail is spam. Risky debtors are given too much credit. Applications for government benefits are backlogged and delayed.

With predictive analytics, millions of decisions a day determine whom to call, mail, approve, test, diagnose, warn, investigate, incarcerate, set up on a date and medicate. By answering this mountain of smaller questions, predictive analytics combats financial risk, fortifies healthcare, conquers spam, toughens crime-fighting, boosts sales and may in fact answer the biggest question of all: How can we improve the effectiveness of all these massive functions across business, government, healthcare, non-profit and law enforcement work?

Eric Siegel, Ph.D., is the founder of Predictive Analytics World ( and the author of “Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie, or Die,” from which this article was adapted with permission of the publishers, Wiley. Upcoming Predictive Analytics World conferences will be held in Boston, San Francisco, Chicago, Washington, D.C., Berlin and London. For more information about predictive analytics, see the Predictive Analytics Guide.

©2013 Eric Siegel. All rights reserved.

business analytics news and articles



Fighting terrorists online: Identifying extremists before they post content

New research has found a way to identify extremists, such as those associated with the terrorist group ISIS, by monitoring their social media accounts, and can identify them even before they post threatening content. The research, “Finding Extremists in Online Social Networks,” which was recently published in the INFORMS journal Operations Research, was conducted by Tauhid Zaman of the MIT, Lt. Col. Christopher E. Marks of the U.S. Army and Jytte Klausen of Brandeis University. Read more →

Syrian conflict yields model for attrition dynamics in multilateral war

Based on their study of the Syrian Civil War that’s been raging since 2011, three researchers created a predictive model for multilateral war called the Lanchester multiduel. Unless there is a player so strong it can guarantee a win regardless of what others do, the likely outcome of multilateral war is a gradual stalemate that culminates in the mutual annihilation of all players, according to the model. Read more →

SAS, Samford University team up to generate sports analytics talent

Sports teams try to squeeze out every last bit of talent to gain a competitive advantage on the field. That’s also true in college athletic departments and professional team offices, where entire departments devoted to analyzing data hunt for sports analytics experts that can give them an edge in a game, in the stands and beyond. To create this talent, analytics company SAS will collaborate with the Samford University Center for Sports Analytics to support teaching, learning and research in all areas where analytics affects sports, including fan engagement, sponsorship, player tracking, sports medicine, sports media and operations. Read more →



INFORMS Annual Meeting
Nov. 4-7, 2018, Phoenix

Winter Simulation Conference
Dec. 9-12, 2018, Gothenburg, Sweden


Making Data Science Pay
Oct. 29 -30, 12 p.m.-5 p.m.

Applied AI & Machine Learning | Comprehensive
Starts Oct. 29, 2018 (live online)

The Analytics Clinic
Citizen Data Scientists | Why Not DIY AI?
Nov. 8, 2018, 11 a.m. – 12:30 p.m.

Advancing the Analytics-Driven Organization
Jan. 28–31, 2019, 1 p.m.– 5 p.m. (live online)


CAP® Exam computer-based testing sites are available in 700 locations worldwide. Take the exam close to home and on your schedule:

For more information, go to