Share with your friends


Analytics Magazine

Disarming ‘Weapons of Math Destruction’

Eric Siegel, Ph.D., is founder of the Predictive Analytics World conference series and executive editor of The Predictive Analytics Times. A renowned speaker, a former Columbia University professor and leader in the field, Siegel is the author of the award-winning book, “Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie, or Die,” the second chapter of which is devoted to ethical issues that arise in predictive analytics’ deployment.By Eric Siegel

Cathy O’Neil, an industry insider and experienced expert, thoroughly covers the sociological downside of data science in her New York Times bestseller and first-of-its kind book, “Weapons of Math Destruction.”

In the world of big data, there’s a lot of music to be faced. With all its upside, data science’s deployment risks being prejudicial, predatory, exploitative, buggy, blindly trusted and secretive. And it has the potential to magnify the consumer’s personal economic struggle rather than remedy it.

These risks permeate across the field. The book’s broad coverage includes examples from all the main business application areas to which predictive models commonly apply: marketing, online ads, credit scoring, insurance, workforce analytics, law enforcement and political campaigns.

By providing such a uniquely comprehensive treatment of data’s downside, the book addresses two dire needs: increasing awareness and opening the door to prolific discussion.

When exercising the power of analytics, what could be more important than that? Establishing and managing ethical data science is as important as including the brakes in a state-of-the-art automobile. O’Neil’s book pioneers much-needed first steps.

Weapons of Math Destruction,” big data, social justice, ethics, Cathy O’Neil

Cathy O’Neil’s best-selling book explores sociological downside of data science.

Risks to Social Justice

“Weapons of Math Destruction” covers a range of ethical dilemmas. The sociological risks of data science include:

Magnifying the social divide. Predicting a poor credit risk is, to some degree, a self-fulfilling prophecy that can hurl those less financially secure into a vicious cycle. This pitfall applies analogously when prison sentencing may be guided in part by a felon’s neighborhood, and when a university places low on U.S. News & World Report college rankings. More generally, data science may amplify capitalism’s tendency to further disempower the disenfranchised, i.e., punish the poor.

Racial prejudice. Law enforcement models that predict crime and recidivism have been shown to intrinsically enact biases against minority groups. They are swayed in part by who you are rather than what you’ve done.

Predatory micro-targeting. Highly targeted online ads are more adept than ever at exploiting vulnerable consumers and separating them from their money.

Opaque, overly trusted and sometimes buggy. Whether buggy (in some demonstrable cases) or unjust, predictive models often are not disclosed transparently – hidden from scrutiny – and in some cases are overly trusted by those who rely on their predictions.

But Data Science is Largely Good

The book’s main shortcoming to keep in mind while you’re (hopefully) reading it is that – in most places – it appears to sweepingly indict and vilify data science.

Ultimately, that’s going too far. A little extra copy could have easily clarified as much up front in the introduction by saying something like, “The risks are dire, measures must be taken to protect social justice when deploying data science, and here are several examples along with suggested adjustments to implement such protective measures.”

Instead, the author begins the book by regaling her personal story of defecting completely from data science’s commercial practice as a generally unethical endeavor, and the remainder of the book covers a list of examples that will sound to many readers like no less than a diatribe. The book’s opening anecdote – on a flawed teacher evaluation metric – leaves out that, instead of doing away with the entire system, such a bug could in fact be remedied. Most mentions of micro-targeting tacitly imply it always necessarily serves greed or otherwise enacts some forms of injustice. The book focuses almost exclusively on the negative – depicting one train wreck after another – with relatively little to balance that out in the way of constructive advice that could actively mitigate risks to social justice within existing practices. The implicit advice is to basically turn it all off.

But, if you read the full book in detail, you will indeed come across this intelligent author stating that, no, math is not inherently evil, and it can also serve for the greater good. She goes further with the following two quotes, each relatively buried within the book (the latter is literally the last sentence of the last chapter before the Conclusion):

“… mathematical models can sift through data to locate people who are likely to face great challenges, whether from crime, poverty or education. It’s up to society whether to use that intelligence to reject and punish them – or to reach out to them with the resources they need. We can use the scale and efficiency that make ‘weapons of math destructions’ so pernicious in order to help people. It all depends on the objective we choose.”

“… with most ‘weapons of math destruction,’ the heart of the problem is almost always the objective. Change that objective from leeching off people to helping them, and a WMD is disarmed – and can even become a force for good.”

I would go further than O’Neil and claim that many deployments of predictive models, even when designed to pursue profit as the objective rather than social justice, do more good than bad. Profit by way of efficiency is not always a bad thing. Moreover, consumers gain value by way of predictive models: less junk mail (and better for the environment), more relevant ads, better movie, music and books recommendations, effective email spam filters, better Google search results, more engaging Facebook feed content, more robust healthcare, and increased safety by more effectively targeting the inspection of buildings and manholes.

This technology is like a knife: Its power can be used for good or for evil. That means it can be dangerous, but the idea of completely eliminating it – or even just its profit-driven deployment – is not on the table.

It’s important not to go too far and criticize the entire field of data science in absolute terms. By painting it completely black, you compromise your own credibility and weaken your valuable voice in the call for social justice.

Having said that, I strongly support O’Neil’s inspirational motion that you go out and pursue social justice rather than – or at least in addition to – profit.


While “Weapons of Math Destruction” conveys an oversimplifying, “black-and-white” position, that’s just one aspect of what amounts to a broad treatment of a multi-faceted, critical topic. I encourage you to look past that aspect in order to gain from this important book. I anointed it five stars on Amazon, and I strongly recommend you read it as well and pass on the word.

Eric Siegel, Ph.D., is founder of the Predictive Analytics World conference series and executive editor of The Predictive Analytics Times. A renowned speaker, educator (he’s a former Columbia University professor) and leader in the field, Siegel is the author of the award-winning book, “Predictive Analytics: The Power to Predict Who Will Click, Buy, Lie, or Die,” the second chapter of which is devoted to ethical issues that arise in predictive analytics’ deployment.

business analytics news and articles



Related Posts

  • 82
    My family and I are spending the second half of my sabbatical in Madrid. We arrived in Spain a couple of weeks ago, and the day we arrived I tried to log in to my bank’s online site to pay my credit card bill.
    Tags: models, neil, data, book, analytics, ethics, professional, science, big
  • 71
    Some years ago, I got a call from “Frank,” a finance director at a start-up company with a cloud-based software solution. Its platform was hosted by one of the large public cloud providers, and that was why he was calling. “The bills for hosting have been outrageous,” said Frank, who…
    Tags: data, good, ethics, professional, science, big
  • 64
    Cathy O’Neil’s provocative book, “Weapons of Math Destruction: How big data increases inequality and threatens democracy,” created quite a stir in the analytics community when it was released last fall.
    Tags: analytics, book, data, weapons, math, destruction, neil, big, models, predictive
  • 57
    By Eric Siegel Note: This article is excerpted from Eric Siegel’s foreword to the recently released book, “Mining Your Own Business: A Primer for Executives on Understanding and Employing Data Mining and Predictive Analytics,” by Jeff Deal and Gerhard Pilcher. For predictive analytics to work, two different species must cooperate…
    Tags: predictive, data, analytics, book, science
  • 57
    Eric Siegel, founder of Predictive Analytics World (PAW, a series of conferences held throughout the year in major U.S. and European cites) and the author of the new book, “Predictive Analytics: The Power to Predict who will Click, Buy, Lie or Die,” is without question a key player in the…
    Tags: analytics, predictive, data, book


Using machine learning and optimization to improve refugee integration

Andrew C. Trapp, a professor at the Foisie Business School at Worcester Polytechnic Institute (WPI), received a $320,000 National Science Foundation (NSF) grant to develop a computational tool to help humanitarian aid organizations significantly improve refugees’ chances of successfully resettling and integrating into a new country. Built upon ongoing work with an international team of computer scientists and economists, the tool integrates machine learning and optimization algorithms, along with complex computation of data, to match refugees to communities where they will find appropriate resources, including employment opportunities. Read more →

Gartner releases Healthcare Supply Chain Top 25 rankings

Gartner, Inc. has released its 10th annual Healthcare Supply Chain Top 25 ranking. The rankings recognize organizations across the healthcare value chain that demonstrate leadership in improving human life at sustainable costs. “Healthcare supply chains today face a multitude of challenges: increasing cost pressures and patient expectations, as well as the need to keep up with rapid technology advancement, to name just a few,” says Stephen Meyer, senior director at Gartner. Read more →

Meet CIMON, the first AI-powered astronaut assistant

CIMON, the world’s first artificial intelligence-enabled astronaut assistant, made its debut aboard the International Space Station. The ISS’s newest crew member, developed and built in Germany, was called into action on Nov. 15 with the command, “Wake up, CIMON!,” by German ESA astronaut Alexander Gerst, who has been living and working on the ISS since June 8. Read more →



INFORMS Computing Society Conference
Jan. 6-8, 2019; Knoxville, Tenn.

INFORMS Conference on Business Analytics & Operations Research
April 14-16, 2019; Austin, Texas

INFORMS International Conference
June 9-12, 2019; Cancun, Mexico

INFORMS Marketing Science Conference
June 20-22; Rome, Italy

INFORMS Applied Probability Conference
July 2-4, 2019; Brisbane, Australia

INFORMS Healthcare Conference
July 27-29, 2019; Boston, Mass.

2019 INFORMS Annual Meeting
Oct. 20-23, 2019; Seattle, Wash.

Winter Simulation Conference
Dec. 8-11, 2019: National Harbor, Md.


Advancing the Analytics-Driven Organization
Jan. 28–31, 2019, 1 p.m.– 5 p.m. (live online)


CAP® Exam computer-based testing sites are available in 700 locations worldwide. Take the exam close to home and on your schedule:

For more information, go to