Share with your friends










Submit

Analytics Magazine

Analyze This!: Victimized by WMD

“They do not listen. Nor do they bend. They’re deaf not only to charm, threats and cajoling but also to logic.”

Vijay Mehrotra is a professor in the Department of Business Analytics and Information Systems at the University of San Francisco’s School of Management and a longtime member of INFORMS.By Vijay Mehrotra

My family and I are spending the second half of my sabbatical in Madrid. We arrived in Spain a couple of weeks ago, and the day we arrived I tried to log in to my bank’s online site to pay my credit card bill.

Quite unexpectedly, I received a message informing me that before logging on I would need to enter a verification code that had just been sent in a text message to my U.S. cell number. Alas, this number is temporarily turned off while I am overseas, so I was unable to log in. I wound up on the phone for more than an hour – at international telecomm rates! – with a customer service representative (who could not successfully help me) and then with her supervisor (who did ultimately help me get on to the online banking site only after having to change my login name). Feeling triumphant, I logged in to my bank’s site, paid my credit card bill and went to bed feeling that the matter had been resolved.

When I woke up the next morning, however, I had received an email from my bank’s fraud detection department saying that it: (a) suspected that the payment I made was fraudulent; (b) was canceling the payment; and (c) was suspending my online access until I called them. After spending another hour on another international phone call, I was informed that the payment could not be re-submitted and that my online access could not be restored due to a technical issue (“a known bug in the system,” the agent told me sheepishly).

Dr. Cathy O’Neil, author of the book “Weapons of Math Destruction” [1], would be quick to say that I had been victimized by classic WMDs, a term that she coined to describe applied mathematical models with the following attributes:

Opacity: It was not at all clear why my access to online banking had been revoked. Yes, I was accessing my account from outside of the country, but I had informed the bank of my travel plans several weeks before leaving. Not only could I not figure it out, but neither could the many bank employees who I spoke with over the phone. Could it have been my foreign last name? Note that my wife had no trouble accessing her accounts at the same bank from Spain.

To O’Neil, this is typical: “… these mathematical models were opaque, their workings invisible to all but the highest priests in their domain,” she writes.

Scale: My bank has tens of millions of customers worldwide. So, the same models that were being used to prevent me from accessing my account online and paying my bills are undoubtedly blocking other customers in other circumstances from accessing their accounts as well. As O’Neil points out, “scale is what turns WMDs from local nuisances to into tsunami forces.”

Damage: The potential cost to me of a late credit card payment, especially on a bill that included international plane flights for my entire family, was substantial. And despite an almost endless stream of apologies from the various bank employees over the phone, there was absolutely nothing either they or I could do about it.

This is just as infuriating to O’Neil as it was to me: “You cannot appeal to a WMD. That part of their fearsome power. They do not listen. Nor do they bend. They’re deaf not only to charm, threats and cajoling but also to logic.”

O’Neil’s book has a special focus on social justice (its subtitle is “How Big Data Increases Inequality and Threatens Democracy”). This focus is evident in her text and in many of the examples of WMDs that she describes, including credit scoring, payday lending, predictive policing [2], criminal recidivism, predatory online marketing, employee selection and staff scheduling. She convincingly argues that many of these models perniciously combine to reduce social mobility for the poor (while also enabling the wealthier classes with increasingly personalized options for all kinds of products and services). She sadly observes that, “Being poor in a world of WMDs is getting more and more dangerous and expensive.”

Two of O’Neil’s key criticisms of WMDs are the narrowness of their objectives and the absence of feedback. For example, when a for-profit university successfully targets students who then take out huge government-guaranteed loans before leaving school with no appreciable improvement in their job prospects, the institution views these models as “successful,” even though the cost to society – and to the unsuspecting students who bought into the sales pitch – are substantial. Moreover, information about situations where models have failed – for example, when an employee screening model filters out an individual who turns out to be extremely successful elsewhere – is too often simply not considered, resulting in models that codify what was reflected in their initial assumptions and inputs. Meanwhile, the book also describes many instances in which data is being captured and aggregated across multiple sources and through various middlemen (all of whom have their own narrow objectives and lack of feedback), propagating data errors and biases across models and industries.

Overall, O’Neil does an excellent job of describing very serious problems that have too often been ignored or discounted by the chorus of analytics cheerleaders, this writer included. Solutions, however, are harder to pin down. In the book’s final chapter, she offers some suggestions, including self-policing, richer models that measure broader metrics, changes in laws surrounding the use of data (including updates to the Fair Credit Reporting Act, the Americans with Disability Act, and the Health Insurance Portability and Accountability Act to prohibit discrimination that is driven by predictive models), calls for businesses to make their models more transparent, and academic research using agent-based simulations to help provide an understanding of the logic underlying WMDs and of the results that they produce.

Ultimately, O’Neil clearly believes that these problems can only be addressed with the active participation of analytics professionals, exhorting us to “come together to police these WMDs, to tame and disarm them.” And my sense is that it will require not only a great deal of intellectual firepower, but also a clear understanding of one’s own values and a great deal of courage. Kudos to O’Neil – a mathematician whose career has included stints as a college professor, Wall Street quant and data scientists – for sounding the alarm. We are all challenged to answer its call.


Vijay Mehrotra (vmehrotra@usfca.edu) is a professor in the Department of Business Analytics and Information Systems at the University of San Francisco’s School of Management and a longtime member of INFORMS.

References & Notes

  1. https://www.goodreads.com/book/show/28186015-weapons-of-math-destruction
  2. For more on this, see http://analytics-magazine.org/analyze-this-a-silver-lining-for-election-blues/

Analytics data science news articles

Analytics data science news articlesSave

Save

Related Posts

  • 82
    Cathy O’Neil, an industry insider and experienced expert, thoroughly covers the sociological downside of data science in her New York Times bestseller and first-of-its kind book, “Weapons of Math Destruction.” In the world of big data, there’s a lot of music to be faced. With all its upside, data science’s…
    Tags: data, book, science, models, analytics, neil, professional, ethics, big
  • 81
    Some years ago, I got a call from “Frank,” a finance director at a start-up company with a cloud-based software solution. Its platform was hosted by one of the large public cloud providers, and that was why he was calling. “The bills for hosting have been outrageous,” said Frank, who…
    Tags: data, ethics, professional, science, big
  • 61
    Many organizations have noticed that the data they own and how they use it can make them different than others to innovate, to compete better and to stay in business. That’s why organizations try to collect and process as much data as possible, transform it into meaningful information with data-driven…
    Tags: data, science, big
  • 49
    July/August 2013 O.R. vs. analytics … and now data science? By Brian Keller In a 2010 survey [1], members of the Institute for Operations Research and the Management Sciences (INFORMS) were asked to compare operations research (O.R.) and analytics. Thirty percent of the respondents stated, “O.R. is a subset of…
    Tags: data, science, analytics
  • 48
    The winners of the first edition of the Dutch Data Science Awards, recently announced in Haarlem, included Quantib, data visualization expert Stef van den Elzen, AgroEnergy and CQM. AIMMS and AIMMS implementation partner ORTEC sponsored the awards along with EY, Lubbers De Jong, Microsoft and Motivaction. 
    Tags: data, science, big, analytics

Analytics Blog

Electoral College put to the math test


With the campaign two months behind us and the inauguration of Donald Trump two days away, isn’t it time to put the 2016 U.S. presidential election to bed and focus on issues that have yet to be decided? Of course not.


Save

Headlines

Gartner: AI technologies to be pervasive in new software products

Market hype and growing interest in artificial intelligence (AI) are pushing established software vendors to introduce AI into their product strategy, creating considerable confusion in the process, according to Gartner, Inc. Analysts predict that by 2020, AI technologies will be virtually pervasive in almost every new software product and service. Read more →

Drone delivery: Professor develops solution to minimize delays in operations

When delivery companies like FedEx, Amazon and UPS launch drones to deliver packages in the near future, one Kennesaw State University computer science professor may be at the crux of solving one of its most complicated problems. Donghyun (David) Kim, assistant professor of computer science and an expert in computer algorithm optimization, is designing a fast-running algorithm to tackle simultaneous coordination problems among multiple delivery trucks and the drones launched from them. Read more →

Tech spending growth limited to about 5 percent through 2018

Forrester predicts U.S. business and government tech spending will continue to grow by 4.8 percent through 2017 and increase to 5.2 percent in 2018. While these forecasts are higher than Forrester’s projections following the 2016 presidential election, they are lower than the expected numbers from a year ago. Read more →

UPCOMING ANALYTICS EVENTS

INFORMS-SPONSORED EVENTS

Essential Practice Skills for High-Impact Analytics Projects
Sept. 26-27, Executive Conference Center, Arlington, Va.

Foundations of Modern Predictive Analytics
Oct. 2-3, VT Executive Briefing Center, Arlington, Va.

2017 INFORMS Annual Meeting
October 22-25, 2017, Houston

2017 Winter Simulation Conference (WSC 2017)
Dec. 3-6, 2017, Las Vegas

CAP® EXAM SCHEDULE

CAP® Exam computer-based testing sites are available in 700 locations worldwide. Take the exam close to home and on your schedule:


 
For more information, go to 
https://www.certifiedanalytics.org.