Share with your friends










Submit

Analytics Magazine

Analyze This!: Victimized by WMD

“They do not listen. Nor do they bend. They’re deaf not only to charm, threats and cajoling but also to logic.”

Vijay Mehrotra is a professor in the Department of Business Analytics and Information Systems at the University of San Francisco’s School of Management and a longtime member of INFORMS.By Vijay Mehrotra

My family and I are spending the second half of my sabbatical in Madrid. We arrived in Spain a couple of weeks ago, and the day we arrived I tried to log in to my bank’s online site to pay my credit card bill.

Quite unexpectedly, I received a message informing me that before logging on I would need to enter a verification code that had just been sent in a text message to my U.S. cell number. Alas, this number is temporarily turned off while I am overseas, so I was unable to log in. I wound up on the phone for more than an hour – at international telecomm rates! – with a customer service representative (who could not successfully help me) and then with her supervisor (who did ultimately help me get on to the online banking site only after having to change my login name). Feeling triumphant, I logged in to my bank’s site, paid my credit card bill and went to bed feeling that the matter had been resolved.

When I woke up the next morning, however, I had received an email from my bank’s fraud detection department saying that it: (a) suspected that the payment I made was fraudulent; (b) was canceling the payment; and (c) was suspending my online access until I called them. After spending another hour on another international phone call, I was informed that the payment could not be re-submitted and that my online access could not be restored due to a technical issue (“a known bug in the system,” the agent told me sheepishly).

Dr. Cathy O’Neil, author of the book “Weapons of Math Destruction” [1], would be quick to say that I had been victimized by classic WMDs, a term that she coined to describe applied mathematical models with the following attributes:

Opacity: It was not at all clear why my access to online banking had been revoked. Yes, I was accessing my account from outside of the country, but I had informed the bank of my travel plans several weeks before leaving. Not only could I not figure it out, but neither could the many bank employees who I spoke with over the phone. Could it have been my foreign last name? Note that my wife had no trouble accessing her accounts at the same bank from Spain.

To O’Neil, this is typical: “… these mathematical models were opaque, their workings invisible to all but the highest priests in their domain,” she writes.

Scale: My bank has tens of millions of customers worldwide. So, the same models that were being used to prevent me from accessing my account online and paying my bills are undoubtedly blocking other customers in other circumstances from accessing their accounts as well. As O’Neil points out, “scale is what turns WMDs from local nuisances to into tsunami forces.”

Damage: The potential cost to me of a late credit card payment, especially on a bill that included international plane flights for my entire family, was substantial. And despite an almost endless stream of apologies from the various bank employees over the phone, there was absolutely nothing either they or I could do about it.

This is just as infuriating to O’Neil as it was to me: “You cannot appeal to a WMD. That part of their fearsome power. They do not listen. Nor do they bend. They’re deaf not only to charm, threats and cajoling but also to logic.”

O’Neil’s book has a special focus on social justice (its subtitle is “How Big Data Increases Inequality and Threatens Democracy”). This focus is evident in her text and in many of the examples of WMDs that she describes, including credit scoring, payday lending, predictive policing [2], criminal recidivism, predatory online marketing, employee selection and staff scheduling. She convincingly argues that many of these models perniciously combine to reduce social mobility for the poor (while also enabling the wealthier classes with increasingly personalized options for all kinds of products and services). She sadly observes that, “Being poor in a world of WMDs is getting more and more dangerous and expensive.”

Two of O’Neil’s key criticisms of WMDs are the narrowness of their objectives and the absence of feedback. For example, when a for-profit university successfully targets students who then take out huge government-guaranteed loans before leaving school with no appreciable improvement in their job prospects, the institution views these models as “successful,” even though the cost to society – and to the unsuspecting students who bought into the sales pitch – are substantial. Moreover, information about situations where models have failed – for example, when an employee screening model filters out an individual who turns out to be extremely successful elsewhere – is too often simply not considered, resulting in models that codify what was reflected in their initial assumptions and inputs. Meanwhile, the book also describes many instances in which data is being captured and aggregated across multiple sources and through various middlemen (all of whom have their own narrow objectives and lack of feedback), propagating data errors and biases across models and industries.

Overall, O’Neil does an excellent job of describing very serious problems that have too often been ignored or discounted by the chorus of analytics cheerleaders, this writer included. Solutions, however, are harder to pin down. In the book’s final chapter, she offers some suggestions, including self-policing, richer models that measure broader metrics, changes in laws surrounding the use of data (including updates to the Fair Credit Reporting Act, the Americans with Disability Act, and the Health Insurance Portability and Accountability Act to prohibit discrimination that is driven by predictive models), calls for businesses to make their models more transparent, and academic research using agent-based simulations to help provide an understanding of the logic underlying WMDs and of the results that they produce.

Ultimately, O’Neil clearly believes that these problems can only be addressed with the active participation of analytics professionals, exhorting us to “come together to police these WMDs, to tame and disarm them.” And my sense is that it will require not only a great deal of intellectual firepower, but also a clear understanding of one’s own values and a great deal of courage. Kudos to O’Neil – a mathematician whose career has included stints as a college professor, Wall Street quant and data scientists – for sounding the alarm. We are all challenged to answer its call.


Vijay Mehrotra (vmehrotra@usfca.edu) is a professor in the Department of Business Analytics and Information Systems at the University of San Francisco’s School of Management and a longtime member of INFORMS.

References & Notes

  1. https://www.goodreads.com/book/show/28186015-weapons-of-math-destruction
  2. For more on this, see http://analytics-magazine.org/analyze-this-a-silver-lining-for-election-blues/

Analytics data science news articles

Analytics data science news articlesSave

Save

Related Posts

  • 82
    Cathy O’Neil, an industry insider and experienced expert, thoroughly covers the sociological downside of data science in her New York Times bestseller and first-of-its kind book, “Weapons of Math Destruction.” In the world of big data, there’s a lot of music to be faced. With all its upside, data science’s…
    Tags: data, book, science, models, analytics, neil, professional, ethics, big
  • 81
    Some years ago, I got a call from “Frank,” a finance director at a start-up company with a cloud-based software solution. Its platform was hosted by one of the large public cloud providers, and that was why he was calling. “The bills for hosting have been outrageous,” said Frank, who…
    Tags: data, ethics, professional, science, big
  • 61
    Many organizations have noticed that the data they own and how they use it can make them different than others to innovate, to compete better and to stay in business. That’s why organizations try to collect and process as much data as possible, transform it into meaningful information with data-driven…
    Tags: data, science, big
  • 49
    July/August 2013 O.R. vs. analytics … and now data science? By Brian Keller In a 2010 survey [1], members of the Institute for Operations Research and the Management Sciences (INFORMS) were asked to compare operations research (O.R.) and analytics. Thirty percent of the respondents stated, “O.R. is a subset of…
    Tags: data, science, analytics
  • 48
    July/August 2014 The story of how IBM not only survived but thrived by realizing business value from big data. By (l-r) Brenda Dietrich, Emily Plachy and Maureen Norton This is the story of how an iconic company founded more than a century ago, and once deemed a “dinosaur” that would…
    Tags: analytics, data, big


Headlines

Fighting terrorists online: Identifying extremists before they post content

New research has found a way to identify extremists, such as those associated with the terrorist group ISIS, by monitoring their social media accounts, and can identify them even before they post threatening content. The research, “Finding Extremists in Online Social Networks,” which was recently published in the INFORMS journal Operations Research, was conducted by Tauhid Zaman of the MIT, Lt. Col. Christopher E. Marks of the U.S. Army and Jytte Klausen of Brandeis University. Read more →

Syrian conflict yields model for attrition dynamics in multilateral war

Based on their study of the Syrian Civil War that’s been raging since 2011, three researchers created a predictive model for multilateral war called the Lanchester multiduel. Unless there is a player so strong it can guarantee a win regardless of what others do, the likely outcome of multilateral war is a gradual stalemate that culminates in the mutual annihilation of all players, according to the model. Read more →

SAS, Samford University team up to generate sports analytics talent

Sports teams try to squeeze out every last bit of talent to gain a competitive advantage on the field. That’s also true in college athletic departments and professional team offices, where entire departments devoted to analyzing data hunt for sports analytics experts that can give them an edge in a game, in the stands and beyond. To create this talent, analytics company SAS will collaborate with the Samford University Center for Sports Analytics to support teaching, learning and research in all areas where analytics affects sports, including fan engagement, sponsorship, player tracking, sports medicine, sports media and operations. Read more →

UPCOMING ANALYTICS EVENTS

INFORMS-SPONSORED EVENTS

INFORMS Annual Meeting
Nov. 4-7, 2018, Phoenix

Winter Simulation Conference
Dec. 9-12, 2018, Gothenburg, Sweden

OTHER EVENTS

Making Data Science Pay
Oct. 29 -30, 12 p.m.-5 p.m.


Applied AI & Machine Learning | Comprehensive
Starts Oct. 29, 2018 (live online)


The Analytics Clinic
Citizen Data Scientists | Why Not DIY AI?
Nov. 8, 2018, 11 a.m. – 12:30 p.m.


Advancing the Analytics-Driven Organization
Jan. 28–31, 2019, 1 p.m.– 5 p.m. (live online)


CAP® EXAM SCHEDULE

CAP® Exam computer-based testing sites are available in 700 locations worldwide. Take the exam close to home and on your schedule:


 
For more information, go to 
https://www.certifiedanalytics.org.