Weapons of Math Destruction

by

Cathy O’Neil

Teachers and parents! Our Teacher Edition on Weapons of Math Destruction makes teaching easy.

Weapons of Math Destruction: Chapter 7: Sweating Bullets Summary & Analysis

Summary
Analysis
American workers have recently coined a new idea: “clopening.” A combination of the words “closing” and “opening,” clopening refers to when an employee works late closing one night and comes in early, sometimes just a few hours later, to open up shop the next morning. While having the same employee or employees close the store one night and open it the next morning makes logistical sense for employers, it can create stress and sleep-deprivation for workers. And because retail and food service schedules often arrive on short notice, some employees might only find out a day or two in advance that they have one or more clopenings coming up.
Here, O’Neil describes a “clopening”—a scheduling quirk that is arguably unfair for employees yet efficient for employers. Though it’s not yet clear how this example directly relates to WMDs, it illustrates the same prioritization of efficiency over fairness that’s common in the Big Data economy.
Themes
Humanity vs. Technology  Theme Icon
Fairness vs. Efficiency  Theme Icon
Irregular schedules and clopenings are both products of the Big Data economy. WMDs that treat workers “like cogs in a machine” create these trends and entrench them into the workplace. Scheduling used to be driven by human observation: if an employee at a family-run store noticed that there were no customers on Tuesday morning but a huge rush on Saturday afternoons, the shop might close Tuesdays and hire additional workers for the Saturday shift. But now, businesses use software to analyze customer traffic, determine exactly how many employees need to be in-store and when, and keep staffing (and spending) at a bare minimum. Gone are the days of student workers studying during downtime on the job—now, every moment of every workday is analyzed and scheduled for maximum efficiency.
By removing the human perspective from scheduling, technology has perhaps made things more cost-efficient for employers. But in the process, it’s made things unfair for workers. When every aspect of a business is optimized for maximum profit, the people who keep the business running become less important than the pursuit of efficiency.
Themes
Humanity vs. Technology  Theme Icon
Fairness vs. Efficiency  Theme Icon
Quotes
U.S. government data shows that over two-thirds of food service workers and over half of retail workers find out about scheduling changes with less than a week’s notice. When The New York Times ran a 2014 article about a Starbucks worker named Jannette Navarro—a single mother struggling to work her way through college—Starbucks promised to change its scheduling practices by eliminating clopenings. But a year later, Starbucks hadn’t made good on their word. Minimal staffing was baked into their company culture and operations. Inefficiency is a huge liability at chains like Starbucks, and individual managers could be punished for a downturn in revenue related to inefficient scheduling.
Once businesses start prioritizing efficiency over their employees’ well-being, it’s hard to stop. Some of these businesses, especially large ones like Starbucks, have models that only make sense profit-wise when workers are exploited so that the maximum profits can be achieved. In this way, O’Neil implies that WMDs are changing the face of global labor practices for the worse, creating irreparable damage. 
Themes
Humanity vs. Technology  Theme Icon
Fairness vs. Efficiency  Theme Icon
Modern-day scheduling technology is rooted in the discipline of applied mathematics called “operations research”—OR for short. Mathematicians used to use OR to help farmers plan crop plantings. During World War II, OR was used to help the U.S. and British militaries optimize their resources. After the war, OR was used in manufacturing and supply chain logistics, and now it underpins huge companies like Amazon, FedEx, and UPS. But these models exploit workers, bending their lives to unfair schedules. Optimization programs are everywhere now, and they’ve contributed to the creation of what O’Neil calls a “captive workforce.”
Even though scheduling software was originally created to help make workers’ lives better and easier, it has become a WMD. It’s widespread, it keeps workers in the dark, and it creates a feedback loop in which workers can’t better their lives because they’re beholden to long, irregular hours. Again, a model created to prioritize efficiency over every other metric has spun out of control, erasing consideration for humanity from its processes.
Themes
Humanity vs. Technology  Theme Icon
Discrimination in Algorithms  Theme Icon
Fairness vs. Efficiency  Theme Icon
Get the entire Weapons of Math Destruction LitChart as a printable PDF.
Weapons of Math Destruction PDF
These over-optimized schedules create anxiety, sleeplessness, and stress that keep workers down, no matter how often they switch jobs in search of a better system. And when workers’ schedules are in chaos, their children grow up without routines. In this way, WMDs are seriously affecting people that they shouldn’t even touch. Efficiency outweighs goodness and justice—and this, O’Neil asserts, is the very nature of capitalism. 
This passage shows how WMDs’ effects trickle down from generation to generation, derailing innocent people’s lives and creating dangerous feedback loops in the process. By prioritizing efficiency, these models are creating chaos and pain for workers—and for their families, as well. 
Themes
Humanity vs. Technology  Theme Icon
Discrimination in Algorithms  Theme Icon
Fairness vs. Efficiency  Theme Icon
In 2008, a company called Cataphora created a software system that used information gathered from employees’ corporate emails and messaging systems to determine what kind of workers they were. People who sent emails that others copied and pasted a lot, for instance, could be seen as ideas generators; other workers were the “neurons” that connected people and only transmitted information.
Again, O’Neil uses this anecdote to show how relentless attempts to optimize profits, single out efficient and productive workers, and cull any employees who aren’t massively efficient all the time are creating destruction and unfairness across the work force.
Themes
Humanity vs. Technology  Theme Icon
Fairness vs. Efficiency  Theme Icon
As this type of analysis became more popular in workplaces around the country, it started to have terrible consequences. Call-center employees were monitored so that their tones of voice and speech patterns could be analyzed for efficiency. And when the 2007 financial crisis hit, less-useful employees across the workforce were laid off based on the software’s determination of their work styles were useful enough. This was the result of “digital phrenology.”
The programs that facilitate invasions of privacy, like monitoring calls and emails, are WMDs. In particular, the example of these programs being used to lay off employees during the financial crisis shows how damaging these data-driven programs can be, since companies will do anything to stay afloat in tough times.
Themes
Humanity vs. Technology  Theme Icon
Fairness vs. Efficiency  Theme Icon
Quotes
People who get fired because of these kinds of algorithmic metrics don’t always deserve to lose their jobs—but because they do, the algorithm is reassured that it’s working, and its criteria become even more entrenched. Now, tech workers and creative types are often beholden to the same crude analysis and efficiency measures that dictate the lives of overworked retail and service workers.
Software designed to analyze employee efficiency is harming the modern workplace. The feedback loops that these WMDs create are unvetted and unreliable—yet they’re being used to excuse permanent changes in the ways people work and the standards to which they’re held.
Themes
Humanity vs. Technology  Theme Icon
Discrimination in Algorithms  Theme Icon
Fairness vs. Efficiency  Theme Icon
In 1983, the Reagan administration warned of a “rising tide of mediocrity” in American schools—SAT scores seemed to be plummeting. The administration’s report suggested that underperforming teachers were the cause of the drop, and that they needed to be weeded out. This alarm was the root of programs that would derail the lives of people like Sarah Wysocki. Teachers are workers too, and they are extremely vulnerable to WMDs.
Here, O’Neil switches topics to explain how today’s WMDs are rooted in the past. In this particular example, she draws a connection between educational practices in the 1980s and the present-day obsession with efficiency and optimization at the cost of fairness. In doing so, O’Neil implies that programs like IMPACT (which cost Wysocki her job) may seem modern and sophisticated, but they’re actually based on arguably outdated concepts like the Reagan administration’s “rising tide of mediocrity.”
Themes
Humanity vs. Technology  Theme Icon
Fairness vs. Efficiency  Theme Icon
When Tim Clifford, a middle school English teacher in New York City, scored abysmally on a model’s evaluation of his performance, he was devastated—but soon, he found out that many of the educators he’d worked with for years were scoring dangerously low, as well. He had tenure, so his job was spared—and the next year, his score shot from a miserable 6 to a brilliant 96. Clifford now knew for sure that the scores were arbitrary and “bogus”—yet they were threatening to ruin teachers’ lives.
Clifford’s experience shows how faulty and imprecise algorithms are toying with teachers’ livelihoods. These models claim to create greater efficiency and better outcomes for students, but their creators fail to ensure that the models are functioning fairly, let alone accurately.
Themes
Humanity vs. Technology  Theme Icon
Fairness vs. Efficiency  Theme Icon
Researchers and analysists following up on the Reagan-era panic in the later 1980s found that the initial outcry was unjustified. Earlier analysts had overlooked the fact that lots of factors (e.g., more students taking the test, and universities opening their doors to more diverse student bodies) were having an impact on changing scores. When these new researchers divided the scores up into subgroups based on economic status, they found that they weren’t dropping that sharply at all. The phenomenon behind this misinterpretation is called Simpson’s paradox, in which a body of data displays one trend when taken as a whole but shows the opposite trend when broken up into groups.
Again, emphasizing that the Reagan administration’s panic over a “rising tide of mediocrity” was uncalled for, O’Neil suggests that present-day evaluations based on this panic are inherently flawed. In addition, researchers’ failure to thoroughly dissect and analyze the data behind teachers’ performance scores means that teachers were unfairly punished.
Themes
Humanity vs. Technology  Theme Icon
Fairness vs. Efficiency  Theme Icon
Data, Transparency, and U.S. Democracy Theme Icon
Botched statistics led to Tim Clifford’s struggles, too. His scores were random. In the effort to make sure that teachers were being measured based on how much they were helping their students to improve, the value-added model being used to gauge success tried to predict what a student’s score would be and reward or warn teachers based on the gap between the expectation and the reality. But because teachers’ classes change every year—and because a class of 25 or 30 students isn’t a big enough data set, the result is a model that is essentially “noise.”
Value-added models are, like many other models and algorithms, aimed at efficiently quantifying a metric that’s difficult to measure: success and growth. But ultimately, these models are failures, because they simply can’t work the way they’re designed to. Yet with no oversight or regulation, these models are used to determine which teachers will keep their jobs and which won’t—so they’re prioritizing a false idea of efficiency over fairness.
Themes
Humanity vs. Technology  Theme Icon
Fairness vs. Efficiency  Theme Icon
Quotes
The scores Clifford and his colleagues were getting were meaningless—but the “bogus WMD” creating them was still gaining traction. The Obama administration sought to reform legislation that judged school districts based on test scores alone, creating a law that would let states turn around underperforming districts on their own terms. Even though strict correlation between a school’s test scores and its overall health is falling out of favor politically, lawmakers and school board officials still aren’t rejecting WMDs outright (or even recognizing that they’re unfair). Value-added modeling, Clifford grimly predicts, isn’t going anywhere anytime soon. Like inhumane scheduling software at major corporations, it’s just too entrenched.
Once WMDs start influencing various sectors of society in significant ways, it’s hard to eradicate them. Even federal legislation, at this point, is doing little to ensure that models and algorithms are performing the jobs they’re supposed to. In many different fields, WMDs are too widespread to be toppled.
Themes
Humanity vs. Technology  Theme Icon
Fairness vs. Efficiency  Theme Icon
Data, Transparency, and U.S. Democracy Theme Icon