Book: Weapons of math destruction

Author

Cathy O’Neil

Summary

A warning of the destructive power of black-box algorithms that govern our lives.

Takeaways

Algorithms and mathematical models are ubiquitously applied and drive decisions in every aspect of our lives. They, for example, determine college admissions, calculate insurance fees, and determine the content in our social media feeds. Many of the algorithms are intransparent, making it impossible to understand and challenge the results. If there is no alignment between the objectives of the models and the interest of the modeled subjects and if there is no feedback loop to improve the model over time, these “weapons of math destruction (WMD)” can cause significant harm.

Quotes

“The first question is: Even if the participant is aware of being modeled, or what the model is used for, is the model opaque, or even invisible?"

“That makes it extra hard to answer the second question: Does the model work against the subject’s interest? In short, is it unfair? Does it damage or destroy lives?"

“The third question is whether a model has the capacity to grow exponentially. As a statistician would put it, can it scale? This might sound like the nerdy quibble of a mathematician. But scale is what turns WMDs from local nuisances into tsunami forces, ones that define and delimit our lives."

“So to sum up, these are the three elements of a WMD; Opacity, Scale, and Damage."