AIQ: How People and Machines Are Smarter Together
by
Nick Polson
and
James Scott
Published 14 May 2018
It makes assumptions more important, because the consequences of a single bad assumption can be amplified a millionfold or more, as some machine keeps repeating the same bad decision over and over again. Said another way: AI allows the fruit of a poison tree to grow exponentially larger. When this happens, it’s usually because people have made poor choices in tending the soil. There are three prime ways in which this can happen: 1. Rage to conclude. 2. Model rust. 3. Bias in, bias out. To illustrate these themes, we’ll ask for a little bit of help from a midcentury American icon: Joe DiMaggio. Born in 1914 to a family of Italian immigrants in California, Joltin’ Joe DiMaggio would go on to become one of the greatest baseball players ever, and a man whose fame transcended his sport.
…
Soon almost every company, large and small, will rely on this kind of data to do business. In this new age, it’s essential that we calm the rage to conclude, by remembering that every unverified assumption is a placeholder—an approximation to be used, for better or worse, only until more data is available. Model Rust You’ve now seen how poor assumptions, embedded into the very DNA of a model, can result in dreadful mistakes. Models aren’t always born rotten, though. Sometimes they get that way from too much rust. One especially famous and ill-fated application of AI illustrates this phenomenon perfectly. This system went online in 2008 in the hopes of addressing an important public health problem, with both money and lives hanging in the balance.
…
If you maintain a model and regularly “season it” with new data—that’s the trial-and-error model-fitting process we talked about in chapter 2—it makes better predictions over time. But if you neglect the model—if you let a crust of old, blackened assumptions build up too far—then patina can easily turn to rust. Neglect it some more, and eventually the model just rots away into nothing. Flu Trends suffered from a serious case of model rust, verging on rot. To understand why, we spoke to Dr. Rosalind Eggo, an infectious-disease researcher at the London School of Hygiene & Tropical Medicine. She was quick to commend Google for having put such a rich data resource to use in the service of a public good, but she also thought that the manner in which Flu Trends missed the 2009 H1N1 pandemic should have raised a red flag.