Elaine Herzberg

The Road to Conscious Machines
by Michael Wooldridge
Published 2 Nov 2018

On 18 March 2018, in Tempe, Arizona, a driverless car owned by Uber hit and killed a 49-year-old pedestrian, Elaine Herzberg, while in driverless mode. As is typically the way with accidents of this kind, there were a number of contributory causes. The car was travelling faster than the automatic emergency braking system could handle, so by the time the car recognized that emergency braking was required, it was too late to be able to do anything about it. Although the car’s sensors recognized that there was an ‘obstacle’ (the victim, Elaine Herzberg) which called for emergency braking, the software seems to have been designed to avoid doing this (suggesting some confusion, or at least, a rather strange set of priorities, in the mind of the programmers).

…

But, most importantly, the ‘safety driver’ in the car, whose main purpose was to intervene in incidents like this, appears to have been watching TV on her smartphone, paying little attention to the outside environment. It may well be that she was too confident in the car’s driverless abilities as well. The tragic death of Elaine Herzberg was entirely avoidable: but the failure was human, not technological. It seems depressingly inevitable that there will be many more tragedies like these before we see practical, mass-market driverless cars. We need to do everything we reasonably can to anticipate and avoid such tragedies. But they will occur in any case; and when they do, we need to learn the lessons from them.

…

The disengagement report must indicate how many miles the relevant car from which company drove in driverless mode, and how many disengagements occurred during these tests. A disengagement is a situation in which a person had to intervene to take over control of the car – what should have occurred in the case of Elaine Herzberg. A disengagement doesn’t necessarily mean that the person had to intervene to avoid an accident – far less a fatality – but nevertheless this data gives an indication of how well the technology is performing. The fewer disengagements per autonomous mile driven the better. In 2017, 20 companies filed disengagement reports with the State of California.

Right of Way: Race, Class, and the Silent Epidemic of Pedestrian Deaths in America
by Angie Schmitt
Published 26 Aug 2020

“State lawmakers gave Floridians the power to tax themselves to fix transportation, and when Tallahassee Republicans disagree with how voters use that power, they fight tooth and nail in court to avenge their political defeats,” he said.36 Chapter 8 Pedestrian Safety on the Technological Frontier Just outside downtown Tempe, Arizona, along a wide, dusty highway, there is a small memorial to the victim of the most famous pedestrian-vehicle crash in history. Two wooden crosses and some flowers stand alongside Mill Avenue, where the dense brick buildings of downtown Tempe thin out into desert and parking lots. Here is where Elaine Herzberg, a forty-nine-year-old homeless woman, was killed by a computer-piloted Uber car on March 18, 2018. She was the first pedestrian ever killed by a self-driving car. That night around 10 p.m., Herzberg had been walking her bike across the street in a dark location about 360 feet from a crosswalk.

…

The red Hyper-brand bike she was riding—the kind you can buy for one hundred dollars at Walmart—was loaded with plastic bags containing her belongings, which might help explain why the laser-based sensor system (or LiDAR, for light detection and ranging) in the Volvo SUV did not immediately recognize her as either a bicyclist or a pedestrian. Normally when a pedestrian is killed, investigators have to resort to unreliable witnesses and guesswork to reconstruct what happened—if anyone even bothers—but this was not an ordinary pedestrian death. The death of Elaine Herzberg was an international media event. The last moments of her life were recorded by a camera and turned over to police and federal investigators and then viewed by people all over the world. In this, the rarest of cases, everyone could see exactly what happened. The video footage, shot in black and white from the dashboard camera, shows the SUV proceeding down the dark desert highway.

…

A 2018 New York Times article highlighted a number of vigilante “attacks” on robot cars by enraged residents armed with sticks and knives. A Chandler, Arizona, man told the New York Times, for instance, “I don’t want to be their real-world mistake.”7 Autonomous vehicle testing does raise a lot of ethical questions. Arizonans—people like Elaine Herzberg—did not explicitly consent to have a potentially life-threatening technology beta-tested on them and their families. In other fields—pharmaceuticals, for example—companies are forbidden from testing potentially lethal new products on patients without their explicit consent. Self-driving car companies, meanwhile, are expected to police themselves.

Road to Nowhere: What Silicon Valley Gets Wrong About the Future of Transportation
by Paris Marx
Published 4 Jul 2022

The engineers had built in a one-second delay, so it did not start slowing until two milliseconds before it collided with the woman. By that time, Vasquez had looked up. She saw what was about to happen but could not stop the vehicle from plowing into the woman at nearly forty miles an hour. The bicyclist died at the hospital sometime later. Her name was Elaine Herzberg, and she was the first pedestrian reported to be killed by an autonomous vehicle. It is easy for a society conditioned to do so to place the blame for the crash on Herzberg. There was no crosswalk where she stepped out, a test found drugs in her system, and she is no longer around to defend herself.

…

Anyone hoping to fix the problems with the existing transportation system and the broader urban landscape that has evolved out of it must dig deeper into the roots of those problems instead of mistakenly believing that adding new technological solutions will address them. In the years following the death of Elaine Herzberg, there have also been concerns about the safety of Tesla’s Autopilot system. It was pitched as a fully self-driving system by Musk, but it is more of an assortment of assisted driving technologies like those that are present in many other vehicles, especially high-end cars and SUVs. As a result, a court in Munich ruled that Tesla could not even call the system “Autopilot” in Germany because it misled consumers about its real capabilities.

Ghost Road: Beyond the Driverless Car
by Anthony M. Townsend
Published 15 Jun 2020

Ford’s AVs, cruising around the college town of Ann Arbor, Michigan, in the summer of 2017, couldn’t even detect a crosswalk unless the intersection was equipped with a special signaling device. When an Uber AV became the first to fatally strike a pedestrian, in Tempe, Arizona, in the summer of 2018, a leading legal expert on AVs speculated that the company’s software probably “classified [Elaine Herzberg] as something other than a stationary object.” And a growing body of evidence indicates that computer-vision algorithms, like the ones employed in Tesla vehicles, may also be better at identifying whites than nonwhites. AV makers would love to see communities shift gears, from spending money on signs that make streets more legible for people, to rendering streets readable to self-driving machines instead.

…

Reising, “Legible London,” SEGD, November 26, 2009, https://segd.org/legible-london. 230people burn calories and spend at local shops: Lilli Matson, “Wayfinding and Walking in London,” Transport for London, May 2013, http://www.impacts.org/euroconference/vienna2013/presentations/London%20Walking%20-%20Vienna%20May%202013.pdf. 230“[The city] is seen in all lights and weathers”: Lynch, The Image of the City, 1. 231“the preprogrammed route was mapped”: Tim Faulkner, “Driverless Little Roady Shuttle Hits a Few Speed Bumps,” ecoRI News, August 3, 2019, https://www.ecori.org/transportation/2019/8/2/little-roady-shuttle-reaches-milestone-hits-speedbumps. 231couldn’t even detect a crosswalk: Ryan Stanton, “Can Self-Driving Pizza Delivery Cars Follow Ann Arbor’s Crosswalk Law?” Ann Arbor, MLive, October 6, 2017, https://www.mlive.com/news/ann-arbor/index.ssf/2017/10/can_ self-driving_pizza_deliver.html. 231“classified [Elaine Herzberg] as something other”: Ryan Felton, “Video Shows Driver in Autonomous Uber Was Looking Down Moments before Fatal Crash,” Jalopnik, March 21, 2018, https://jalopnik.com/video-shows-driver-in-fatal-autonomous-uber-crash-was-l-1823970417. 231computer-vision algorithms: Katyanna Quach, “Racist Self-Driving Scare Debunked, inside AI Black Boxes, Google Helps Folks Go with TensorFlow,” The Register, March 10, 2019, https://www.theregister.co.uk/2019/03/10/ai_ roundup_080319/. 231marking of pavement to help orient drivers: Bill Loomis, “1900–1930: The Years of Driving Dangerously,” Detroit News, April 26, 2015, https://www.detroitnews.com/story/news/local/michigan-history/2015/04/26/auto-traffic-history-detroit/26312107/. 232dinged for excessive idling: Andrew Small, “CityLab Daily: The Race to Code the Curb,” CityLab, April 2, 2019, https://www.citylab.com/authors/andrew-small/. 233“cities need to invest in the mapping in ways”: Kevin Webb (@kvnwebb), “We’re building smart ways to encode information like curb regulation,” Twitter, November 30, 2018, 1:41 a.m., https://twitter.com/kvnweb/status/1068485225607585798. 234“an exponential runaway beyond”: Vernor Vinge, “The Coming Technological Singularity: How to Survive in the Post-Human Era,” VISION-21 Symposium, NASA Lewis Research Center and Ohio Aerospace Institute, March 30–31, 1993, https://edoras.sdsu.edu/~vinge/misc/singularity.html. 234John von Neumann had raised: John Brockman, Possible Minds (New York: Penguin Press, 2019), 8. 234“AI enthusiasts have been making claims”: Vinge, “The Coming Technological Singularity.” 234“I have set the date 2045”: Christianna Reddy, “Kurzweil Claims That the Singularity Will Happen by 2045,” Futurism, October 5, 2017, https://futurism.com/kurzweil-claims-that-the-singularity-will-happen-by-2045/. 235“machine learning” to disguise the true nature: Chris Smith et al., “The History of Artificial Intelligence,” University of Washington, December 2006, https://courses.cs.washington.edu/courses/csep590/06au/projects/his tory-ai.pdf. 235“a deep level of understanding”: Rodney Brooks, “Post: [For&AI] The Origins of Artificial Intelligence,” Robots, AI, and Other Stuff (blog), April 27, 2018, https://rodneybrooks.com/forai-the-origins-of-artificial-intelligence/. 235“Contemporary neural networks do well on challenges”: Gary Marcus, “Deep Learning: A Critical Appraisal,” New York University, accessed January 22, 2019, https://arxiv.org/pdf/1801.00631.pdf. 236“If . . . driverless cars should also disappoint”: Marcus, “Deep Learning.” 236Frey tallied his predictions of the jobs: Carl Benedikt Frey and Michael A.

A Brief History of Motion: From the Wheel, to the Car, to What Comes Next
by Tom Standage
Published 16 Aug 2021

Disengagements are to be welcomed if they occur when the car realizes that it cannot handle the situation. AVs that misunderstand their surroundings and then fail to disengage can be deadly. That happened on March 18, 2018, when an Uber AV, being tested in Tempe, Arizona, struck and killed a pedestrian, Elaine Herzberg, as she wheeled her bicycle across a four-lane road. Although it was dark, the car’s radar and lidar detected her six seconds before the crash. But the perception system got confused: it classified her as an unknown object, then as a vehicle, and finally as a bicycle, whose path it could not predict.

Carmageddon: How Cars Make Life Worse and What to Do About It
by Daniel Knowles
Published 27 Mar 2023

Uber and Lyft, the taxi companies, had invested heavily in trying to develop the technology, on the basis that getting rid of the drivers was probably their best hope of becoming seriously profitable. But Uber abandoned its project in 2020 after spending $1 billion on it, selling its unit to a firm called Aurora. Lyft sold its own unit in 2021. An accident that killed a pedestrian probably pushed Uber to give up. Elaine Herzberg, a forty-nine-year-old, was wheeling a bicycle laden with shopping bags across the road when Uber’s adapted-Volvo plowed into her. Investigators allege that the driver, Rafaela Vasquez, who was meant to be monitoring the vehicle, had been streaming an episode of The Voice, the reality TV show, at the moment of the crash.

User Friendly: How the Hidden Rules of Design Are Changing the Way We Live, Work & Play
by Cliff Kuang and Robert Fabricant
Published 7 Nov 2019

Consider two examples from recent history: when a driverless Uber killed a pedestrian in Arizona while driving at night; and in Hawaii when, during a routine drill, a hapless employee sent a nuclear missile warning that reached tens of thousands of people. Uber had been testing its self-driving cars on the open road for a few years until the night of March 18, 2018, when, in Tempe, Arizona, one of them, driving forty miles per hour, killed Elaine Herzberg while she was crossing the street.23 A week later, the Tempe chief of police said she suspected that Uber was not at fault.24 The day after that, I saw a headline on my phone that read, “Woman Killed by Driverless Car Likely Homeless.” The inference wasn’t subtle: Maybe it was her fault, the homeless being who they are.

The Alignment Problem: Machine Learning and Human Values
by Brian Christian
Published 5 Oct 2020

Reliance on Newtonian mechanics did not make the troublesome perihelion of Mercury go away; it was still there, nagging at Einstein, two hundred years after Newton. In human affairs, however, this danger is very real. In the National Transportation Safety Board review of the self-driving Uber car that killed pedestrian Elaine Herzberg in Tempe, Arizona, in 2018, the analysis reveals that the “system never classified her as a pedestrian . . . because she was crossing . . . without a crosswalk; the system design did not include a consideration for jaywalking pedestrians.”39 We must take caution that we do not find ourselves in a world where our systems do not allow what they cannot imagine—where they, in effect, enforce the limits of their own understanding.