You’re riding in a driverless car with a loved one. It’s cruising down a one-way, single-lane road with a barricade to the left and a wall to the right.
Just a few feet ahead, three pedestrians are hurrying across a crosswalk even though it’s flashing a red signal. The driverless vehicle’s wheels are faster than the pedestrians’ feet, and a collision is inevitable.
What should happen next? Should the vehicle know to swerve into a wall and sacrifice its passengers to avoid the pedestrians, or should it protect its passengers at all costs?
A new study, published in the journal Science on Thursday, presented that scenario, and others, to about 2,000 people in a series of six surveys.
The surveys revealed that the majority of respondents believed autonomous vehicles should be programmed to be “utilitarian,” attempting to save the most lives (in this case, the pedestrians) while sacrificing as few as possible (the passengers), said Jean-Francois Bonnefon, a psychological scientist at the Toulouse School of Economics in France and a co-author of the study.
Yet, most respondents also indicated that they would not want to purchase a vehicle that was programmed to be utilitarian.
The surveys revealed that 76% of respondents believed it is more moral for a driverless vehicle to sacrifice one passenger rather than 10 pedestrians when faced with such a scenario. However, 81% of respondents said they would rather own a car that protected them and their family members at all costs.
For the study, conducted between June and November, respondents were presented with crash scenarios as well as questions gauging their personal opinions about riding in an autonomous vehicle and their likelihood of buying one.
“We were surprised that so many people expressed a strong moral preference for cars that would kill them, as passengers, for the greater good,” he said. “[We were] even more surprised that so many people would renounce buying a driverless car if there was a regulation in place that would force them to buy the self-sacrificing cars that they morally approved of! People think that utilitarian cars are morally right, but they prefer to buy cars that protect them at all costs.”
The research team, which also included MIT professor Iyad Rahwan, noted that autonomous vehicles could help the environment, decrease traffic jams and reduce traffic accidents involving human error. However, we’re facing a “social dilemma” if most of us have a personal incentive to ride in a self-protective vehicle instead of a utilitarian one.
“There may indeed be stark differences in the difficulty of selling both models,” said Azim Shariff, assistant professor of psychology and social behavior at the University of California, Irvine and a co-author of the study.
“Though the public might overwhelmingly recognize the utilitarian model as the more morally appropriate to have on the roads, individual consumers will be — according to our data — significantly more drawn toward the self-protective ones,” he said. “As a result, without new regulation or changes in social norms, manufacturers will likely be more successful at selling the self-protective version.”
Jonathan Handel, an adjunct professor at the University of Southern California’s Gould School of Law, said that such regulation is needed now because driverless cars are already being programmed and are on the road.
“This is an insightful study that shows that the ‘not in my backyard’ problem extends to driverless cars. Call it the ‘not in my back seat’ problem,” said Handel, who was not involved in the new study. “People want manufacturers and society to do the right thing but not at their own expense … It’s a difficult problem, but one that can be addressed by regulation.”
A representative from the U.S. Department of Transportation’s National Highway Traffic Safety Administration said the administration plans to release updated guidelines on autonomous vehicles and automated safety technology this summer.
In April, NHTSA Administrator Mark Rosekind announced that the administration is committed to continuing a discussion about autonomous vehicle safety as well as other emerging technologies.
“NHTSA is developing operational guidance for the safe deployment of automated vehicles. This guidance will provide manufacturers and other stakeholders with guidelines for how NHTSA expects safe automated vehicles to behave in a variety of conditions,” Rosekind said.
“In this new era, what are the metrics by which we will measure the safety value and roadworthiness of new technologies?” he asked. “This is a question that we are asking all stakeholders. And let’s be honest; it’s not an easy question to answer. But it will be critical to understand as we are developing new ways to assess new technologies, and analyze their life-saving potential.”
