Hacker News new | past | comments | ask | show | jobs | submit login

Well, if no one is tracking it, there is no count and that would thus be countless.

But in all seriousness, I am curious about this as well.




It's not "countless" it's "uncountable" because we couldn't agree on what a software caused death was.

In my CS program we had an ethics class that included stories of bad X-ray machine software that overdosed people. Bad, bad bad. I don't think many people died as a direct result, but 10 years later there was probably a spiked cancer incidence. Did software kill people? Well yeah....kinda.

In airplane systems, there have been a number of cases where bad alerting / warning systems basically either misled the pilot or lulled them into a false sense of security prior to events that caused crashes. Did software kill people? Uhm, yes, I think, sort of, but not directly?

It's only when we get fully sentient AI that arms itself and decides to clean up the human pestilence that we'll be able to draw a straight line there. :)


Errors in the software of the radiation therapy machine Therac-25 directly lead to the death or serious injury of six patients. Three patients died within weeks or months. http://radonc.wdfiles.com/local--files/radiation-accident-th...

Errors in the software of a MIM-104 Patriot resulted in failure to locate and intercept an incoming missile and the death of 28 soldiers. http://www.gao.gov/products/IMTEC-92-26

Errors in the software of a Chinook helicopter may have lead to a crash that killed 29 people. http://www.publications.parliament.uk/pa/ld200102/ldselect/l...

Errors in the software of Toyota's throttle control system may have lead to the death of 89 people. https://betterembsw.blogspot.ch/2014/09/a-case-study-of-toyo...

More: http://www.baselinemag.com/c/a/Projects-Processes/Eight-Fata...


There was no error in the Toyota throttle control system. It was a pedal error with floor mats, and a separate pedal design error for other models.[1], combined with operator error. If you are interested, I highly recommend Malcolm Gladwell's podcast Revisionist History, which did an episode on this[2]. Long story short, your brakes can easily stop your engine at full open throttle, and in not much more space than braking without any throttle. Unfortunately, a foible of human behavior seems to lead us to get flustered and often not do the right thing in situations like these.

1: https://en.wikipedia.org/wiki/Sudden_unintended_acceleration...

2: http://revisionisthistory.com/episodes/08-blame-game


I agree that the pedal design, human error and floor mat problems are much more likely causes. I don't claim it was a software error for sure. My understanding is that indeed no specific software error was identified, but it also was never ruled out for sure. Neither of the two links seem to contain anything to that effect either.


To my memory, the podcast lays out a fairly comprehensive argument that it was just human error in the case of mechanical problems (as we'll as noting the car computers in all the cases show the brake wasn't pushed), and back it up with decades of research showing that this is a common problem, so that's about as close to definitive as you can get in this situation IMHO.


> Errors in the software of the radiation therapy machine Therac-25…

Caused by unsigned 8-bit integer overflow combined with (presumably) treating 0 as falsy. May or may not have been solved by using Rust (but probably wouldn't have because it sounds like a logic error in "clever" code).

> Errors in the software of a MIM-104 Patriot…

Caused by loss of precision in floating-point calculations. Would not have been solved by using Rust.

The other two were more likely human or mechanical than (unspecified) software errors.


Integer overflow is defined to panic in debug builds, and either do that or two's compliment overflow in release builds. The current implementation overflows. However, zero is not false.


I assume that treating 0 as false was intentional in this case (it probably was written in PDP-11 ASM) and a direct translation of the Therac code therefore would be

  if counter > 0 {
      deploy_radiation_shield();
  }


Yeah, you'd certainly have to see the actual code to be sure, I bet you're right.


How many people have died from terrible UIs in apps that are commonly used while driving? Obviously, personal responsibility is primary here, but I don't think the app designers and builders should be absolved of all guilt.

I was just thinking about this the other day when using Spotify in the car. Spotify's UI is pretty good, but after they lost all my saved songs, I've had to resort to only using playlists, which require 3x as many clicks to add a song.


Three deaths from the Therac-25: https://en.wikipedia.org/wiki/Therac-25.




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: