this post was submitted on 08 Oct 2023
603 points (97.5% liked)
196
15693 readers
4763 users here now
Be sure to follow the rule before you head out.
Rule: You must post before you leave.
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
Even then, I wouldn’t want it to have any functionality to update the code it runs once it’s implanted. And I’d want that code to be incredibly well tested and verified alongside the hardware. No bugs beforehand means no reason to update it later.
No bugs is a hard thing to accomplish, especially for an immerging technology (eg 0-day vulnerability)
https://en.m.wikipedia.org/wiki/Therac-25
https://youtu.be/nU5HbUOtyqk
Not sure that counts? This was unfortunately due to a completely untested system, designed by one guy way over his head (ethically should have reported this to some governing body), and a company who lied about the non existent testing. This wasn’t just a singular bug but an entire failure throughout.
Yet, afterwards, the code running medical devices has been subject to the same standards that we set for tools themselves. The code embedded in a life support machine can’t fail.
I think you also proved my point anyway, the problem was a system set up such that testing wasn’t done. Not that the testing itself wasn’t possible. It’s just expensive. So companies won’t do it unless they’re forced too by regulation.
Ohhh, yeah. I have no idea why back then code wasn’t seen for what it is. I’ve been told by older people that back then the idea that if it compiles it’s fine, was ok… or something along those lines. I think today we even still of a ton of those issues due to every framework and language being so different, lacking standardization.
Throughout every thing I’ve ever learned, the biggest realization I’ve had was that without forcing policies, companies will do whatever is necessary to line their pockets.