Reminds me of the Therac-25 incident..
this post was submitted on 04 Dec 2023
825 points (98.1% liked)
Software Gore
5310 readers
156 users here now
Welcome to /c/SoftwareGore!
This is a community where you can poke fun at nasty software. This community is your go-to destination to look at the most cringe-worthy and facepalm-inducing moments of software gone wrong. Whether it's a user interface that defies all logic, a crash that leaves you in disbelief, silly bugs or glitches that make you go crazy, or an error message that feels like it was written by an unpaid intern, this is the place to see them all!
Remember to read the rules before you make a post or comment!
Community Rules - Click to expand
These rules are subject to change at any time with or without prior notice. (last updated: 7th December 2023 - Introduction of Rule 11 with one sub-rule prohibiting posting of AI content)
- This community is a part of the Lemmy.world instance. You must follow its Code of Conduct (https://mastodon.world/about).
- Please keep all discussions in English. This makes communication and moderation much easier.
- Only post content that's appropriate to this community. Inappropriate posts will be removed.
- NSFW content of any kind is not allowed in this community.
- Do not create duplicate posts or comments. Such duplicated content will be removed. This also includes spamming.
- Do not repost media that has already been posted in the last 30 days. Such reposts will be deleted. Non-original content and reposts from external websites are allowed.
- Absolutely no discussion regarding politics are allowed. There are plenty of other places to voice your opinions, but fights regarding your political opinion is the last thing needed in this community.
- Keep all discussions civil and lighthearted.
- Do not promote harmful activities.
- Don't be a bigot.
- Hate speech, harassment or discrimination based on one's race, ethnicity, gender, sexuality, religion, beliefs or any other identity is strictly disallowed. Everyone is welcome and encouraged to discuss in this community.
- The moderators retain the right to remove any post or comment and ban users/bots that do not necessarily violate these rules if deemed necessary.
- At last, use common sense. If you think you shouldn't say something to a person in real life, then don't say it here.
- Community specific rules:
- Posts that contain any AI-related content as the main focus (for example: AI “hallucinations”, repeated words or phrases, different than expected responses, etc.) will be removed. (polled)
You should also check out these awesome communities!
- Tech Support: For all your tech support needs! (partnered)
- Hardware Gore: Same as Software Gore, but for broken hardware.
- DiWHY - Questioning why some things exist...
- Perfect Fit: For things that perfectly and satisfyingly fit into each other!
founded 1 year ago
MODERATORS
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
can anyone with more insight explain what checks and balances had to have failed for this to make it to the field? i understand that this is like obviously potentially lethal but i don't really know how this kind of thing would normally be prevented.
Usually something in the testing process, or perhaps the testing process itself is lacking. For medical applications it should be pretty rigorous as the consequences if something slips through can be very bad.
If this is a new feature, then every step of the process designed to make sure it works failed. Which those are precisely will depend on the project, it could mean that multiple devs and QA had a look and either missed it or didn't think to test for it. Where I work the developer implementing a feature tests it, then 2 other developers review the code, one of them also tests it, then it goes to dedicated QA who will test it more in depth and also do regression tests (checking that existing functionality still works). The testing QA member also checks with another QA member about anything they may have missed in their test steps. But this can vary heavily, also depending on the general model of development cycle (agile or waterfall) etc - though I'm working on much less critical software, no ones going to get injured even if nothing works correctly.
If the bug was introduced through an update to this or another feature, their regression tests might be lacking.
It's also possible (though imo extremely negligient for such an application) that they don't have dedicated QA in the first place, and even don't require their devs to test comprehensively in place of dedicated QA.
Or, they found the bug, but management didn't want to allocate the resources to fix it.
Imo something like this slipping through shows negligience of some form, it's impossible to guarantee bug-free software, but this is not some obscure, hard to reproduce error.