In almost every major accident, you’ll see systematic failures that meant that systems which were supposed to provide redundancy - both physical systems and management systems - were undermined, because they broke, their breaking didn’t have any immediate obvious negative consequence, and therefore it was assumed that they were unnecessary.
It’s a fallacy that you see time and time again - the “got away with it once; therefore it’s safe” fallacy. And it’s really hardwired into us - I’ve lost count of the number of times in conversation that someone has dismissed the threat of global warming by comparison to other bullets that we’ve dodged as a species: from nuclear war, to H5N1, to the Millennium Bug and CFCs.
We’re just... bad at risk analysis, on a fundamental level.
the comments in this discussion are well worth while
https://www.metafilter.com/179830/Normalization-of-Deviance
> In almost every major accident, you’ll see systematic failures that meant that systems which were supposed to provide redundancy - both physical systems and management systems - were undermined, because they broke, their breaking didn’t have any immediate obvious negative consequence, and therefore it was assumed that they were unnecessary.
>
> It’s a fallacy that you see time and time again - the “got away with it once; therefore it’s safe” fallacy. And it’s really hardwired into us - I’ve lost count of the number of times in conversation that someone has dismissed the threat of global warming by comparison to other bullets that we’ve dodged as a species: from nuclear war, to H5N1, to the Millennium Bug and CFCs.
>
> We’re just... bad at risk analysis, on a fundamental level.
the comments in this discussion are well worth while
https://www.metafilter.com/179830/Normalization-of-Deviance