This isn’t a gloat post. In fact, I was completely oblivious to this massive outage until I tried to check my bank balance and it wouldn’t log in.
Apparently Visa Paywave, banks, some TV networks, EFTPOS, etc. have gone down. Flights have had to be cancelled as some airlines systems have also gone down. Gas stations and public transport systems inoperable. As well as numerous Windows systems and Microsoft services affected. (At least according to one of my local MSMs.)
Seems insane to me that one company’s messed up update could cause so much global disruption and so many systems gone down :/ This is exactly why centralisation of services and large corporations gobbling up smaller companies and becoming behemoth services is so dangerous.
Didn’t Crowdstrike have a bad update to Debian systems back in April this year that caused a lot of problems? I don’t think it was a big thing since not as many companies are using Crowdstrike on Debian.
Sounds like the issue here is Crowdstrike and not Windows.
They didn’t even bother to do a gradual rollout, like even small apps do.
The level of company-wide incompetence is astounding, but considering how organizations work and disregard technical people’s concerns, I’m never surprised when these things happen. It’s a social problem more than a technical one.
They didn’t even bother to test their stuff, must have pushed to prod
(Technically, test in prod)
Everyone has a test environment
Some are lucky enough to also have a separate production environment
A crowdstrike update killed a bunch of our Linux VMs that had a newer kernel a month or so ago.
*crowdstrike
@monoboy @Thorned_Rose yes we had the same problem on RHEL 9.4 around May 15, the falcon kernel module was incompatible with the new kernel, and crashed the kernel at boot. A fix was released on May 23…