Choices have consequences.  Businesses have centralised the management of their systems so they can reduce the costs of having many systems and more speedily and reliably implement software updates.  They then outsource these software updates to external third parties - in this case, Crowdstrike.  Crowdstrike look after many businesses, and so when they get a cold they all suffer. When there is a critical mass of businesses connected to the third party, you get a rolling cascade of IT failures.

Businesses make decisions according to costs, benefits, and risks.  However, while we’re very good at counting the costs and OK at identifying the benefits, we are often blindsided by ‘unknown unknowns’.   We don’t think about the broader consequences of our choosing new software and don’t recognise the risk impact.

Boards receive reports of well-known and easy-to-measure risks, but often don't receive reports that properly measure the potential consequences of IT risks that can completely shut down a business - or many businesses. Basically, we don’t recognise and measure new (or rare) risks well.  Directors and managers need to improve their understanding of  risks arising from their decisions, and make sure that they are measuring and report.

It is too early to come to any conclusion, however, it seems the organizations using CrowdStrike Falcon are experiencing the outages. Unless there are any ransom notes coming from any of the notorious ransomware groups linked to these outages, we should remain patient and wait to hear from CrowdStrike which is a trusted company providing services worldwide.

Today’s technology outage—an unprecedented global crisis—sparked off in the USA, is now ominously rippling across the globe. This sudden, severe disruption halts everyday activities and starkly exposes the fragility of our heavily digitised world. From banking to healthcare, education to government, no sector remains untouched, highlighting an urgent need for a worldwide strategic overhaul of our critical infrastructures. This crisis calls for immediate collaborative action to enhance resilience through robust safeguards and fail-safes, especially in life-critical networks. As we increasingly pivot to a future dominated by digital and AI innovations, this outage is a resounding wake-up call: we must fortify our digital bastions to safeguard against such catastrophic interruptions, ensuring our readiness and security in an interconnected era. 

As a result of this outage, at least three critical sectors could be affected significantly.

In the medical industry, a technology outage can result in the loss of access to electronic medical records, critical patient data, and communication systems essential for patient care. This could delay surgeries, medication administration, and emergency responses, potentially endangering lives. 

In the banking sector, an outage can cripple financial transactions, including ATM withdrawals, online banking, and payment processing. This disruption can lead to significant financial losses for consumers and institutions, and undermine public trust in the financial system

For the airline industry, technology outages can ground flights, disrupt ticketing and check-in processes, and affect air traffic control. This can lead to massive delays, financial losses, and compromise passenger safety and security. Each of these scenarios highlights the catastrophic potential of technology failures across critical industries.

Today’s event should serve as a crucial wake-up call.

What's happened today is that an update to a thing called Falcon Sensor, which comes from a company called CrowdStrike and is a Windows-based tool to detect and respond to cybersecurity threats, seems to have caused a problem with Windows (it looks like Windows 10). That means that the machines that have had this update, effectively are doing a thing called the 'blue screen of death'. This means their machines want to reboot, but then they can't be rebooted, and so the machines basically become useless.

This has become a global phenomenon because CrowdStrike is a very large company, and a lot of companies and organisations use them to detect and protect against threats. The issue will affect very, very large numbers of machines around the world. It's not a cyber attack, but it's just an interaction of the two pieces of software.

The widespread outages show the risks in relying on a single technology for vital services. There need to be alternate communication links using different software. This does create an added security and maintenance burden, as multiple products need to be looked after and protected. But if you put all your eggs in one basket, you can end up with it on your face.

The global IT outage appears to have been caused by an issue with the CrowdStrike antivirus software. It appears to have impacted Windows machines with this software installed, causing them to crash (blue screen error) and get stuck in boot loops. 

An update to their software appears to have been rolled out globally without proper testing. Antivirus software is typically given access to a deep set of permissions (kernel-level access) on computers to protect against viruses and malware. The flip side, however, is that if this very software malfunctions, then it can crash the computer, as we have observed with this outage. 

This incident appears to violate every good software engineering practice we know. It also points to the need for mechanisms that can protect a computer's operating system from potentially misbehaving anti-virus software.

A widespread IT outage struck Australia on July 19, 2024, impacting numerous sectors like banking, media, telecommunications, supermarkets, and airlines. The culprit appears to be a technical glitch with CrowdStrike's Falcon sensor, a security software program commonly used on business computers. This malfunction caused crashes that disrupted critical systems.

Consumers faced inconveniences like difficulties with online banking, using EFTPOS at terminals, and accessing online accounts. Communication through customer service lines and business websites was also hampered. Airline check-ins and airport operations may have been slowed down as well.

While the outage is not yet resolved, it highlights our heavy reliance on technology for daily activities. With Australians making over 730 electronic transactions per year on average, our dependence on technology is more critical than ever. Thankfully, there are no reports suggesting this was a cyberattack. Both CrowdStrike and Microsoft are working to address the issue and prevent similar occurrences.

This incident really highlights the privileged role of large technology companies in our national technology posture. What's most important is that we learn from it. Adversaries of many kinds are watching our reaction, and learning how they can attack more efficiently in future.

Large-scale outages like this are rare, so this really is a great opportunity for adversaries to learn how we respond when things don’t go as planned. Response times, response language, and remediation strategies are all useful pieces of information to an attacker who wants to identify vulnerability and gaps.

There is currently a major global technical outage affecting multiple companies and services. Some are attributing this to security services offered by CrowdStrike. Others attribute it to Microsoft or Amazon Authorities and industry will be monitoring, but at this stage it is too early to draw conclusions.

While the outage may easily be a result of misconfiguration by one of these companies, or ‘interference’ between products, the global impact is enormous. It is possible that there is a security breach, but to me, this is instinctively unlikely.

CrowdStrike Falcon has been linked to this widespread outage. CrowdStrike is a global cyber security and threat intelligence company. Falcon is what is known as an Endpoint Detection and Response (EDR) platform, which monitors the computers that it is installed on to detect intrusions (i.e., hacks) and respond to them. That means that Falcon is a pretty privileged piece of software in that it is able to influence how the computers it is installed on behave.

For example, if it detects that a computer is infected with malware that is causing the computer to communicate with an attacker, then Falcon could conceivably block that communication from occurring. If Falcon is suffering a malfunction then it could be causing a widespread outage for two reasons: 1 - Falcon is widely deployed on many computers, and 2 - Because of Falcon’s privileged nature. 

Falcon is a bit like anti-virus software: it is regularly updated with information about the latest online threats (so it can better detect them). We have certainly seen anti-virus updates in the past causing problems e.g. here.

It is *possible* that today’s outage *may* have been caused by a buggy update to Falcon.

The near global outage appears to have been caused by a failure of systems associated with the Crowdstrike Falcon endpoint security monitoring software. Crowdstrike is a global multi-national software solutions provider.

In Australia, many businesses and organisations have found that their software systems have failed due to the software system outage. The reliance on centrally managed global software solutions can lead to significant security risks.

Australian governments have, for too long, acquiesced to companies that store Australian data overseas and manage critical systems from global headquarters out of Australian jurisdictions.