CrowdStrike Outage
We’re concluding our live coverage
CrowdStrike advise customers to check their support portal for updates.
We’re ending our live coverage now, thank you for joining us. We urge everyone to remain vigilant as bad actors will try to exploit events like this. Please ensure that you’re engaging with official CrowdStrike representatives.
Sunday 21 July 14.28
Claire O’Neil, Australian Minister for Home Affairs and Minister for Cyber Security shared an update on X that CrowdStrike informed her that they are now close to rolling out an automatic fix to the issue, as are Microsoft which should mean affected businesses will be back online soon.
CrowdStrike informed the meeting this morning that they are now close to rolling out an automatic fix to the issue with their update, as is Microsoft. This should increase the speed at which systems across the economy are back online.
— Clare O’Neil MP (@ClareONeilMP) July 21, 2024
——————————————————————————————————–
Summary
- Cybersecurity firm Crowdstrike have confirmed this is not a security incident or cyberattack. The issue has been identified, isolated and a fix has been deployed.
- CrowdStrike are continuing to provide continuous updates through their Support Portal at https://supportportal.crowdstrike.com/s/login/
- WARNING: Increased Phishing Attempts During IT Outages: Please be aware of opportunistic phishing attempts culminating from this Global IT outage event. Be extra cautious of emails, calls or texts claiming to be from IT support. Always verify the sender’s details and never click on links or open attachments provided through unexpected channels.
18:01
The National Cyber Security Centre of Ireland (NCSC) has updated its alert notice related to the issue. The notice details further workarounds that also address Bitlocker recovery related KBs.
Read the alert here: CrowdStrike_BSOD_Loop_Issue.pdf (ncsc.gov.ie)
17:32
Below is an automated CrowdStrike BSOD Workaround in Safe Mode using a Group Policy.
You can set up a GPO to run a script during Safe Mode. Here’s how you can do this:
1. Create the PowerShell Script
Create a PowerShell script that deletes the problematic CrowdStrike driver file causing BSODs and handles the Safe Mode boot and revert:
# CrowdStrikeFix.ps1
# This script deletes the problematic CrowdStrike driver file causing BSODs and reverts Safe Mode
$filePath = "C:\Windows\System32\drivers\C-00000291*.sys"
$files = Get-ChildItem -Path $filePath -ErrorAction SilentlyContinue
foreach ($file in $files) {
try {
Remove-Item -Path $file.FullName -Force
Write-Output "Deleted: $($file.FullName)"
} catch {
Write-Output "Failed to delete: $($file.FullName)"
}
}
# Revert Safe Mode Boot after Fix
bcdedit /deletevalue {current} safeboot
2. Create a GPO for Safe Mode
- Open the Group Policy Management Console (GPMC).
- Right-click on the appropriate Organizational Unit (OU) and select
Create a GPO in this domain, and Link it here...
. - Name the GPO, for example,
CrowdStrike Fix Safe Mode
.
3. Edit the GPO
-
- Right-click the new GPO and select
Edit
. - Navigate to
Computer Configuration -> Policies -> Windows Settings -> Scripts (Startup/Shutdown)
. - Double-click
Startup
, then clickAdd
. - In the
Script Name
field, browse to the location where you savedCrowdStrikeFix.ps1
and select it. - Click
OK
to close all dialog boxes.
- Right-click the new GPO and select
4. Force Safe Mode Boot Using a Script
Create another PowerShell script to force Safe Mode boot and link it to a GPO for immediate application:
# ForceSafeMode.ps1
# This script forces the computer to boot into Safe Mode
bcdedit /set {current} safeboot minimal
Restart-Computer
5. Create a GPO to Apply the Safe Mode Script
- Open the Group Policy Management Console (GPMC).
- Right-click on the appropriate Organizational Unit (OU) and select
Create a GPO in this domain, and Link it here...
. - Name the GPO, for example,
Force Safe Mode
. - Right-click the new GPO and select
Edit
. - Navigate to
Computer Configuration -> Policies -> Windows Settings -> Scripts (Startup/Shutdown)
. - Double-click
Startup
, then clickAdd
. - In the
Script Name
field, browse to the location where you savedForceSafeMode.ps1
and select it. - Click
OK
to close all dialog boxes.
6. Apply the GPOs
-
- Make sure the
Force Safe Mode
GPO is applied to the affected computers first. - The computer will boot into Safe Mode and execute the
CrowdStrikeFix.ps1
script. - Once the issue is fixed, the script will revert the boot settings to normal mode.
- Make sure the
*We have not yet tested this approach in a live environment.
Source: Automated CrowdStrike BSOD Workaround in Safe Mode using Group Policy · GitHub
—————————————————————————————————-
16:28
A further update from George Kurtz on LinkedIn advising customers to keep an eye on his posts on LinkedIn, X and the CrowdStrike website.
Today was not a security or cyber incident. Our customers remain fully protected. We understand the gravity of the situation and are deeply sorry for the inconvenience and disruption. We are working with all impacted customers to ensure that systems are back up and they can deliver the services their customers are counting on. As noted earlier, the issue has been identified and a fix has been deployed. There was an issue with a Falcon content update for Windows Hosts. For the latest information that we will continuously update, please refer to the CrowdStrike website, my posts on LinkedIn, and my posts on X. I will continue to provide updates to our community and the industry as they become available.
13:00
‘We’re deeply sorry,’ CrowdStrike CEO says – and it ‘could take some time’ for systems to recover
George Kurtz, Crowdstrike CEO, has given an interview on the issue. “We’re deeply sorry,” he says.
He explained that the global issues were caused by a single faulty content update. “That update had a software bug in it and caused an issue with the Microsoft operating system.”
“We identified this very quickly and remediated the issue.”
When asked how a single update can cause such global chaos, said: “We have to go back and see what happened here. Our systems are always looking for the latest attacks from adversaries that that are out there.”
Asked if there was any possibility that this could have been a cyberattack, he sais no.
“It wasn’t a cyber attack. It was related to this, this content update.”
He stated that systems had been fixed at their end. As for when we can expect everything to be back up and running as normal, Mr Kurtz says there could be a wait for some users. “So, it could be some time for some systems, it [won’t] just automatically recover.”
“But it’s our mission… to make sure that every customer is fully recovered.”
11:15am
A message posted to LinkedIn by CrowdStrike President, CEO & Founder, George Kurtz read:
CrowdStrike is actively working with customers impacted by a defect found in a single content update for Windows hosts. Mac and Linux hosts are not impacted. This is not a security incident or cyberattack. The issue has been identified, isolated and a fix has been deployed. We refer customers to the support portal for the latest updates and will continue to provide complete and continuous updates on our website. We further recommend organizations ensure they’re communicating with CrowdStrike representatives through official channels. Our team is fully mobilized to ensure the security and stability of CrowdStrike customers.
10:55am
The issue has had a widespread impact on infrastructure globally. Microsoft issued the following notice at 10.40 a.m. BST (5:40 a.m. ET)
“We have been made aware of an issue impacting Virtual Machines running Windows Client and Windows Server, running the CrowdStrike Falcon agent, which may encounter a bug check (BSOD [blue screen of death) and get stuck in a restarting state. We approximate impact started around 19:00 UTC on the 18th of July. We can confirm the affected update has been pulled by CrowdStrike. Customers that are continuing to experience issues should reach out to CrowdStrike for additional assistance,” the company added.
CrowdStrike Outage: What we know
10am
A widespread CrowdStrike outage occurred on July 19, 2024, affecting numerous organisations globally, primarily starting in Australia and spreading to other countries. The root cause was identified as a faulty update to CrowdStrike’s Falcon sensor, a key component of their cybersecurity software suite.
CrowdStrike’s Director or Overwatch, Brody Nisbet, has confirmed the issue via a post on X stating:
There is a faulty channel file, so not quite an update. There is a workaround…
1. Boot Windows into Safe Mode or WRE.
2. Go to C:\Windows\System32\drivers\CrowdStrike
3. Locate and delete file matching “C-00000291*.sys”
4. Boot normally.
In a later post he wrote “That workaround won’t help everyone though and I’ve no further actionable help to provide at the minute”.
Cause and Mechanism
1. CrowdStrike released an update for their Falcon sensor, which is widely used for detection and monitoring by cyber and IT teams.
2. This update contained an error that caused Windows computers to experience a “Blue Screen of Death” (BSOD) during startup.
3. Affected systems entered a recovery boot loop, preventing them from starting up correctly.
Impact – initially reported in Australia, the issue quickly became global, affecting organisations in the UK, US, and other countries.
Sectors Affected:
– Banking and Financial Services: Operational halts, affecting customer transactions
– Airlines: Grounded flights and check-in system failures
– Media: Broadcast failures, including major networks like Sky News
– Retail: Disruptions in supermarket operations
– Government Services: Some public services affected, though critical services like 911 remained operational in most areas
We are actively working with our clients and will issue a further update with additional remediation methods as soon as we have them.
If you have any concerns please reach out to your Account Manager.
![](/wp-content/uploads/2024/05/Ekco-Office.png)
Question?
Our specialists have the answer