On July 19th, a failure was detected in the update of the CrowdStrike Falcon platform, significantly affecting Windows operating systems globally. This issue has impacted various companies and institutions. The update caused a BSOD (Blue Screen of Death) error, rendering devices completely inoperable and requiring recovery mode to restore them.
Below you can find all the details of what happened, how to identify if your devices are affected, and how to resolve the issue:
How to Identify Affected Devices
Solucionar el error en entornos de nube pública o virtual
Issue Details
The failure originated in an update to the detection logic of the Memory Scanning service in Falcon, which caused excessive CPU usage on Windows systems. CrowdStrike has identified and reversed the cause, but affected systems require a reboot to restore normal functionality.
You can find all the updated information provided by the CrowdStrike team in this Tech Alert:https://supportportal.crowdstrike.com/s/article/Tech-Alert-Windows-crashes-related-to-Falcon-Sensor-2024-07-19
How to Identify Affected Devices
The following devices have been impacted by the CrowdStrike issue:
- Hosts that were online before 05:27 UTC may be affected.
- Hosts that come online after 05:27 UTC will not be affected.
- Hosts running Windows 7 or Windows Server 2008 R2 are not affected.
- Mac and Linux hosts are not affected.
For users with the Investigate module of CrowdStrike, an advanced query can be run to identify devices with the affected version:
How to Fix the Error
At Sofistic, we acted quickly from the onset of this incident. Our technical support team has maintained constant communication with CrowdStrike to resolve the situation and minimize adverse effects on our clients. Through our additional technical support service, we have provided detailed information and continuous assistance to resolve the issue for our clients promptly, ensuring they feel supported at all times.
The steps to resolve the issue include:
- Restart the systems in Safe Mode or the Windows Recovery Environment.
Navigate to the folder C:\Windows\System32\drivers\CrowdStrike.- Locate and delete the file matching
C-00000291*.sys. - Restart the systems normally.
- This GPO can be used for automated deployment: https://gist.github.com/whichbuffer/7830c73711589dcf9e7a5217797ca617
- If there are devices with BitLocker that ask for the key to enter Safe Mode and apply the workaround, here is a possible solution: https://gist.github.com/whichbuffer/7830c73711589dcf9e7a5217797ca617
How to Fix the Error in AWS
As noted by researcher "Ido Naor on X", if your machines in AWS have been affected, the issue can also be resolved. Below are the steps:
- Locate the C: drive in the EBS and create a snapshot of the EBS.
- Launch a new EBS using the snapshot.
- Attach it to a temporary instance and access the temporary VM to rename the .sys extension in the #CrowdStrike folder.
- Then, proceed with detaching the EBS.
How to Fix the Error in Public or Virtual Cloud Environments
- Unmount the disk volume from the affected virtual server.
- Create a snapshot or backup of the disk volume.
- Attach the volume to a new virtual server.
- Navigate to the directory %WINDIR%\System32\drivers\CrowdStrike.
- Delete the file that matches C-00000291.sys*.
- Unmount the volume from the new virtual server.
- Reattach the fixed volume to the affected virtual server.
Frequently Asked Questions
Has the issue affected Sofistic's service?
No, Sofistic's MDR service has remained 100% operational at all times despite this issue.
Which update is faulty?
It appears that the faulty update was sent around 4 a.m. UTC.
Is there any risk for unaffected machines?
No, CrowdStrike confirmed that the faulty updates have been removed and there is no risk to unaffected machines.
