CrowdStrike Update Causes Global Windows System Crashes and Outages

CrowdStrike Update Causes Global Windows System Crashes and Outages


A recent update to CrowdStrike Falcon, a leading cybersecurity solution, has resulted in widespread crashes of Windows systems, significantly disrupting operations for numerous organizations and services across the globe. The impact has been far-reaching, affecting critical infrastructure such as airports, TV stations, and hospitals, and even extending to emergency services.

This unprecedented event highlights the vulnerabilities and potential risks associated with software updates in today's interconnected world. The issue has led to significant outages, rendering entire companies and vast networks of computers non-operational, and sparking concerns over the stability and reliability of essential digital systems.

The root of the problem lies in a defective component within the latest CrowdStrike Falcon Sensor update. Users have reported their Windows systems being caught in boot loops or showing the infamous Blue Screen of Death (BSOD) after installing the update. This malfunction is not confined to individual users but spans across workstations and servers alike, creating a domino effect of outages.



Workaround for the Faulty CrowdStrike Update


Users have been reporting their Windows systems entering boot loops or displaying the Blue Screen of Death (BSOD) following the installation of the latest CrowdStrike Falcon Sensor update. CrowdStrike has acknowledged the issue, identifying a problematic content deployment as the cause and has since reversed the changes.

Symptoms described by CrowdStrike include hosts experiencing a bug check or blue screen error linked to the Falcon Sensor. The issue stems from a Channel File within the update, which can be individually addressed without removing the entire Falcon Sensor update.



Steps to Resolve the Issue:


The following mitigation steps are provided by CrowdStrike to those affected by the incident.

1. Boot Windows into Safe Mode or the Windows Recovery Environment.
2. Navigate to the C:\Windows\System32\drivers\CrowdStrike directory.
3. Locate and delete the file matching “C-00000291*.sys”.
4. Reboot the host normally.

George Kurtz, President and CEO of CrowdStrike, has stated that the company is actively working with customers to resolve the issue caused by a defect in a single content update for Windows hosts. He encourages affected organizations to communicate through official channels and assures that the CrowdStrike team is fully mobilized to restore security and stability.



Updated Guidance from CrowdStrike


In an updated statement, CrowdStrike clarified that the problematic Channel File “C-00000291*.sys” with a timestamp of 0409 UTC has been reverted. The corrected version of the file, with a timestamp of 0527 UTC or later, is now available.

For cloud and virtual environments, CrowdStrike provides a seven-step procedure to roll back to a snapshot before 04:09 UTC:

1. Detach the operating system disk volume from the impacted virtual server
2. Create a snapshot or backup of the disk volume before proceeding further as a precaution against unintended changes
3. Attach/mount the volume to a new virtual server
4. Navigate to the %WINDIR%\System32\drivers\CrowdStrike directory
5. Locate the file matching “C-00000291*.sys”, and delete it.
6. Detach the volume from the new virtual server
7. Reattach the fixed volume to the impacted virtual server


Impact on Global Operations


Despite the fix and workaround, the fallout from the faulty update has already affected many large organizations across various sectors. Although CrowdStrike is working to mitigate the issue, companies will likely experience residual effects for some time. System administrators face a challenging task, especially with extensive computer fleets, remote employees, off-premises data centers, and cloud environments where safe mode booting may not be feasible.



REFERENCE


About the Author

Ruben George