Syxsense Provides Guidance for Major Microsoft Outage Caused by CrowdStrike
Updated 19th July 09:00 PDT
Customers from various sectors reported widespread device failures beginning early in the day.
The crashes were quickly identified as blue screen of death (BSOD).
Investigation revealed that the source of these crashes was linked to the csagent.sys file, part of the CrowdStrike Falcon agent, which is responsible for endpoint protection.
CrowdStrike has acknowledged the issue and is actively working on a permanent fix. In the meantime, affected organizations can apply a temporary workaround to restore functionality to their systems.
The steps involve booting into Safe Mode or the Windows Recovery Environment, navigating to the CrowdStrike directory, and deleting the problematic C-00000291*.sys file. Here are the detailed steps:
- Boot Windows into Safe Mode or the Windows Recovery Environment.
- Navigate to the C:\Windows\System32\drivers\CrowdStrike directory.
- Locate and delete the file matching C-00000291*.sys.
- Reboot the system normally.
Implementing these steps has proven effective in mitigating the immediate issue and restoring affected devices to operational status.
CrowdStrike has released a statement acknowledging the problem and expressing their commitment to resolving the issue swiftly. They are working around the clock to develop and distribute a comprehensive fix. Additionally, they are conducting a thorough investigation to understand how this issue occurred and to prevent similar incidents in the future.
Lessons Learned from Syxsense
This incident highlights the critical importance of robust testing and monitoring of software updates, including Patch Management and Vulnerability Management.
Rob Brown, Head of Customer Success said, “A well-planned update can enhance security and fix known issues. However, deploying updates across an entire organization simultaneously can also introduce significant risks. One effective strategy to mitigate these risks is ‘phased’ deployment. This approach ensures that updates are rolled out in stages, allowing for controlled and safe expansion while minimizing the potential impact of unforeseen issues.
*******
Content below originally published 2024/07/19 at 5:14 am PDT
Breaking News Update: Microsoft Major Outage July 2024
We have had multiple calls from customers reporting global blue screen crashes since early this morning on all devices.
It appears the cause is the csagent.sys used by CrowdStrike.
Workaround Steps:
To resolve the issue, please follow the steps below:
- Boot Windows into Safe Mode or the Windows Recovery Environment
- Navigate to the C:\Windows\System32\drivers\CrowdStrike directory
- Locate the file matching “C-00000291*.sys” and delete it.
- Boot the host normally.
Implement these steps immediately to mitigate the impact on affected systems.
CrowdStrike have corrected the update causing the issue – so automated updates will work on systems not already affected.
The manual steps above will still be necessary to fix impacted systems.
More details can be found here: https://www.forbes.com/sites/barrycollins/2024/07/19/huge-windows-blackout-hits-banks-airports-and-more/
Do you need help keeping up patches? Syxsense’s automated patch management capabilities help enterprises patch faster and more accurately. Schedule a consultation with us to learn how we can help you.
