Calls for improvements will soon turn into demands when new rules come into force
The UK's finance regulator is urging all institutions under its remit to better prepare for IT meltdowns like that of CrowdStrike in July.
The Financial Conduct Authority (FCA) said issues at unregulated third parties were the leading cause of operational disruption within Blighty's financial institutions between 2022 and 2023.
Many major organizations were affected to varying degrees by CrowdStrike's software cockup over the summer, including some of the world's leading banks and trading houses.
JPMorgan Chase's trade execution systems were reportedly affected, some Bloomberg terminals were rendered inaccessible, the London Stock Exchange was hit, and ION Group, UBS, CMC Markets, and others also all reported issues.
"These outages emphasize firms' increasing dependence on unregulated third parties to deliver important business services," the FCA said in a statement. "This highlights the importance of firms continuing to become operationally resilient in line with our rules.
"We encourage all firms, regardless of how they were affected by the CrowdStrike incident, to consider these lessons, to improve their ability to respond to and recover from future disruptions."
For those of you who somehow missed out on what will be remembered as one of the defining IT events of 2024, back in July, CrowdStrike pushed a now-infamous channel file update to its Falcon EDR platform. That update contained a critical logic error, causing Falcon to crash so hard that Windows did too, displaying blue screens of death on 8.5 million PCs worldwide. A bad time was had by many trying to fix this.
Soon, many financial institutions in the UK will be forced by the FCA to become resilient to these kinds of events. The regulator's rules (PS21/3) governing third-party events like CrowdStrike's, requiring in-scope organizations to implement robust business continuity measures that mitigate the worst impacts of incidents like IT outages, came into force in March 2022. The deadline to become compliant - March 2025 - is fast approaching.
The FCA said those who had already met the requirements of PS21/3 demonstrated the best response to the CrowdStrike outage. They were able to effectively prioritize which systems to bring back online first, minimizing the operational impact on the business and wider market, as well as consult prepared incident response and communications plans.
If they mapped their systems and third-party relationships, organizations demonstrated a stronger ability to manage their exposure to limit the overall impact of the incident.
From a technical perspective, some affected institutions were forced to identify single points of failure in their tech stacks and make changes accordingly. For example, some sought alternative products or operating systems, while others decided to review their change management processes relating to software updates.
The FCA urged all regulated organizations to ensure their update-testing procedures were up to scratch and amend them where necessary so any faults can be contained more easily. This especially applies to institutions whose services are relied upon by other key players in the industry.
Other recommendations included preparing external comms templates, such as website banners so all customers and stakeholders are comprehensively informed about any issues in a timely manner. Plus, the usual incident response preparations you'd typically expect any organization to have in place.
Despite the widespread impact on financial markets, the institutions involved largely got on with things and recovered relatively quickly. Little fuss has been made of the incident since.
The same can't be said for Delta Air Lines, however, which recently launched legal proceedings against CrowdStrike, looking to recoup at least some of the circa $500 million in revenue it claims to have lost thanks to the outage.
Delta faced significant challenges, taking longer than most to return to service. It blamed CrowdStrike and Microsoft, and in response they pointed the finger straight back, saying the airline refused their offers of free technical support.
CrowdStrike also alleged Delta was running on aging IT equipment, a major factor in why it took so long to recover.
Shortly after Delta filed its lawsuit against the cybersecurity company, CrowdStrike itself launched a counter-suit alleging "Delta's own negligence" led to the issues it faced. ®