Using sabotage to improve

Given a long enough time frame everything breaks.  How ready is your business for a serious disaster?   This is a short account of some of the preparations the technology team at the FT have been making to ensure we can keep delivering world class business news and analysis even when things go very, very wrong.

Over the past few years at the FT we have steadily been increasing our disaster recovery testing with planned failovers and deliberate acts of sabotage to our own estate. Taking a leaf from the Netflix simian army, we have gradually improved our services’ resilience with what we have learnt from these exercises.  For example, powering down DNS or Active Directory servers have highlighted areas of weakness that we have been able to mitigate or remedy completely. Continue reading “Using sabotage to improve”