Failure Isolation And Blast Radius Control In Large-Scale Distributed Systems

Read full text
Authors: Sathwik Rao Sirikonda, Shashidhar Bhat, Rishabh Jain, Vikas Katoch
Cited by: 1

Abstract

The contemporary scale distributed systems microservices, Kubernetes, service meshes, and multi-region platforms can increase the speed of delivery, but increase the paths of failure propagation with the increase of the dependency graphs. This study analyses failure isolation (fault containment boundaries at service, host, zone, region, and organizational boundaries) and blast radius (the scope of impact of users, services, and geographic areas over time). Cascading failures are shown to be driven by retries, shared-control demands, noisy-neighbor contention, and setup errors that are transformed into fleet-wide failures. Empirical motivation is based on recorded events: instability due to changes accounts for the majority of outages; recovery following the 21 June 2022 incident in Cloudflare affected 19 data centers; and the disaster in AWS us-east-1 on 7 December 2021 had a total impact exceeding 8 hours. The research integrates operational instruments for capping propagation, such as bulkheads, circuit breakers, rate limiting, separation of control planes, zonal and cell design, and progressive delivery and rollback. It generates a measurement-based framework based on community-derived MTTD/MTTR, SLO burn rates, and blast-radius indices to direct architecture and operations. The code of methods is 30-50 public incidents (2016-2025), and it statistically compares containment outcomes across layers. The future of research focuses more on uniform reporting, predictive containment, and safe automation of serverless and self-healing systems.

Fields of Study & Subjects

Software System Performance and Reliability Infrastructure Resilience and Vulnerability Analysis Advanced Optical Network Technologies

Publication details

Journal / source
International Journal of Advances in Signal and Image Sciences
Publisher
XLE Science
Volume
12
Issue
3s
Pages
2003–2022
Publication date
9 March 2026