Back to Home

    Network Chaos Experiment Incident

    Network chaos experiment causing latency spikes in front-end service through catalog-db-main dependency chain. Here's how Coroot identified the root cause and provided the fix.

    Coroot's Root Cause Analysis

    Root Cause: A network chaos experiment was started, introducing network delays between the catalog service and db-main database.

    The experiment caused cascading latency increases throughout the dependency chain: db-main → catalog → front-end and order services. The experiment was actively injecting network delays, causing context cancellations and timeouts in upstream services, with latency compliance dropping to 97.06%.

    Coroot root cause analysis showing network chaos experiment incident

    Coroot's automated analysis immediately identified the active chaos experiment as the root cause

    Detailed Analysis

    Coroot provided comprehensive charts and metrics showing exactly what happened during the incident:

    Detailed network chaos experiment analysis showing latency impact and dependency chain

    Complete analysis showing network delays, cascading latency impact, resource correlation, and log evidence