Service disruption — upstream network incident
Resolved
Jun 17, 2026 at 12:43pm UTC
Post-mortem — service disruption on 17 June 2026
Summary. VoiceDock experienced a ~45-minute disruption caused by a network fault at our cloud infrastructure provider's datacenter. Inbound and outbound calls and the dashboard were unavailable during the window. No data was lost or corrupted.
Impact (≈11:18–12:04 UTC). Calls could not be configured and the dashboard could not load data, because both depend on our database host, which was isolated at the network level. The host itself stayed healthy — only its datacenter network path failed. Call recordings (stored separately) were unaffected.
Timeline (UTC).
- 10:54 — last call processed normally.
- 11:18 — automated monitoring detected the API and database becoming unreachable; on-call was alerted within ~1 minute.
- ~11:30 — investigation confirmed the host was fully unreachable at the network level; cause traced to the provider.
- 11:48 — status page updated with the identified cause.
- ~12:03 — the provider restored the network segment and the host returned automatically.
- 12:08 — all services verified healthy; incident resolved.
Root cause. A network switch ("leaf") fault in our provider's datacenter isolated the server hosting our database and platform at the network level — an upstream provider incident outside our direct control.
What went well. Sub-minute detection and alerting, customer communication via this status page during the window, and nightly-validated backups (restore and secret decryption both verified).
What we're improving.
1. Off-site backups — moving database backups to independent, off-provider storage so recovery never depends on the affected host.
2. Redundancy — evaluating a standby in a second location and a managed, multi-region database option to remove this single-provider, single-location failure mode.
We apologise for the disruption and thank you for your patience.
Affected services
Updated
Jun 17, 2026 at 12:08pm UTC
Connectivity has been fully restored. The upstream network incident at our cloud provider has been resolved, and all VoiceDock services — inbound and outbound calls and the dashboard — are operating normally again. We are continuing to monitor to confirm stability. Thank you for your patience, and we apologise for the disruption.
Affected services
Created
Jun 17, 2026 at 11:48am UTC
We have identified the cause of the current disruption. A network fault at our cloud infrastructure provider's datacenter has interrupted connectivity to part of the VoiceDock platform. This is an upstream incident outside our direct control; the provider's engineers are actively working on a resolution.
During this period, inbound and outbound calls and the dashboard may be unavailable. No data has been lost or is at risk — the disruption is limited to network connectivity.
We are monitoring closely and will post a further update as soon as service is restored or we have a clearer timeline. We apologise for the inconvenience.
Affected services