Analytics Disruption
Incident Report for Subsplash
Postmortem

Analytics post mortem
From 10/11/18 10:00 PM- 10/12/18 10:25 AM we experienced a disruption to our analytics servers that left about 1/3 of clients unable to view analytics in their dashboard. The disruption was caused when we deployed an analytics improvements. One of our servers was not updated with the correct config settings causing it to have trouble responding to analytics requests. Safeguards are currently in the works to put automated processes in place to check for inconsistent configuration settings prior to deployment of a new service.

Posted about 2 months ago. Oct 17, 2018 - 14:21 UTC

Resolved
All issues with analytics visualization have been resolved. Less than 1/3 of traffic to the analytics service was affected. We discovered a discrepancy in one out of three servers that was not successfully updated with a config change during a recent deployment to improve analytics. The impact of this issue was less than initially assumed.
Posted about 2 months ago. Oct 12, 2018 - 17:38 UTC
Investigating
We are currently investigating an issue that is preventing some clients from accessing their analytics in the Subsplash Dashboard. All other systems are unaffected, and the underlying analytics service is fine. But dashboard analytics visualizations are unavailable for some clients. We'll keep you posted as we work to rectify the issue.
Posted about 2 months ago. Oct 12, 2018 - 16:43 UTC
This incident affected: Dashboards.