Previous incidents

Terraform wanted to perform one last reboot and we let it. Server should be stably back online now.

We were experimenting with using Terraform for deployment. It caused b to be offline for an hour. As always: no data was lost, because we need an entire column to be red before that happens.

A deployment took a little too long and looked like a brief outage. Because we never deploy to more than one machine at a time, no service outage was caused by this brief machine outage.

There was a DNS hiccup in the provider. But because we have redundant TLDs, with both .net and .com, no user data was lost and the service remained responsive.

triplepat.com recovered.

d.triplepat.com went down and up briefly during a deployment. It was detected by our down detector, so that means it counts as a tiny outage. As always, we run 5 master servers in parallel, so the service was still operating with 4x redundancy.

d.triplepat.com recovered.

We updated all the software on the node and that included the kernel and therefore also a reboot, which took a little longer than expected and caused this one node to have 5 minutes of downtime. 4 out of 5 servers continued just fine, and thanks to our master-master database design, we remained fully operational.