Eerdere incidenten

We rebooted the d-server to bring in the latest kernel and tried to see if it would come back fast enough to slip between uptime probes. It did not. It came close though, as shown by the fact that d.triplepat.net ostensibly stayed up the entire time, despite it being the same server.

We set up real-time back-up for the user database (as opposed to hourly backup, which we already had) and we discovered a bug in our deployment script in the process, which meant the services did not come back up the same in production as they had in testing. This caused the deployment to triplepat.com to fail, which halted the deployment with a non-working triplepat.com, while all the mirrors kept working fine.

The bug is fixed, the user-service was down briefly, and the checkin-service ke...

Re-deploying to investigate resource usage issues caused brief downtime. The check-in service worked without a hiccup while this server was down.