Terug naar overzicht
Uitvaltijd

b.triplepat.com is down

mrt 29 om 10:57 HDT
Getroffen services
b.triplepat.com

Opgelost
mrt 31 om 01:12 HDT

The machine b.triplepat.com became unavailable and eventually rebooted due to a "power event" and corresponding Google Cloud outage in its datacenter. When it came back up, the containers did not all start successfully at boot-time.

It looks like the internal DNS for the docker-compose network didn't come-up successfully, which meant that the nginx config could not resolve the internal names for redirects, which meant that nginx refused to start. (Another point for https://isitdns.com)

The power event in GCP: not our fault
The DNS stuff not working after boot: at least a little bit our fault, and certainly something we might want to work to remediate.

No user check-ins affected, because all other servers were up and running fine this whole time. No data loss. Resiliency works!

The only TODO item is to make sure our systems reboot cleanly.

Bijgewerkt
mrt 30 om 20:50 HDT

b.triplepat.com recovered.

Aangemaakt
mrt 29 om 10:57 HDT

b.triplepat.com went down.