Opération #260

Titre Network issue on paris1
État terminé
Date de début 1 fév. 2015 01:07
Date de fin 1 fév. 2015 01:39
Serveurs concernés


1 fév. 2015 01:07

We're investigating.

1 fév. 2015 01:17

The primary router seems to be down, the traffic is routed by router2.

1 fév. 2015 01:41

router1 has frozen at the exact same time the monthly RAID check started, for an unknown reason (no traceback or any log whatsoever). We've rebooted it, it's now routing traffic again.

The traffic didn't fail over to router2 until 10 minutes after router1 was down, which isn't normal. We're investigating what happened.

1 fév. 2015 01:49

Our logs show that our internal gateway monitoring tool didn't fail-over until 10 minutes after the first downtime was detected, in IPv4. That's obviously not normal. The IPv6 fail-over was done very quickly, though.

1 fév. 2015 18:27

We've found and fixed the bug that caused the slow fail-over in our monitoring tool.