Statut

Vérifiez l'état de nos services d'un seul coup d'œil

Titre Incoming mails delayed
ID Opération #109
État terminé
Date de début 13 juin 2013 10:49
Date de fin 13 juin 2013 11:19
Serveurs concernés
  • smtpin1

Messages

13 juin 2013 10:49

We’re investigating.

13 juin 2013 11:00

We’ve found the issue, it should be resolved in a few minutes.

13 juin 2013 11:14

Issue resolved. More details will follow shortly.

13 juin 2013 11:38

All emails delayed will be received later, as the sender hosts will retry to deliver their messages.

Here’s what happened: our main database server has been automatically protected by a DDoS protection device by our provider due to a small scale attack. This mitigation device unexpectedly blocked the PostgreSQL traffic between some of our servers. Unable to contact the main database, our mail servers refused all incoming mails.

We have a hot standby database server, in case the main database server is unreachable. Unfortunately, due to a misconfiguration in our firewalls, that server was not accessible. This was fixed.

The traffic from our MX servers to our main database server has been moved inside our internal VLAN to avoid issues with the mitigation device. We’re also working on radical changes to avoid such issues from our provider to happen again. We’ll communicate on this in a few weeks.