DDOS - Server 120
-
Wednesday, 18th February, 2015
-
13:01pm
Wednesday 18 February 2015
We are aware of issues with Server 120 and we would ask you not to open chats or tickets at this time as this will delay our resolution of this issue. All information we know will be posted here the moment we know it so please refresh this page. Thanks for your patience while we work to resolve this issue.
7.19pm
The DDOS mitigation is now working as it should have done earlier and the network admins have confirmed the same. We are continuing to monitor this server and we sincerely apologise for the sluggish performance throughout the day.
5.26pm
The server is back on line now.
We are still working with our partners to resolve this issue. We have a contract in place for DDOS mitigation and it appears the setup we are paying for has failed us today. There is a UDP atack which is a type of DDOS attack. The server has fielded 18M+ UDP attack packets in a short space of time. The contract we have in place should easily be mitigating these packets and it is not. We are talking to our partners right now about why we are paying for advanced DDOS protection that failed when we needed it on what is a relatively simple and small attack.
4.05pm
We see more traffic and we are working on this again. We do apologise.
3.20pm
We have suspended an account on the server that is the subject of the DDOS attack. We have also deleted the DNS Zone. We need to talk to our network admins to ascertain why our DDOS protection setup that every IP range sits behind did not scrub this traffic before it hit the server. We apologise for this as in theory with our DDOS system most attack traffic should get scrubbed before hitting the server. We need to look at what was specific about this attack traffic so we can make sure in future similar traffic is scrubbed
Our senior network admin will be working with later today to make any required changes to ensure if something similar happens again we can scrub the attack traffic much more quickly..
3.15pm
Unfortunately the issue has re-appeared and we are checking this abnormal traffic with urgency. We apologise again for this issue.
2.40pm
The server is back on line and connections are normal. We still see heavy traffic but we are monitoring it now for you.
2.05pm
We now have access to the server via KVM and are working with priority to troubleshoot. We will post an update soon.
1.35pm
Unforunately the data centre remote hands have not connected the KVM device yet. We are holding now to get an urgent update. We apologise for this extended delay.
1.25pm
The server is still having issues. Our data centre have a delay in connecting the KVM device for us. The KVM device will be connected by 1.35pm we have been assured by data centre remote hands. Unfortunately there is little we can do until this is connected. We appreciate your extended patience.
1.10pm
The server became unresponsive and needed a reboot. We are hooking up a KVM device to the server so we have options should this happen again. At the moment the server is up but the MySQL servivce is down. We still see significant traffic hitting the server and we are going to continue working to mitigate this. We thank you for your patience.
12.45pm
We saw a massive inbound traffic spike directed against this server which we believe to be a denial of service attack. We do have all our IP ranges protected by Arbor Networks Anti DDOS scrubbing devices. We are working with the network team to ascertain why in this instance the attack traffic is getting through to the server. We will have some updates soon and will post here once we have them.
