[Resolved] 2 Servers Read Only

=========================
20 October: 4.00pm

We have checked a random selection of sites and they all resolve fine.  If you see any issues please open a helpdesk ticket but as far as we can see all the sites we have tested appear to be working as they should.

We apologise once again for this issue.

=========================
20 October: 3.20pm

All data has been restored to server 34.  We are just running Easyapache to match configurations and then we need to swap the IPs over.  We anticipate this all being completed within 30 minutes at which point sites will start working.

=========================
20 October: 3.00pm

Server 34 transfers have nearly completed.  We should be able to bring all websites on this shared server on line within the hour. 

=========================
20 October: 2.45pm

Fles are still copying across.  Unfortunately there is nothing we can do to speed this up

=========================
20 October: 2.00pm

Just a quick update on server 34.  /home has been copied.  We are copying other needed files now.

=========================
20 October: 1.34pm

Just a quick update on the other affected server (a client VPS).  Cpanel installation has completed on the new server built for the affected VPS client.  We are currently copying the data across.  Once that completes we will switch the IPs.  Further updates as we have them.  Thanks for being so patient

=========================
20 October: 1.20pm

Just a quick update on server 34.  Cpanel installation has completed on the new server 34 we built.  We are currently copying the data across.  Once that completes we will switch the IPs.  Further updates as we have them.  Thanks for being so patient

=========================
20 October: 12.00noon

Unfortunately these two servers will not boot up and we are working on restoration.  As stated earlier there is no hardware issues and no disk issues so there is no data loss and we will be restoring from the live data and not from backup.

Next update 2pm

=========================
20 October: 11.00am

A little bit of technical speak.  We just mounted server 34 to a temporary directory /root/server34 and we can see the data.  For some reason it seems that lvm (logical volume manager) could not mount it properly.  We continue to work on this. 

=========================
20 October: 10.45am

We checked and the data centre checked the drives are healthy and the RAID array is optimal and there are no errors from the logs to help determine why 2 servers on this platform went read only.  We are continuing to work on this with top priority.

Just to confirm server 34 shared server and one server belinging to a client are affected and are read only.  All other services are on line at this time.

=========================
20 October: 10.00am

VPS11 is being taken off line so we can check the RAID BIOS.  Downtime will be 20 minutes.  This will affect all servers on VPS11 platform and we will expediate this.

=========================
20 October: 9.30am

At this time we are facing an issue with 2 servers on our VPS 11 Platform.  Two servers (server 34 shared server and one server belonging to a client) have gone read only.  We have performed a file system check on both servers and they are still read only.  We do not see any hardware issues and many other servers from the same platform are working normally.  We are working on this as top priority firstly to determine why this is happening and secondly what we can do to fix this problem.
=========================