gpg: Signature made Fri Apr 11 09:07:15 2008 UTC using DSA key ID BF15490B
gpg: Good signature from "Andy Smith <andy@strugglers.net>"
gpg: aka "Andrew James Smith <andy@strugglers.net>"
gpg: aka "Andy Smith (UKUUG) <andy.smith@ukuug.org>"
gpg: aka "Andy Smith (BitFolk Ltd.) <andy@bitfolk.com>"
gpg: aka "Andy Smith (Linux User Groups UK) <andy@lug.org.uk>"
gpg: aka "Andy Smith (Cernio Technology Cooperative) <andy.smith@cernio.com>"
Hi,
At approximately 09.10 GMT curacao lost networking and the familiar
symptoms of the kernel bug we've been experiencing were present on
the console. Usually this lasts only a minute or two before
recovering, but after 5 minutes it showed no sign of letting up so I
was forced to power cycle the server. All customers on curacao
experienced a hard reboot.
The server was booted again by 09.20 but starting up all domains can
take some time. Almost everyone [1] was running by 09.32 but almost
every domain then wanted to fsck its filesystems so this resulted
in delayed boot for many, and initial poor IO performance, which is
now over with. As far as I am aware, all services on curacao are
now restored so if you believe this is not the case please contact
support ASAP.
Judging by past experiences this problem now will not reoccur for at
least a couple of weeks, although it is rather unpredictable so I
can't be sure. New hardware has already been ordered and I would
imagine I will have it installed in a matter of weeks, my intention
being to move all customers that are currently on curacao onto the
new server. Either the different Xen configuration or the different
hardware should avoid this issue.
In the meantime there will be a day of service credit for every
paying custoemr on curacao.
Regards,
Andy
[1] One domain did not start due to broken grub config. I started
it manually, but you would be advised to make sure that a
bootable kernel is set as your default.
--
http://bitfolk.com/ -- No-nonsense VPS hosting
Encrypted mail welcome - keyid 0x604DE5DB
