Author: Conrad Wood Date: To: users, announce Subject: Re: [bitfolk] 2021-02-20 0100Z onwards: Emergency reboot of host
"talisker"
On Sat, 2021-02-20 at 03:28 +0000, Andy Smith wrote: > Hi,
>
> In the last couple of hours I unfortunately had to reboot host
> "talisker" including full shutdown & boot of all the VMs on it.
>
> It seems from logs that problems started at approximately 01:00. The
> first alerts came in at 01:22 when customers started trying to
> reboot their VMs. Symptoms for customers were stalling of tasks,
> unable to shut down properly, unable to boot again after forcibly
> shutting down.
>
> I spent some time trying to investigate but it wasn't making things
> any better so by about 02:30 I decided to issue a reboot. Customer
> VMs were all back up and running by about 02:45.
>
> I continue to investigate what the root cause may be and am keeping
> a close eye on things.
>
> Apologies for the disruption this will have caused you.
>
> Thanks,
> Andy
