Hello,
On Thu, Mar 18, 2021 at 02:27:02PM +0000, Andy Bennett wrote:
> I think the rolling upgrades thing has really confused the issue for me.
Yes, having both things happening at the same time is really
confusing and we wouldn't have had them both happen at the same time
if there had been a choice.
It's also unfortunate because some of the servers that have already
been emptied as part of rolling upgrades are ones that will now have
to be reracked, whereas if I had know then that this would happen
they could have been reracked when they were empty. This would have
been doubly great because not only does it not affect anyone at the
time but also provides a "safe" server for future placement.
I've also considered just buying new hardware to place in the new
rack and emptying then junking the existing hardware. I would do
that to avoid this disruption, but it won't work - the lead times
and problems encountered with new hardware over the last year have
been awful so we can't get ahold of it and get it racked fast
enough. And even if it were in place tomorrow, it would be a very
hard push to get several hundred customers moved within 4 or 5
weeks. Though I'd give it a good go.
> I thought that the rolling upgrades would involve a host server name change.
It will/does, because when I talk about rolling upgrade what that
means is:
1. We open support tickets with every single customer on host X
(this is 40 to 60 people/tickets) telling them that we want to
move their VPS to another host and asking when it would be
convenient to do that.
2. We do the moves for each individual VPS according to the
customer's preference until there's none left on host X
3. We reinstall the OS on host X at our leisure
So in step 2 between 40 and 60 people's VPSes are moved from host X
to any of our other hosts according to best fit. And then later on
when we're working on host Y, host X is once again a valid
place to put customer VPSes, because it's a rolling upgrade.
> So people might be on one server now, that is scheduled to move to the new
> rack at a certain time. And then between now and then they would have their
> rolling upgrade and end up on a server with a different name that might be
> moved at a different time.
Yes, or not moved at all, because not all of our servers are in the
rack that is being rebuilt.
I think I can avoid exposing "safe" customers to the rerack, but I
can't avoid moving customers between two servers that are both
scheduled to be reracked.
FYI we're currently working on emptying server "jack" and the majority
of those customers will need to move to server "hobgoblin". Both of
those servers will be part of the rerack. We'll probably find time
to move on to emptying another before the date of the first rerack.
In terms of timings it's looking like we will rerack one server on
one night inside a 2 hour window expecting it to take around 30
minutes. Then assuming that goes well we will do the remaining six a
week later inside perhaps a 4 hour window. We're not going to try to
give precise times for each server, only for the start and end of
the window.
Cheers,
Andy
--
https://bitfolk.com/ -- No-nonsense VPS hosting
