Hello,
Going through a total reboot as we are just now, it naturally leads
to quite a few VMs sitting waiting for the customer to unlock their
LUKS (encrypted block devices) before they will boot. That's a major
downside of encrypting your disk.
Various methods to automatically unlock LUKS have existed for many
years, but can be somewhat fiddly to configure. That's obviously
less secure as well, unless you really put some effort and some
infrastructure into it.
Mandos is an example of something you might use if you had another
server somewhere:
https://www.recompile.se/mandos/man/intro.8mandos
Basically it stores an OpenPGP private key inside your initrd, and
uses that to decrypt the LUKS passphrase that is stored on the
mandos-server.
The threat model for that is that an attacker with BitFolk-level
access can read your (unencrypted) initrd to get the OpenPGP key out
and also read the encrypted passphrase out of the mandos-server, put
the two together to decrypt the passphrase and unlock your LUKS.
If your mandos-server was not at BitFolk then that would be slightly
more robust, but there is still the issue that an attacker at
BitFolk could use a copy of your initrd to impersonate your VM
during a maintenance window.
You can of course instead do something a lot simpler like literally
just have a script in your initrd contact a remote server to get the
passphrase. The threat model is still about how your VM identifies
itself to the secrets server in a way that can't be faked.
For example if your script does an SSH to your secrets server, it
means storing the private SSH key unencrypted inside your initrd, so
an attacker with BitFolk-level access can read those SSH keys at any
time without your knowledge and impersonate your VM to your secrets
server to get a plain text copy of your passphrase.
I can think of various ways you might harden that (or Mandos). Like
you might require that the request comes from your IP address and
within a certain time window, like within a few hours either side of
a known maintenance window. I'm sure you can see how it would still
be possible for an attacker with BitFolk-level access to impersonate
your VM during a maintenance window in order to get a copy of your
secrets to use at a later date.
I'm guessing that of those customers who use disk encryption:
- You mostly do it to try to hamper access by a fairly unskilled
attacker who gains access to BitFolk's systems, or who physically
steals the storage, or to guard against improper disposal of
storage.
- You haven't set up automatic unlocking because it's too fiddly, or
you lack other servers elsewhere to serve the secrets, or you can
see no way to make it as secure as you want.
I was wondering if there was anything BitFolk could do to make it
easier for those of you using LUKS, to have some form of automated
unlock, perhaps only during maintenance events? If any of you have
any thoughts I'd be interested to hear.
It all really hinges on whether you care about someone impersonating
your VM to get the LUKS passphrase, and if you do, if it is even
possible to come up with a way for your initrd to identify itself
that can't be subverted.
Note that while it would be possible for BitFolk to run a secrets
service and provide you with a minimally invasive package that adds
a script to your initrd to unlock it, there isn't really any way to
secure that against an attacker with BitFolk-level access.
They could for example just take a snapshot of your block device and
boot it during the maintenance window, unlock your LUKS, copy out
your data, shut it down and then boot your real VM. The only record
would be in BitFolk's secrets server that it received two requests
for unlock. Which could be made to lie to you if you asked it how
many requests it had seen.
So really, BitFolk probably should not provide a secrets server. I'd
be happy to investigate a few options for interacting with remote
servers that you provide yourself, though. Mostly a documentation
exercise.
== On threat models ==
I think it is important to be realistic about what using LUKS on a
virtual server actually can protect you against.
When I say "an attacker with BitFolk-level access" I mean
either BitFolk staff acting inappropriately, or outside attackers
who have gained root access, or agents of the state who can compel
BitFolk to do almost anything with your data.
Against one of those sorts of attackers who is really motivated and
specifically interested in you, you have no chance. It's trivial to
dump your VM's memory image to disk, and probably quite feasible to find
the LUKS keys within that - I've seen it done with KVM a few years
ago. Then snapshot the block devices and take away a copy to decrypt
at leisure.
Even if you *were* running on bare metal, there are techniques to
take dumps of the contents of DIMMs that are still in a running
server but we don't have to speculate about that because you're not
on bare metal and dumping VM memory is easy with root access to the
hypervisor.
Most attackers aren't like that though, not even the state.
Criminals pick easier targets; the state prefers to either sniff
traffic for long periods of time or just seize all the hardware.
If you're doing something where you think the state might go as far
as reading your LUKS keys out of a memory dump, you probably should
not be doing it on a virtual machine in the UK!
Another thing to consider is your Xen console. The kind of attacker
we're talking about can read and write to it, so if you leave
yourself logged in they can just be root straight away, and if you
ever log in they can read the root password that you type, then
reuse it later. When it comes to unlocking LUKS you are actually
safer SSHing in to an sshd that's running in your initrd, not using
the Xen console.
It is really hard to secure VMs against hypervisor-level attackers
with a particular interest, and so most uses of LUKS on VMs should
only be for protecting against simpler attacks. It's still a LOT
more secure than not using encrypted storage though…
Cheers,
Andy
--
https://bitfolk.com/ -- No-nonsense VPS hosting
