On Mon, 2019-11-11 at 14:09 +0000, Andy Smith wrote:
> Hi Conrad,
>
> On Mon, Nov 11, 2019 at 11:30:50AM +0000, Conrad Wood wrote:
> > I use a combination of fail2ban and some hooks in my software to
> > build
> > up a blacklist of IPs over time.
>
> At the moment I am also using Fail2Ban for SSH attackers and
> dictionary attacks against a few different web apps.
>
> Last time we talked about this on the list people suggested I use
> multiple different jails with much longer ban times for repeat
> offenders, so I started doing that too.
>
> I currently null route them rather than firewall them but that's
> much of a muchness.
>
> http://strugglers.net/~andy/blog/2019/09/04/fail2ban-iptables-and-config-management/
>
> That is happening on each host (and even on each of BitFolk's own
> guests) without sharing the bans. I've often thought about sharing
> some sort of database.
>
> > My question is if it's feasible to have a bitfolk-hosted blacklist
> > of
> > IPs. If we were all to report our probes and scans into a (to-be-
> > build)
> > bitfolk system, we'd probably protect each other more quickly and
> > effectively.
>
> …so when I start thinking about sharing some sort of database, my
> next thought is, "could the customers contribute to and benefit from
> that as well?" It's been on my mind from time to time!
>
> > Of course I am not certain if that's an option, (e.g. not sure if
> > the
> > inbound routers are powerful enough)
>
> I'm not really worried about load of it. It's probably beneficial to
> drop a packet rather than have it go through the router, through the
> hypervisor, into the guest, be logged as objectionable, do all that
> again and be dropped in the guest's firewall.
Ah yes, using it on the hypervisor should drop CPU usage quite a bit
too. Good idea.
>
> My major concern is how to prevent customers (and me) from
> accidentally or maliciously blocking too many things for everyone
> else. Do you have any ideas?
It ought to be a score based system, e.g. each VPS reports a
probe/malicious source IP with a 'confidence level'.
The bitfolk system would then sum/other algorithm the difference
confidence levels into a score.
Ideally, customers could choose from which confidence level onwards
they would like packets dropped, but I think that could also be
bitfolk-managed.
Equally, a "good" ip could be reported too. Assuming an API of sort I
could authenticate against bitfolk and submit my IPs as 'known good'.
>
> After that, my lesser concerns are around how much work this would
> be:
>
> - Presumably it is going to need some sort of REST API for
> submission and querying.
Preferrably gRPC ;)
>
> - As the ban actions would be happening in a device that you don't
> have control of, possibly there would need to be reporting of the
> packets that have hit the ban for you in particular. Otherwise any
> time anything network-wise goes wrong, you're going to be in the
> dark about whether it's down to this.
prometheus could gather stats per VPS. (e.g. how many were dropped for
a given VPS). there are several iptables exporters for that case. (I
also have an iptables exporter in golang - happy to contribute that
source)
I think if each dropped packet (by source & vps) were to be logged in a
system and customers can choose to scrape that or not when/if required
the time this information needs to be stored could be very short.
For example, if you keep that log or database for 24 hours it would
probably be sufficient (and would keep it small(ish))
>
> - This doesn't sound like a service that people would pay for, or
> even be so impressed by that it would influence a purchasing
> decision. That doesn't stop me from considering improvements like
> this but it does mean that it would happen at the expense of
> something else that needs doing.
You are probably right. What do you think of a github account for
bitfolk where we could all contribute? That might raise awareness about
the cool things that bitfolk does and might just add a few customers.
(And might help you with the maintenance of the contributed code)
>
> Copping out and just making a blocklist that people can choose to
> use themselves or not seems easier: at least they can see themselves
> what they are dropping. But then it's questionable if there is any
> value to that beyond just using existing blocklists.
I think it needs to be opt-in as well.
