Re: [bitfolk] Debian mirrors and the apt-cacher (Was Re: 21 …

Top Page

Reply to this message
Author: Adam Spiers
Date:  
To: Andy Smith
CC: users
Subject: Re: [bitfolk] Debian mirrors and the apt-cacher (Was Re: 21 critical Exim security issues need addressing)
On Sat, May 08, 2021 at 09:04:19PM +0000, Andy Smith wrote:
>Hello,
>
>On Sat, May 08, 2021 at 06:04:44PM +0100, Adam Spiers wrote:
>>On Sat, May 08, 2021 at 09:15:46AM +0000, Andy Smith wrote:
>>>The reason why "deb.debian.org" isn't in the list is because that mirror
>>>is a geo-ip thing that redirects to what it thinks is the nearest fastly
>>>CDN node. It's mainly used for giving out to people when you don't know
>>>where in the world they are so giving them a geographical mirror would
>>>not be appropriate. Also for mobile users, for example.
>>>
>>>Doesn't make a lot of sense for things permanently located in London,
>>>which is why the default setting is "ftp.uk.debian.org".
>>
>>OK. IMHO the problem seems to be a combination of the following:
>>
>>- https://backports.debian.org/Instructions/ only mentions
>> deb.debian.org, and says nothing about alternative mirrors.
>
>Debian's instructions are probably that way because they don't know
>what country the viewer may be in, so directs people to a geo-ip
>redirector. All Debian mirrors are meant to carry the same stuff so
>anything you can get from deb.debian.org you should also be able to
>get from ftp.uk.debian.org or ftp.de.debian.org and so on.


Added to the wiki.

>>- https://tools.bitfolk.com/wiki/Apt-cacher#Available_mirrors doesn't
>> mention deb.debian.org, and even though backports.debian.org is
>> listed, it doesn't explain how to derive a source correctly pointing
>> to a Bitfolk mirror of backports.
>
>Once you know that ftp.uk.debian.org/debian/ can be used any place
>that deb.debian.org/debian/ is mentioned then hopefully you can see
>that what you wanted was:
>
>    deb http://apt-cacher.lon.bitfolk.com/debian/ftp.uk.debian.org/debian/ buster-backports main 


Yep.

>Please do edit the article in a way that you think would make that
>obvious to anyone in future!


Done.

>>Prior to emailing the list, I tried several combinations of hosts and paths
>>before giving up and just using deb.debian.org.
>
>The other thing the article should make clear, but obviously
>doesn't, is that if you try to use anything that's not in the list
>of "available mirrors" then it won't work, and you should just use
>it directly.
>
>So even if someone did not spot that they could use
>"ftp.uk.debian.org" instead of "deb.debian.org", I would have hoped
>that they would then think, "oh, it's not in the list, so I will
>need to use it directly". Which you did eventually do but not until
>getting frustrated trying a few things.
>
>So if you can think of some wording that would have made it clear
>that any attempt to jam "deb.debian.org" onto the end of the
>apt-cacher URI would not work, because it's not in the allowed list,
>please do add it!


Done.

>The other thing is, maybe it was not clear that just not using the
>apt-cacher is not a big deal? All it does is save your data transfer
>quota the bytes of downloading the packages and maybe is quicker if
>another customer already downloaded them. That was a bigger deal back
>in 2007 when customers had 100GB/month to use. Not so much today
>when it's 4TB. Just following the Debian backports instructions
>directly leads to something that works the way it's expected to.


Also added to the wiki. From above I inferred that downloads from
apt-cacher are not counted towards customer data transfer quota.

>>I'm unsure how to derive the correct cached source URL, given that
>>the official backport instructions don't give a source URL which uses
>>ftp.uk.debian.org or backports.debian.org (or any of the other mirrors
>>listed in the wiki page).
>
>Maybe this is clearer now that I've explained that
>"deb.debian.org/debian" and "ftp.uk.debian.org/debian" are
>essentially interchangeable?


Yes.

>>I suspect a few small additions in the places I've highlighted
>>(Debian pages too, not just Bitfolk) would have made things
>>clearer.
>
>Just make the changes to the BitFolk wiki articles that you think
>make things clearer. If you make mistakes then I or someone else
>will correct them hopefully without destroying the intent of what
>you added. As it stands I don't know how to make them clearer maybe
>because I don't know what people don't know, if you see what I mean.
>Like the fact that the article pre-dates the existence of
>deb.debian.org and I didn't know that Debian pages were using
>"deb.debian.org/debian" more consistently as opposed to saying
>things like "YOUR-DEBIAN-MIRROR/debian".


Right ;-) Please make any corrections necessary.

>I don't think I want to add deb.debian.org to the apt-cacher allowed
>list as I don't really see the point in it doing a geo-ip
>redirection back to some other UK mirror. May as well just
>explicitly use ftp.uk. I think I'd rather that people understood
>that if they want to use a mirror that's not cached by apt-cacher
>then just use it directly.


Yes that makes sense.

There is one remaining question. backports.debian.org is listed as
one of the available mirrors, but I still have no idea how one would
go about using that, or even whether it still makes sense to do so, rather than just using

     deb http://apt-cacher.lon.bitfolk.com/debian/ftp.uk.debian.org/debian/ buster-backports main


It was the presence of backports.debian.org in the list combined with
the lack of any mention of that host in the official backports
instructions which confused me most. The fact that it's listed in the
wiki makes it look very much like that's what you should use to get
backports, but AFAICS that's not (necessarily) the case.