Re: pkg (aka pkgng) 1.0 released

On 7 Sep 2012, at 13:02, Ivan Voras <ivoras_at_freebsd.org> wrote:

> On 7 September 2012 13:54, Matthew Seaman
> <m.seaman_at_infracaninophile.co.uk> wrote:
>> On 09/07/12 12:30, Ivan Voras wrote:
>>> On 06/09/2012 18:44, Matthew Seaman wrote:
>>>> On 06/09/2012 16:37, Ivan Voras wrote:
>>> 
>>>>> Hi,
>>>>> 
>>>>> It looks like the pkg port installs pkg.conf.sample with the line:
>>>>> 
>>>>> PACKAGESITE         : http://pkg.freebsd.org/${ABI}/latest
>>>>> 
>>>>> ... which is finally a good step in the direction of making pkgng
>>>>> working by default, except that the "pkg.freebsd.org" site doesn't exist
>>>>> in DNS. Instead, pkgbeta.freebsd.org still exists. I suppose one should
>>>>> be a DNS CNAME for the other?
>>>> 
>>>> It's a SRV record:
>>>> 
>>>> seedling:~:% dig +short IN SRV _http._tcp.pkg.freebsd.org
>>>> 10 10 80 pkgbeta.FreeBSD.org.
>>> 
>>> Hi,
>>> 
>>> What are the benefits of doing it this way?
>> 
>> Yeah -- it's a bit OTT right now given there's just the one publicly
>> available pkg repository available.  It will pay off later when there
>> are more pkg repositories available -- repositories can be added to (or
>> removed from) the list in the SRV record without end-users having to
>> know the details.
>> 
>> It may also be possible to replicate what portupdate has done and use
>> geolocation based services to steer end users to a nearby repository
>> site automatically.
> 
> Ok, but all that can be done the "normal" way with A records.
> 
> As far as I can tell, the intended benefits of the SRV record system
> is to disentangle services and hosts, so that, e.g. the same
> user-visible DNS name (e.g. "pkg.freebsd.org") resolves to a different
> host if asked for the HTTP service and the FTP service.

That is one feature of SRV, but that part isn't really something we use for anything, nor plan to (other than the fact that you specify SRV records that way).

> I suppose this could work in FreeBSD's case if the record was created
> differently, instad of the "normal" _http._tcp record, introduce a new
> one, e.g. _pkgng._tcp, so when a web browser visits "pkg.freebsd.org"
> it gets a regular web page or some other user-visible content, and the

There is no plan to add an A record to pkg.freebsd.org, so a browser would never be able to resolve it. Adding an A record there would IMO be more likely to mislead people to think that pkg(8) (is it section 8?) fetches packages from there.

> specially made pkg client will resolve it to another service and
> another (possibly) server. This also can be done without the SRV
> record, by e.g. inspecting client's HTTP headers.
> 
> I'm not saying that the SRV record is technically wrong in this case,
> I just don't see how is it useful (and surely there will be others
> trying to ping "pkg.freebsd.org" and complaining when it fails to
> resolve).

As Chip Marshall mentions, SRV records allow us to load balancing and failover. Both are rather important in allowing us to scale pkg distribution and changing it without requiring client side changes. The fallback handling of SRV allows us, among other things, to have thin frontend servers which only have a subset of packages, and then automatic fallback to the full mirrors.

Once the initial pkg distribution sites are set up, we will also create some more SRV records, so the people who want can prioritize mirrors close to them based on regions while still falling back to mirrors longer away from them.

Oh, and anyone using portsnap or freebsd-update are already using SRV records.

For anyone interested in more details, they can read my document describing the system, which bapt implemented the SRV support in pkgng from: https://docs.google.com/document/d/1MmQOV2IUfRtdlzd1i63SJQgngjPJ8eERaSa68Xydyc0/edit

-- 
Simon L. B. Nielsen