Re: src: continued use of Subversion for getting updates

From: Steffen Nurpmeso <steffen_at_sdaoden.eu>
Date: Fri, 25 Dec 2020 22:40:41 +0100
Ulrich Spörlein wrote in
 <X+ZUa7NyatH2ktYI_at_acme.spoerlein.net>:
 |On Wed, 2020-12-23 at 15:35:45 +0100, Steffen Nurpmeso wrote:
 |>Jeffrey Bouquet wrote in
 |> <E1ks2I6-0005nh-BQ_at_rmmprod05.runbox>:
 |>|On Wed, 23 Dec 2020 11:13:07 +0100, Johan Hendriks <joh.hendriks_at_gmail.c\
 |>|om> wrote:
 |>|> On 23/12/2020 09:49, Warner Losh wrote:
 |>|>> On Wed, Dec 23, 2020, 1:48 AM Graham Perrin <grahamperrin_at_gmail.com> \
 |>|>> wrote:
 |> ...
 |>|> First of all a big thank you for all your time and effort you and all
 |>|> the other people put in this tremendous task.
 |>
 |>Yes, it is great to have FreeBSD as a stable git repository now,
 ...
 |>I really dislike that vendor imports have been tagged.  Because
 |>there is only one tag namespace you cannot avoid getting all this
 |>cruft.  I mean, it is too late now, but one could have used
 |>per-vendor import branches and step them via "git rm -rf '*' &&
 |>tar -xf newball && git add . && git commit bla" or whatever, and
 |>then join them in.  It does not matter for those who have all the
 |
 |That's basically what was done? I don't understand what you're saying 
 |here ...

Well, cgit-beta did not have had all these tags if i recall
correctly, did it?  I mean it has been two months or so since
i last had it because "git fetch" bailed here due to the errors
that i have reported, and fetching more than a gigabyte for
brand-new fetches devastates here.

But i _think_ all the tags below refs/tags/vendor/ like

  vendor/wpa/2.9
  vendor/wpa_supplicant/0.3.8
  vendor/wpa_supplicant/0.5.10
  vendor/wpa_supplicant/0.5.11
  vendor/x86emu/4.6
  vendor/xe/1.13

etc. did not exist in cgit-beta?  I surely would have said
something once comments have been requested, wouldn't i?
The thing is if i do

  #?0|kent:free-src.git$ git ls-remote|wc -l
  From https://git.freebsd.org/src.git
  6814

This is a tremendous amount of head references that need to be
compared.

  #?0|kent:free-src.git$ cd ../net-src.git/
  #?0|kent:net-src.git$ git ls-remote|wc -l
  From https://github.com/NetBSD/src.git
  609
  #?0|kent:net-src.git$ cd ../open-src.git/
  #?0|kent:open-src.git$ git ls-remote|wc -l
  From https://github.com/openbsd/src.git
  34
  #?0|kent:open-src.git$ cd ../dfly-src.git/
  #?0|kent:dfly-src.git$ git ls-remote|wc -l
  From git://git.dragonflybsd.org/dragonfly.git
  349

Linux is a little bit more

  #?0|kent:linux.git$ git ls-remote|wc -l
  From https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/
  7019

but this specific repository tracks all release candidates and all
the update releases, which alone go into the hundreds.

Anyhow, the difference is the number of local references that
effectively have to be compared against remote heads.  For example
my local Linux repo is this:

  [remote "origin"]
  url = https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/
  fetch = +refs/heads/linux-4.19.y:refs/remotes/origin/linux-4.19.y
  fetch = +refs/heads/linux-5.10.y:refs/remotes/origin/linux-5.10.y
  fetch = +refs/heads/master:refs/remotes/origin/master

And if i "sr" (show-ref) i get:

  #?0|kent:linux.git$ git sr|wc -l
  925

which is a lot due to Linux update policy, but it only relates to
the project itself, there is not one reference of what is
effectively vendor data like for example

    Merge tag 'wireless-drivers-next-2020-12-12' of
    git://git.kernel.org/pub/scm/linux/kernel/git/kvalo/wireless-drivers-next

may it be in refs/tags or anywhere else, whereas with FreeBSD and

  [remote "origin"]
  url = https://git.freebsd.org/src.git
  fetch = +refs/heads/releng/5.5:refs/remotes/origin/releng/5.5
  fetch = +refs/heads/releng/6.4:refs/remotes/origin/releng/6.4
  fetch = +refs/heads/releng/7.4:refs/remotes/origin/releng/7.4
  fetch = +refs/heads/releng/8.4:refs/remotes/origin/releng/8.4
  fetch = +refs/heads/releng/9.3:refs/remotes/origin/releng/9.3
  fetch = +refs/heads/releng/10.3:refs/remotes/origin/releng/10.4
  fetch = +refs/heads/releng/11.4:refs/remotes/origin/releng/11.4
  fetch = +refs/heads/releng/12.2:refs/remotes/origin/releng/12.2
  fetch = +refs/heads/stable/12:refs/remotes/origin/stable/12
  fetch = +refs/heads/main:refs/remotes/origin/main

there is

  #?0|kent:free-src.git$ git sr|wc -l
  2137

but if i go for "the real" FreeBSD itself it is just

  #?0|kent:free-src.git$ git sr | grep -v vendor | wc -l
  19

namely

  a62107ed19a1095158f454132a3b6ec536a4de7c refs/remotes/origin/main
  8411c9ac24aabdbbde468778b35da58dc3c15178 refs/remotes/origin/releng/10.4
  4adbf1f6686ffdc4c0555a018e14ec63b5981534 refs/remotes/origin/releng/11.4
  2120d07af09cb830873554ba5405c5d3e51b41cc refs/remotes/origin/releng/12.2
  d4c364b3dcf6dfe4e20c70d31912aad7e6219c14 refs/remotes/origin/releng/5.5
  0df48150179039ce25e09b48c85aa39bffdbb4c4 refs/remotes/origin/releng/6.4
  1c7fe7463d0204f08806b9da6d5d50fb7f75c5ad refs/remotes/origin/releng/7.4
  554a1309dbd71bb59ed4e97ea5e0bc829dad0f04 refs/remotes/origin/releng/8.4
  b06b7e647fe7b4eefbd29369cf0c24e1902bf72a refs/remotes/origin/releng/9.3
  f4d0bc6aa6b90cbb0ea6cb993d9a10e36f5f4a4c refs/remotes/origin/stable/12
  aed278eb8da79777d64e06bbc94ee0a018885477 refs/tags/release/10.3.0
  20dbcb9a99d5c4b8ceb4897f27c53ab9fb853167 refs/tags/release/11.4.0
  906434f95e83f03e0ac62d2544dad1656b5df1e9 refs/tags/release/12.2.0
  1185954e6d623ada4ef32f25687ecd5e5c003545 refs/tags/release/5.5.0
  44b4768239e96f6d6ba8a43cb2542e1bc83dec94 refs/tags/release/6.4.0
  c31c07d1a9283bd28bb00ce519dfca69e529f452 refs/tags/release/7.4.0
  174c77559c8cc317c743876f55f02bec233735f4 refs/tags/release/8.1.0
  45cf4d6a59488ea4113bde749bb61de33882cc46 refs/tags/release/8.4.0
  1fb916120b12fdc14c7f93aa5f14849db0efa166 refs/tags/release/9.3.0

and thus

  #?0|kent:free-src.git$ git sr | grep vendor | wc -l
  2118

Which is a pity since all these references will be checked during
"git fetch" unless i am mistaken.

 |>repository, but you decided to loose one of the strengts of git,
 |>selective tracking.  Also i think it causes updates to require
 |>more network traffic, i see this with the repos i have at
 |>repo.or.cz, the one with few heads/tags is minimal, the other
 |>requires hundreds of kilobytes just for the check that happens
 |>many times a day.  All these references have to be compared each
 |>and every time.  I think.  On the other hand, a few years back
 |>i accidentally "heard" a discussion about improving the network
 |>protocol and that "head" reference matching, iirc, so it may
 |>change in the future.
 |
 |That's a valid point, we debated whether to keep vendor tags and decided 
 |for now to replicate what we have in SVN. We can still delete all the 
 |vendor tags on the main repo anytime we want ...

I personally would track that in the commit message of the import
on the vendor branch that anyway exists(!), and then when merging
this into the mainline, but not create a real tag in the tag
namespace.  Also the backups/ and such, because why?

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)
Received on Fri Dec 25 2020 - 20:40:52 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:41:26 UTC