Re: src: continued use of Subversion for getting updates

On Fri, Dec 25, 2020 at 3:25 PM Steffen Nurpmeso <steffen_at_sdaoden.eu> wrote:

> Warner Losh wrote in
>  <CANCZdfpNM19OyQtXH0OLHxzrQf4dqpBcRZzfBCuKfRyo5Wcziw_at_mail.gmail.com>:
>  |> On Fri, Dec 25, 2020 at 2:41 PM Steffen Nurpmeso <steffen_at_sdaoden.eu>
>  |> wrote:
>  |>> Ulrich Spörlein wrote in
>  |>>  <X+ZUa7NyatH2ktYI_at_acme.spoerlein.net>:
>  |>>|On Wed, 2020-12-23 at 15:35:45 +0100, Steffen Nurpmeso wrote:
>  |>>|>Jeffrey Bouquet wrote in
>  |>>|> <E1ks2I6-0005nh-BQ_at_rmmprod05.runbox>:
>  |>>|>|On Wed, 23 Dec 2020 11:13:07 +0100, Johan Hendriks
>  |>> <joh.hendriks_at_gmail.c\
>  |>>|>|om> wrote:
>  |>>|>|> On 23/12/2020 09:49, Warner Losh wrote:
>  |>>|>|>> On Wed, Dec 23, 2020, 1:48 AM Graham Perrin <
>  |>> grahamperrin_at_gmail.com> \
>  |>>|>|>> wrote:
>  |>>|> ...
>  |>>|>|> First of all a big thank you for all your time and effort you and
>  |>> all
>  |>>|>|> the other people put in this tremendous task.
>  |>>|>
>  |>>|>Yes, it is great to have FreeBSD as a stable git repository now,
>  |>>  ...
>  |>>|>I really dislike that vendor imports have been tagged.  Because
>  |>>|>there is only one tag namespace you cannot avoid getting all this
>  |>>|>cruft.  I mean, it is too late now, but one could have used
>  ...
>  |>>|That's basically what was done? I don't understand what you're saying
>  |>>|here ...
>  |>>
>  |>> Well, cgit-beta did not have had all these tags if i recall
>  |>> correctly, did it?  I mean it has been two months or so since
>  |>> i last had it because "git fetch" bailed here due to the errors
>  |>> that i have reported, and fetching more than a gigabyte for
>  |>> brand-new fetches devastates here.
>  |>
>  |> It had them, but not under the refs/head/vendor space but under the
>  |> refs/vendor space.
>
> These are not tags but branches.  I have nothing against the
> branches, of course.  Only the tags are the problem.
>
>  |> The multiple gigabyte fetch is because we changed the hashes two or
> three
>  |> times in the last few months.
>
> Yes i know.  No problem (well, for me, of course), i tried it at
> least once more by the end of November, but the server did not
> finish my request (the simple "git fetch" in a non-clean repo).
>
>  |> But i _think_ all the tags below refs/tags/vendor/ like
>  |>>
>  |>>   vendor/wpa/2.9
>  |>>   vendor/wpa_supplicant/0.3.8
>  |>>   vendor/wpa_supplicant/0.5.10
>  |>>   vendor/wpa_supplicant/0.5.11
>  |>>   vendor/x86emu/4.6
>  |>>   vendor/xe/1.13
>  |>>
>  |>> etc. did not exist in cgit-beta?  I surely would have said
>  |>> something once comments have been requested, wouldn't i?
>  |>
>  |> They did exist. They were under refs/vendor rather than
> refs/head/vendor
>  |> though.
>
> Under refs/tags/vendor?  refs/tags/ is the "special" namespace
> managed by "git tag", this is different than the rest.
>
>  |>> The thing is if i do
>  |>>
>  |>>   #?0|kent:free-src.git$ git ls-remote|wc -l
>  |>>   From https://git.freebsd.org/src.git
>  |>>   6814
>  |>>
>  |>> This is a tremendous amount of head references that need to be
>  |>> compared.
>  ...
>  |>> there is
>  |>>
>  |>>   #?0|kent:free-src.git$ git sr|wc -l
>  |>>   2137
>  |>>
>  |>> but if i go for "the real" FreeBSD itself it is just
>  |>>
>  |>>   #?0|kent:free-src.git$ git sr | grep -v vendor | wc -l
>  |>>   19
>  |>
>  |You might be happier tracking on github, once we start pushing there as
> the
>  |vendor branches won't be published there.
>
> No problem with any number of branches, Warner.  Just tags under
> refs/tags this is above.
>

They were tags in the cgit-beta as well (and a few branches). I don't
believe that detail changed, but my old copies of the repo are gone.


>   ...
>  |>> and thus
>  |>>
>  |>>   #?0|kent:free-src.git$ git sr | grep vendor | wc -l
>  |>>   2118
>  |>>
>  |>> Which is a pity since all these references will be checked during
>  |>> "git fetch" unless i am mistaken.
>  |>
>  |Yes.  So far it's been doing it quite quickly for me, but I'm decently
>  |connected...
>
> Yes, terrible here, shared with many.
>

You may be happier running custom refs, or grab from github. The source of
truth has a lot of stuff. We may work to prune some of the vendor branches
into their own repos in the future, but today there's a lot of stuff that's
there, some of which is for the convenience of the developer and you may
need to trim (at least in the short term).

  ...
>  |>>|That's a valid point, we debated whether to keep vendor tags and
>  |>> decided
>  |>>|for now to replicate what we have in SVN. We can still delete all the
>  |>>|vendor tags on the main repo anytime we want ...
>  |>>
>  |>> I personally would track that in the commit message of the import
>  |>> on the vendor branch that anyway exists(!), and then when merging
>  |>> this into the mainline, but not create a real tag in the tag
>  |>> namespace.  Also the backups/ and such, because why?
>  |>
>  |We need tags to keep track of what's been done, and to revert and do
> other
>  |management things with vendor imports.
>
> But why?  You have the commit on a topic/vendor branch, and yppou
> revert nothing but the commit.  In fact doing so messes the tag,p
> it has to be retagged when you do re-commit an import proper,
> which requires a forced push even!
>

I'm not sure I follow. Unless I misunderstand, you are describing a
different problem with different issues.

But the nice thing about refs is that you can always change them later if
the current scheme isn't working out... So far, it is for most people, so
we're not changing the vendor import stuff right away.  I've taken note of
your issues and will keep that in mind if there's others with the same
problems.

Warner