Re: src: continued use of Subversion for getting updates

Warner Losh wrote in
 <CANCZdfpidSxN-3fqWC8z=9ANytzF9y9WcgmEejGTsihfa-Pr-Q_at_mail.gmail.com>:
 |On Fri, Dec 25, 2020 at 3:25 PM Steffen Nurpmeso <steffen_at_sdaoden.eu> \
 |wrote:
 |> Warner Losh wrote in
 |>  <CANCZdfpNM19OyQtXH0OLHxzrQf4dqpBcRZzfBCuKfRyo5Wcziw_at_mail.gmail.com>:
 |>|> On Fri, Dec 25, 2020 at 2:41 PM Steffen Nurpmeso <steffen_at_sdaoden.eu>
 |>|> wrote:
 |>|>> Ulrich Spörlein wrote in
 |>|>>  <X+ZUa7NyatH2ktYI_at_acme.spoerlein.net>:
 |>|>>|On Wed, 2020-12-23 at 15:35:45 +0100, Steffen Nurpmeso wrote:
 ...
 |>|>>|>I really dislike that vendor imports have been tagged.  Because
 ...
 |>|>>|That's basically what was done? I don't understand what you're saying
 |>|>>|here ...
 |>|>> Well, cgit-beta did not have had all these tags if i recall
 ...
 |>|> It had them, but not under the refs/head/vendor space but under the
 |>|> refs/vendor space.
 |>
 |> These are not tags but branches.  I have nothing against the
 ...
 |>|> They did exist. They were under refs/vendor rather than
 |> refs/head/vendor
 |>|> though.
 |>
 |> Under refs/tags/vendor?  refs/tags/ is the "special" namespace
 |> managed by "git tag", this is different than the rest.
 ...
 |>|>> there is
 |>|>>
 |>|>>   #?0|kent:free-src.git$ git sr|wc -l
 |>|>>   2137
 |>|>>
 |>|>> but if i go for "the real" FreeBSD itself it is just
 |>|>>
 |>|>>   #?0|kent:free-src.git$ git sr | grep -v vendor | wc -l
 |>|>>   19
 |>|>
 |>|You might be happier tracking on github, once we start pushing there as
 |> the
 |>|vendor branches won't be published there.
 |>
 |> No problem with any number of branches, Warner.  Just tags under
 |> refs/tags this is above.
 |
 |They were tags in the cgit-beta as well (and a few branches). I don't
 |believe that detail changed, but my old copies of the repo are gone.

Yes mine is gone, for a month.  Then i am sorry that i did not
speak out by then.

  ...
 |>|>> Which is a pity since all these references will be checked during
 |>|>> "git fetch" unless i am mistaken.
 |>|>
 |>|Yes.  So far it's been doing it quite quickly for me, but I'm decently
 |>|connected...
 |>
 |> Yes, terrible here, shared with many.
 |
 |You may be happier running custom refs, or grab from github. The source of
 |truth has a lot of stuff. We may work to prune some of the vendor branches

I am not interested in the entire repo, yes.  For almost a decade
i had only the sources as such, without history, for me this is an
improvement.  As i am not a BSD developer it is for snooping
around only.  And when i report bugs they are not fixed, but i am
not alone with this.  Nonetheless: interest in FreeBSD here, ok.

 |into their own repos in the future, but today there's a lot of stuff that's
 |there, some of which is for the convenience of the developer and you may
 |need to trim (at least in the short term).

 |  ...
 |>|>>|That's a valid point, we debated whether to keep vendor tags and
 |>|>> decided
 |>|>>|for now to replicate what we have in SVN. We can still delete all the
 |>|>>|vendor tags on the main repo anytime we want ...
 |>|>>
 |>|>> I personally would track that in the commit message of the import
 |>|>> on the vendor branch that anyway exists(!), and then when merging
 |>|>> this into the mainline, but not create a real tag in the tag
 |>|>> namespace.  Also the backups/ and such, because why?
 |>|>
 |>|We need tags to keep track of what's been done, and to revert and do
 |> other
 |>|management things with vendor imports.
 |>
 |> But why?  You have the commit on a topic/vendor branch, and yppou
 |> revert nothing but the commit.  In fact doing so messes the tag,p
 |> it has to be retagged when you do re-commit an import proper,
 |> which requires a forced push even!
 |
 |I'm not sure I follow. Unless I misunderstand, you are describing a
 |different problem with different issues.

No, maybe i was mistaken.  I never did a vendor import myself.
..But looking at the history i see lots of import disasters ;-))
Look for example at llvm 10.x this year, it consumed three _tags_!

I am luckily free to say that this is merde (without the intention
to annoy the poor one who did it) and am going to talk.  I presume
most of you can do git(1) better than i, i use it for a decade
(almost exactly in fact!), but have never stepped forward, and
have a very primitive way of daily use.

There should be a real per-vendor branch, say [vendor/sqlite].
The way FreeBSD seems to do vendor imports this should even be
easy, since vendor stuff is usually in separate directories.
Create it once it is needed first, merge into main via --no-ff so
that you get real "merge commits".  You can see the difference
below.

Then there are basically two options.  One is to simply switch to
the vendor branch, work and commit there as often as necessary,
and then merge to main when done.  The other is to instead rebase the
vendor branch to main first _before_ you start the work, any
_maybe_, _possibly_ again if the final push fails.

The former approach is perfect for vendor stuff which really
stands for itself, you can see below why this is so (the history
of the vendor branch is really only about the vendor branch, at
least after it has been created).

The latter is the only way to go if there are conflicts because
the vendor package reaches out into the base system, like i would
expect for llvm.  Then you want to rebase to main so that any
merge conflict resolving happen on the actually current state of
the art, of course, not on some old tree.

Like this the history of the vendor branch is not "clean" in that
it includes commits of the main branch that happened in the
meantime, but .. i would not care about this.
(One could also get the same vendor-branch-history-only also for
this kind, of course, easily even.  For this i would checkout the
branch, "git rm -rf '*'", do "git archive --format=tar --prefix=
main | tar -xf -", then "git add .", then "git commit -m
SYNC-WITH-MAIN", .. and then do the normal vendor branch stuff.)

Anyway it is important (i would say) to use --no-ff, so that you
get real merge commits.  This also avoids these terrible commit
messages of the actual merges one can see in FreeBSD history.
Consider this:

  #!/bin/sh

  doit() {
    mkdir y
    cd y
    git init

    touch 1 && git add 1 && git commit -m 1
    touch 2 && git add 2 && git commit -m 2

    git checkout -b vendor/x
    mkdir x
    cd x
    touch x1 && git add x1 && git commit -m x1
    cd ..

    git checkout master
    git merge --no-ff vendor/x -m MERGE-X-1

    touch 3 && git add 3 && git commit -m 3
    touch 4 && git add 4 && git commit -m 4

    git checkout vendor/x
    [ -n "$1" ] && git rebase master
    cd x
    touch x2 && git add x2 && git commit -m x2
    cd ..

    git checkout master
    git merge --no-ff vendor/x -m MERGE-X-2

    touch 5 && git add 5 && git commit -m 5

    git checkout vendor/x
    [ -n "$1" ] && git rebase master
    cd x
    touch x3 && git add x3 && git commit -m x3
    cd ..

    git checkout master
    git merge --no-ff vendor/x -m MERGE-X-3

    touch 6 && git add 6 && git commit -m 6

    echo ===VENDOR
    git log --oneline vendor/x
    echo ===MASTER
    git log --oneline master
    echo ===ALL
    git log --oneline --graph --all

    cd ..
    rm -rf y
  }

  echo ===== WITHOUT
  doit

  echo ===== WITH
  doit y

I personally would not just tag about anything.  You do not need
to, because all you need to do is to "git log
VENDOR-BRANCH-NAME", and there you see "import v1.2.3"
etc. messages.  Of course, if you always rebase the entire branch
then maybe it makes sense, but you could still use git(1) powers
via "git log --oneline --reverse --topo-order --merges [--all]",
which only gives the merge commits -- and if you give the name of
the topic/vendor branch, then you should see only merges of the
branch that is of interest, again!

 |But the nice thing about refs is that you can always change them later if
 |the current scheme isn't working out... So far, it is for most people, so

This requires forced pushes and thus special configuration.
"Normally", it is said, this should be avoided.  The fossil scm
does not even support it i think.  I for myself do that, but only
on development branch, release and stable and master branches are
immutable (but in disasters).

 |we're not changing the vendor import stuff right away.  I've taken note of
 |your issues and will keep that in mind if there's others with the same
 |problems.

Fine.

Ciao, and a nice weekend everybody,

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)