Re: src: continued use of Subversion for getting updates

From: Steffen Nurpmeso <steffen_at_sdaoden.eu>
Date: Sat, 26 Dec 2020 19:06:56 +0100
Hello.

Ulrich Spörlein wrote in
 <X+cT9s0pIWILiOT0_at_acme.spoerlein.net>:
 |On Sat, 2020-12-26 at 02:55:30 +0100, Steffen Nurpmeso wrote:
 |>Warner Losh wrote in
 |> <CANCZdfpidSxN-3fqWC8z=9ANytzF9y9WcgmEejGTsihfa-Pr-Q_at_mail.gmail.com>:
 |>|On Fri, Dec 25, 2020 at 3:25 PM Steffen Nurpmeso <steffen_at_sdaoden.eu> \
 |>|wrote:
 |>|> Warner Losh wrote in
 |>|>  <CANCZdfpNM19OyQtXH0OLHxzrQf4dqpBcRZzfBCuKfRyo5Wcziw_at_mail.gmail.com>:
 |>|>|> On Fri, Dec 25, 2020 at 2:41 PM Steffen Nurpmeso <steffen_at_sdaoden.eu>
 |>|>|> wrote:
 |>|>|>> Ulrich Spörlein wrote in
 |>|>|>>  <X+ZUa7NyatH2ktYI_at_acme.spoerlein.net>:
 |>|>|>>|On Wed, 2020-12-23 at 15:35:45 +0100, Steffen Nurpmeso wrote:
 |> ...
 |>|>|>>|>I really dislike that vendor imports have been tagged.  Because
 |> ...
 |>|>|>>|That's basically what was done? I don't understand what you're \
 |>|>|>>|saying
 |>|>|>>|here ...
 |>|>|>> Well, cgit-beta did not have had all these tags if i recall
 |> ...
 |>|>|> It had them, but not under the refs/head/vendor space but under the
 |>|>|> refs/vendor space.
 |>|>
 |>|> These are not tags but branches.  I have nothing against the
 |> ...
 |>|>|> They did exist. They were under refs/vendor rather than
 |>|> refs/head/vendor
 |>|>|> though.
 |>|>
 |>|> Under refs/tags/vendor?  refs/tags/ is the "special" namespace
 |>|> managed by "git tag", this is different than the rest.
 |> ...
 |>|>|>> there is
 |>|>|>>
 |>|>|>>   #?0|kent:free-src.git$ git sr|wc -l
 |>|>|>>   2137
 |>|>|>>
 |>|>|>> but if i go for "the real" FreeBSD itself it is just
 |>|>|>>
 |>|>|>>   #?0|kent:free-src.git$ git sr | grep -v vendor | wc -l
 |>|>|>>   19
 |>|>|>
 |>|>|You might be happier tracking on github, once we start pushing there as
 |>|> the
 |>|>|vendor branches won't be published there.
 |>|>
 |>|> No problem with any number of branches, Warner.  Just tags under
 |>|> refs/tags this is above.
 |>|
 |>|They were tags in the cgit-beta as well (and a few branches). I don't
 |>|believe that detail changed, but my old copies of the repo are gone.
 |>
 |>Yes mine is gone, for a month.  Then i am sorry that i did not
 |>speak out by then.
 |
 |They were annotated tags, git doesn't care where you actually store 
 |them. Yes, refs/tags/ is special but more in terms of automatically 
 |pulling them down, not in whether that has commit objects or tag 
 |objects.

I know that.  The git maintainer is prowd of being able to use
neat tricks like storing the OpenPGP key in the repo like that.
_I_ was only talking about refs/tags from the beginning, was i.

  ...
 |>|You may be happier running custom refs, or grab from github. The \
 |>|source of
 |>|truth has a lot of stuff. We may work to prune some of the vendor \
 |>|branches
 |>
 |>I am not interested in the entire repo, yes.  For almost a decade
 ...
 |You say you're not interested in the entire repo, but as soon as you 
 |fetch `main` or any of the stable branches you get the entire repo 
 |anyway. So I don't understand what you're talking about ...

:)  That depends on the project does it.  Just imagine NetBSD with
all of its topic branches and all the non-starters, and the
starters, the fully fledged regression tests etc etc etc, a real
fan or project mentor etc wants to have it all, but to me this is
useless.  I would consider also tracking the doc branch.

Interesting would be how many savings can be achieved by storing
it all in one repository, source, tests, doc, etc etc.
The Plan9 people with their fossil/venti storage system did
measure that, and even though they used all that image, sound and
video media the curve of newly created storag€ blocks (of unique
hashes) became flatter and flatter over time.

  ...
 |>|>|>>|That's a valid point, we debated whether to keep vendor tags and
 |>|>|>> decided
 |>|>|>>|for now to replicate what we have in SVN. We can still delete \
 |>|>|>>|all the
 |>|>|>>|vendor tags on the main repo anytime we want ...
  ...
 |>|>|We need tags to keep track of what's been done, and to revert and do
 |>|> other
 |>|>|management things with vendor imports.
 |>|>
 |>|> But why?  You have the commit on a topic/vendor branch, and yppou
  ...
 |>|I'm not sure I follow. Unless I misunderstand, you are describing a
 |>|different problem with different issues.
 |>
 |>No, maybe i was mistaken.  I never did a vendor import myself.
 |>..But looking at the history i see lots of import disasters ;-))
 |>Look for example at llvm 10.x this year, it consumed three _tags_!
 |>
 |>I am luckily free to say that this is merde (without the intention
 |>to annoy the poor one who did it) and am going to talk.  I presume
 ...
 |>There should be a real per-vendor branch, say [vendor/sqlite].
 |>The way FreeBSD seems to do vendor imports this should even be
 |>easy, since vendor stuff is usually in separate directories.
 |>Create it once it is needed first, merge into main via --no-ff so
 |>that you get real "merge commits".  You can see the difference
 |>below.
 |
 |Umm, no offense but wtf are you talking about? I asked this earlier but 
 |you didn't reply to that part of the question. Of course the vendor 
 |branches are stand-alone vendor branches.
 |
 |% git log --reverse --compact-summary -n2 origin/vendor/sqlite3

  ...
 |% git ls-tree -r origin/vendor/sqlite3

Ok fine -- great!  I cannot do _that_ locally.
(But the tags i have, there you go.)

  ...
 |They are tagged because we tagged them in SVN and in SVN it's very 
 |annoying to get the history of a branch only. And yes, we talked in the 
 |WG about dropping the tags eventually, as they serve little purpose.

Especially if you can simply log the vendor branch i'd say.

  ...
 |Please explain how this is different from the above? I'm really not sure 
 |what you're saying here ...

Not much, not much.  Maybe enforcing --no-ff when doing
git(1)-based merges in the future.

  ...
 |>|But the nice thing about refs is that you can always change them 
 |> later if
 |>|the current scheme isn't working out... So far, it is for most people, so
 |>
 |>This requires forced pushes and thus special configuration.
 |>"Normally", it is said, this should be avoided.  The fossil scm
 |>does not even support it i think.  I for myself do that, but only
 |>on development branch, release and stable and master branches are
 |>immutable (but in disasters).
 |
 |Adding or removing refs or tags doesn't need a force push. Please stop 
 |spreading this misinformation.

Everything is a reference, is it.  It depends on the refspec
specification and/or --force, as described for "<refspec>" unders
OPTIONS in git-push(1).
Whereas it may be allowed for free-standing tags, i hope i never
tried messing around with any published tags, shouldn't tags be
some fixed points on a timeline.
Also much depends upon your local configuration, i wrote mine many
years ago and am using aliases ever since mostly, so i am not
competent to talk about that.  Many, many things have been added
to git compared to when i learned it.

 |As for your network trouble, have you measured the bandwidth difference 

That is German D-Netz trouble.  That has nothing to do with
FreeBSD.  With FreeBSD there was trouble because of a server
misconfiguration i would say, i had cloned cgit-beta and then you
rewrote the history, and then it became impossible for me to
simply re-sync via "git fetch", as i have shown in my posts.

 |between: git fetch and git fetch --no-tags and if there's a big 
 |difference I'd suggest to simply use the later?

Whereas .. i could surely do that, it was more about FreeBSD and
its repository layout.

 |I just tried to measure the difference with trafshow and it was pretty 
 |much 9k up 560k down for both flags. Then I deleted all my local copies 
 |of the tags and it went to 8k up, 560k down. Probably a measurement 
 |error. Am I holding this wrong?

I have absolutely zero idea of how much this increases the
necessary network bandwidth for the FreeBSD project.  I know that
the difference goes into megabytes for my small things each day.
I know the git people were talking about improving the efficiency
of the compare-the-head-references step, it flew by one day.

Ciao,

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)
Received on Sat Dec 26 2020 - 17:08:12 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:41:26 UTC