Re: Please check the current beta git conversions

From: Steffen Nurpmeso <steffen_at_sdaoden.eu>
Date: Thu, 03 Sep 2020 21:14:10 +0200
For one: thanks all, it now works!

Steffen Nurpmeso wrote in
 <20200903151825.G_Rv9%steffen_at_sdaoden.eu>:
 |Renato Botelho wrote in
 | <d0224511-dec0-44f4-4904-ad5b55f4edc7_at_FreeBSD.org>:
 ||On 02/09/20 20:20, Steffen Nurpmeso wrote:
 ||> Ed Maste wrote in
 ||>   <CAPyFy2Ajr3C8LM=x=_WOhV9XXMFN-80_8AP5SSta7a1FgaTGRA_at_mail.gmail.com>:
 ||> 
 ||> I tried simply updating my github clone by switching
 ||> 
 ||>    url = https://cgit-beta.freebsd.org/src.git
 ||>    #url = https://github.com/freebsd/freebsd.git
 ||> 
 ||> and whereas ls-remote worked fine fetch -v --dry-run aborted as
 ||> well as normal fetch, after dumping dozens of POSTs
 ||> 
 ||>    POST git-upload-pack (gzip 1472 to 804 bytes)
 ||>    ...
 ||>    POST git-upload-pack (gzip 976722 to 483608 bytes)
 ||>    POST git-upload-pack (chunked)
 ||>    error: RPC failed; HTTP 413 curl 22 The requested URL returned \
 ||>    error: 413
 ||>    fatal: the remote end hung up unexpectedly
 ||> 
 ||> Maybe clone from scratch instead, but mysterious it is?
 ||> Good night, and Ciao from Germany,
 ||
 ||github and cgit-beta repositories are not the same.  Commit hashes won't 
 ||match so you cannot simply change the URL.
 |
 |Yes i know that, but most of the blobs are of course the same, no?

Having said that, i switched to git in i think 2011 and never ever
read anything about the protocol or almost anything else ever
since ...  Except that rev-parse changed its output order which
broke a (primitive) script (git-topic-creator) of mine, but that
was in 2013.

So: it may well be that git does not really take care for finding
local data blobs unless there are commits with shared ancestors or
what (the two repositories anyway show up as "warning: no common
commits").  Let's see where this ends...

That reminds me of the fossil/venti storage system of Plan9, with
the short-time local cache and the hash-based backing store, which
just stores digested blocks.  There was a graphic which showed how
data consumption exploded at the beginning, but the curve went
flatter and flatter, and it was explicitly noted that the masses
of people in Bell Labs did a lot with multimedia, videos and such.
Which i found surprising.

Old github pack
  -r--r----- 1 steffen code 1526603739 Aug  2 01:47 pack-92bcd61e50e1a09e27e8aa4a67e0878468f01f79.pack
  -r--r----- 1 steffen code  136779336 Aug  2 01:47 pack-92bcd61e50e1a09e27e8aa4a67e0878468f01f79.idx

  warning: no common commits
  remote: Enumerating objects: 920487, done.
  remote: Counting objects: 100% (920487/920487), done.
  remote: Compressing objects: 100% (254705/254705), done.
  remote: Total 4909613 (delta 876272), reused 665791 (delta 665782), pack-reused 3989126
  Receiving objects: 100% (4909613/4909613), 1.25 GiB | 396.00 KiB/s, done.

Hm.  It definetely must have downloaded masses of duplicate data.
I must admit, the largest repo which had to be updated very often
(despite the borked NetBSD thing on github, years ago) was
ghostscript.  There i looked, but find it quite different, if
i recall correctly.  I did not blame git for the mess there.

  Resolving deltas: 100% (3429285/3429285), done.
  From https://cgit-beta.freebsd.org/src
   + 606d234876ce...101374bc1b34 releng/5.5         -> origin/releng/5.5  (forced update)
   + 2132c100d27f...385a94b8751c releng/6.4         -> origin/releng/6.4  (forced update)
   + c462a28e0cae...f864cd1bca00 releng/7.4         -> origin/releng/7.4  (forced update)
   + 3ffe9881628f...b4323ad1fd9f releng/8.4         -> origin/releng/8.4  (forced update)
   + 34073209c9c9...6845fb3a0339 releng/9.3         -> origin/releng/9.3  (forced update)
   + 57fbb64699be...c032bc8ca5d4 releng/10.3        -> origin/releng/10.4  (forced update)
   + da2894488950...889f726543f5 releng/11.4        -> origin/releng/11.4  (forced update)
   + e1a11e350964...6fb517db6dea releng/12.1        -> origin/releng/12.1  (forced update)
   + a9ff142f7dcc...6a26cffb5427 stable/12          -> origin/stable/12  (forced update)
   + c660a28c68a8...88fc88f77538 main               -> origin/main  (forced update)

A masterpiece!

   + 16be7204f705...ef7e23176ea6 refs/notes/commits -> refs/notes/commits  (forced update)
   * [new tag]                   release/10.3.0     -> release/10.3.0
   * [new tag]                   release/11.4.0     -> release/11.4.0
   * [new tag]                   release/12.1.0     -> release/12.1.0
   * [new tag]                   release/5.5.0      -> release/5.5.0
   * [new tag]                   release/6.4.0      -> release/6.4.0
   * [new tag]                   release/7.4.0      -> release/7.4.0
   * [new tag]                   release/8.1.0      -> release/8.1.0
   - [new tag]                   release/8.4.0      -> release/8.4.0
   - [new tag]                   release/9.3.0      -> release/9.3.0

  -r--r----- 1 steffen code 1344409784 Sep  3 20:44 pack-0f656d44c6c8301d98ee51b69c3aacd8db9255ac.pack
  -r--r----- 1 steffen code  137470236 Sep  3 20:44 pack-0f656d44c6c8301d98ee51b69c3aacd8db9255ac.idx

I then garbage collected aggressively and pruned the two into

  -r--r----- 1 steffen code 1332929811 Sep  3 21:11 pack-36abb45e074081791b5f3595bdefe8b48b0223bb.pack
  -r--r----- 1 steffen code  147240528 Sep  3 21:12 pack-36abb45e074081791b5f3595bdefe8b48b0223bb.idx

And you know what always amazes me:

  $ du -sh /x/os/linux.git/.git/objects/pack/
  1.7G    /x/os/linux.git/.git/objects/pack/

which is

        url = https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/
        fetch = +refs/heads/linux-4.19.y:refs/remotes/origin/linux-4.19.y
        fetch = +refs/heads/linux-5.8.y:refs/remotes/origin/linux-5.8.y

Ciao.
And thanks for fixing!

--steffen
|
|Der Kragenbaer,                The moon bear,
|der holt sich munter           he cheerfully and one by one
|einen nach dem anderen runter  wa.ks himself off
|(By Robert Gernhardt)
Received on Thu Sep 03 2020 - 17:14:13 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:41:25 UTC