Re: tar/libarchive gzip problem [was: portsnap corrupted]

I think I know what this is.  I tried recently to add
support for concatenated gzip files.  This involves looking
ahead for another GZip header after the end of the compressed
data.  The current version screws this up and ends up trying
to match the file CRC as a Gzip header.  About 1 in 256 files
will match the first byte, which triggers the subsequent meltdown.
This also explains why neither of us saw it in testing.
(I knew the code didn't actually work for concatenated gzip
files but didn't realize it would break decode of some
regular non-concatenated files.)

The attached patch simply disables this additional
header check, which should fix the immediate problem.
Please try it and let me know.

Apologies,

Tim

Simon L. Nielsen wrote:
> Hey Tim,
> 
> I think one of the recent changes to tar or libarchive broke gzip
> handling in some cases.  See more below.
> 
> [portsnap extract fails with gzip error]
> 
> I'm not sure why I didn't run into it in my tests, but I think the
> problem is in tar / libarchive's handling of gzip files.  Taking one
> random "broken" file [1] it fails with tar's build in decompression,
> but works using external zcat.
> 
> [1] http://portsnap1.freebsd.org/f/19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d.gz
> http://people.freebsd.org/~simon/tmp/19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d.gz
> 
> 
> Broken system:
> 
> [simon_at_eddie:/tmp] tar --version
> bsdtar 2.5.903a - libarchive 2.5.903a
> [simon_at_eddie:/tmp] uname -a
> FreeBSD eddie.nitro.dk 8.0-CURRENT FreeBSD 8.0-CURRENT #1: Tue Dec 30 22:28:33 CET 2008     simon_at_eddie.nitro.dk:/FreeBSD/obj/FreeBSD/system-CURRENT/sys/EDDIE  i386
> [simon_at_eddie:/tmp] tar tvf 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d.gz > /dev/null
> tar: Error opening archive: Invalid GZip header (saw 99 at offset 1)
> [simon_at_eddie:/tmp] zcat 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d.gz | tar tf - > /dev/null 
> [simon_at_eddie:/tmp] zcat 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d.gz | sha256 
> 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d
> 
> 
> OK system:
> 
> [simon_at_benji:/tmp] tar --version
> bsdtar 2.5.5 - libarchive 2.5.5
> [simon_at_benji:/tmp] uname -a
> FreeBSD benji.s 7.1-RC2 FreeBSD 7.1-RC2 #0: Tue Dec 23 15:18:30 UTC 2008     root_at_logan.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC  i386
> [simon_at_benji:/tmp] tar tvf 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d.gz > /dev/null 
> [simon_at_benji:/tmp] zcat 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d.gz | sha256 
> 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d
> 
> 
> Another OK system:
> 
> [simon_at_ref8-amd64:files] tar --version
> bsdtar 2.5.5 - libarchive 2.5.5
> [simon_at_ref8-amd64:files] uname -a
> FreeBSD ref8-amd64.freebsd.org 8.0-CURRENT FreeBSD 8.0-CURRENT #2 r184542:185402: Fri Nov 28 19:14:40 UTC 2008     peter_at_ref8-amd64.freebsd.org:/scratch/src/sys/amd64/compile/REF8-AMD64  amd64
> [simon_at_ref8-amd64:files] tar tvf 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d.gz > /dev/null
> [simon_at_ref8-amd64:files] zcat 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d.gz | sha256
> 19faa3b8bd15bb8f4cd9f665a7623887729f3bd834d780e8b069df979f228e8d
>