Re: Ports doesnt respect fetch environment settings

From: Tom Evans <tevans.uk_at_googlemail.com>
Date: Mon, 21 Jun 2010 12:38:46 +0100
On Mon, Jun 21, 2010 at 12:07 PM, Gary Jennejohn
<gljennjohn_at_googlemail.com> wrote:
> Yes.  When you ran fetch by hand you didn't have the -ApRr on the CL.
> Could it be that the 'p' flag is causing problems?
>
> Try running fetch by hand again with those flags and see what happens.
> If it fails, try removing the 'p' flag.
>
> --
> Gary Jennejohn
>

Yes, I went through the same logic - its not the 'p' flag, its the 'A'
flag, which is supposed to prevent it following 302 redirects.

In this case, it refuses to retry the request when it receives a 407.

> # /usr/bin/fetch -ApRr -v -S 37867 http://googlecl.googlecode.com/files/googlecl-0.9.5.tar.gz
looking up proxy
connecting to proxy:3128
requesting http://googlecl.googlecode.com/files/googlecl-0.9.5.tar.gz
proxy requires authorization
fetch: http://googlecl.googlecode.com/files/googlecl-0.9.5.tar.gz:
Proxy Authentication Required
root_at_strangepork '12:13:28' '/usr/ports/net/googlecl'
> # /usr/bin/fetch -pRr -v -S 37867 http://googlecl.googlecode.com/files/googlecl-0.9.5.tar.gz
looking up proxy
connecting to proxy:3128
requesting http://googlecl.googlecode.com/files/googlecl-0.9.5.tar.gz
proxy requires authorization
looking up proxy
connecting to proxy:3128
requesting http://googlecl.googlecode.com/files/googlecl-0.9.5.tar.gz
local size / mtime: 37867 / 1276839258
remote size / mtime: 37867 / 1276839258

That doesn't seem right!

Looking in lib/libfetch/http.c it tries to fetch the file in a loop.
libfetch first tries without proxy authentication, even if you specify
it in your environment.
If the request fails due to proxy authentication being required, it
sets a flag to add proxy auth details next time through the loop, and
continues.
If the '-A' flag is set however, it will only go through this loop one
time, and so does not attempt to use the supplied proxy
authentication.
Comments in the source code imply that this is a change in behaviour
introduced at the start of the year to support digest authentication;
prior to that it would have attempted proxy auth on the first request.

The flag for '-A' is documented as:

     -A      Do not automatically follow ‘‘temporary’’ (302) redirects.
         Some broken Web sites will return a redirect instead of a
         not‐found error when the requested object does not exist.

where as the behaviour is:

Do not attempt to download this file more than once, for any reason.

Having seen this, the bug is that we wish to go thru the loop one more
time to retry with proxy authentication added, but the loop may exit
on the next iteration. This diff allows it to go through the loop one
more time, and now fetches the file correctly.

Incidentally, having fixed fetch to work with '-A' thru a proxy that
requires proxy auth, I now dont require anything in FETCH_ENV or
FETCH_*_ARGS, it works correctly with the PROXY_* environment
variables.

Patch attached.

Cheers

Tom

Received on Mon Jun 21 2010 - 09:38:48 UTC

This archive was generated by hypermail 2.4.0 : Wed May 19 2021 - 11:40:04 UTC