Re: OpenSSL breaks factor(6)

On Fri, Dec 27, 2019 at 08:42:53PM -0800, Rodney W. Grimes wrote:
> > On Fri, Dec 27, 2019 at 07:00:04PM -0800, Rodney W. Grimes wrote:
> > > > On Fri, Dec 27, 2019 at 01:47:17PM -0800, Steve Kargl wrote:
> > > >  
> > > > This patch now includes a fix for hexadecimal conversion.  It
> > > > simple scans the string for a hex digit in [a,...,f] and assumes
> > > > that a hexadecimal string has been entered.  A string that includes
> > > > character from the decimal digits is assumed to by a decimal
> > > > representation.  
> > > 
> > > It looks to me that the old code did the common method of
> > > try to convert as decimal, if that fails, try it as hex,
> > > if that fails report an error.
> > > 
> > > Why is is that this common logic no longer works?
> > 
> > AFAICT, BN_dec2bn and BN_hex2bn from OpenSSL scan from left
> > to right, does a conversion with what is possible, and reports
> > success.  That is, for 1abc, BN_dec2bn can convert 1 to 1 and
> > reports success.  The local implementations of these functions,
> > when OpenSSL is not used, does not do this partial conversion.
> 
> I think I see now, the local implementaton checks for whole
> string conversion with a test for newline or null as the last
> byte converted by strtoul, the OpenSSL does not do this.
> 

My patch fixes that.  The manpage documents that '1abcp' should
convert '1abc'.  The 'p' simply terminates the conversion.  The
local implementations actually flags an error.  I suspect the
logic never worked as intended.  The use of OpenSSL functions 
in factor(6) was introduced in r104722 by fanf_at_.

> So why ever use the, um, IMHO broken for this application,
> SSL versions of these functions?  Or if we do need to use
> them for some reason apply the whole string conversion
> checks as wrappers around them?

You'll need to ask fanf_at_, but I suspect the SSL version 
was introduced to allow the factoring of integers that
exceed (uint64_t)(-1).

-- 
Steve