Re: OpenSSL breaks factor(6)

> On Sat, Dec 28, 2019 at 10:46:52PM -0500, Garance A Drosehn wrote:
> > On 27 Dec 2019, at 17:42, Steve Kargl wrote:
> > >
> > > This patch now includes a fix for hexadecimal conversion.  It
> > > simple scans the string for a hex digit in [a,...,f] and assumes
> > > that a hexadecimal string has been entered.  A string that includes
> > > character from the decimal digits is assumed to by a decimal
> > > representation.
> > 
> > What if the user wants to factor a hexadecimal value which does not
> > happen to include [a...f]?
> > 
> > How about recognizing a prefix of '0x' as a way to indicate the value
> > is hexadecimal?
> > 
> 
> An interested user will need to add that support.  AFAIK, factor(6)
> has never recognized the 0x prefix, and I'm not trying to add new 
> features.  I'm simply fixing factor(6) to match its documentation, 
> and trying to ensure WITH_OPENSSL and WITHOUT_OPENSSL give the 
> same results where applicable.
> 
> The logic is to first try to convert the string to a decimal if
> the leading digits are members of the set [0,...,9].  If this
> conversion fails, then try to convert the string as a hexadecimal
> number.  A problem occurs because OpenSSL's BN_dec2bn does not fail
> for a number like '1abc' (converts it to 1) whereas the local
> implementation of BN_dec2bn fails during the conversion, and so
> the BN_hex2bn function is executed and '1abc' is converted.

Wasnt the hex conversion undocumented? 
Since it seems to have issues, and is of dubious value
might it might be best to just remove it?

> -- 
> Steve
-- 
Rod Grimes                                                 rgrimes_at_freebsd.org