Re: CFT: BSD-licensed grep [Fwd: cvs commit: ports/textproc/bsdgrep Makefile distinfo]

Konrad Jankowski <konrad.jankowski_at_bluemedia.pl> writes:
> Dag-Erling Smørgrav <des_at_des.no> writes:
> > In any case, this is a libc issue, right?  As long as sort / grep
> > uses the API correctly, they will work fine once libc is fixed?
> Correct.  Given sort uses strcoll()/wcscoll()/strxfrm()/wcsxfrm() and
> call setlocale().  I don't know about grep.

For grep, I believe it should simply be a matter of calling setlocale(),
using wide strings, and using a multibyte regex engine (for appropriate
values of "simply").

Another thing I'm unsure about is the matter of input and output.  Do
mbstowcs() / mbtowc() simply trust the input to conform to LC_CTYPE and
convert accordingly?  When reading UTF, do they recognize and handle
BOMs, or simply treat them as zero-width non-breaking space?  In the
absence of a BOM, do they assume that the input follows the system's
native byte order?

(IMHO, the API is broken, since there is no way for the same program to
simultaneously handle streams with different encodings, but I guess it's
too late to fix that)

DES
-- 
Dag-Erling Smørgrav - des_at_des.no