Bug#894217: [reprotest] kk_KZ.RK1048 again

[Agustin Henze]

Hi Ximin,

On Tue, 17 Apr 2018 21:01:00 +0000 Ximin Luo <infinity0 at debian.org> wrote:
> Inaki Malerba:
> > Hi Ximin
> >> More generally, I'd argue that build programs shouldn't fail simply when LC_ALL is unrecognised, for example gcc works perfectly fine.
> >>
> >> $ LC_ALL=ououi gcc -c /dev/null 2>/dev/null; echo $?
> >> 0
> > 
> > Totally agree.
> > Normally, it should fallback to a known config, but, as you can see in
> > the following example, the problem it's not exactly that the LC_ALL is
> > unrecognized but the LC itself.
> > A similar problem happens with Sphinx.
> > 
> > Sorry if that's not what you meant.
> > 
> > ```
> > # LC_ALL=kk_KZ.RK1048 help2man
> > Unknown encoding 'RK1048' at /usr/bin/help2man line 56.
> > 
> > # LC_ALL=bleble help2man
> > perl: warning: Setting locale failed.
> > perl: warning: Please check that your locale settings:
> > [..]
> Well, this detail does not change my main point. Why should any program silently accept LC_ALL=oeuieoui (and print warnings, that's fine) but blow up when running LC_ALL=kk_KZ.XXXXX? It should at most print a warning, not crash. Try it yourself:
> 
> $ LC_ALL=kk_KZ.XXXXXX gcc -c /dev/null 2>/dev/null; echo $?
> 0
> 

Thank you for taking time testing it :). This example does work for me as well.

> BTW help2man seems to work fine on my Debian testing+unstable system with both LC_ALL=kk_KZ.RK1048 and LC_ALL=kk_KZ.XXXXXX and perl 5.26.1-5; I can't reproduce the behaviour you're describing.
> 

Doesn't work for me, please find attached a Dockerfile and its output (I've
stripped out the non-important parts). As you can see there, help2man, Perl,
and Python2 blow up. I didn't any further testing, but most of the programs
written on python2 or Perl will fail if they don't handle the locale user
encoding. I'm not sure it is a good idea that every program has to handle this,
I think it is an OS config and its responsibility as well. Does not seem a good
way to go repeating all the code on every program to run safely in a wrongly
configured OS.

> I'd suggest to go with the solution that I suggested, i.e. keep filing those bugs you're filing for buggy programs (the fix is to ignore unknown encodings parsed from LC_ALL) and:

I agree it is the best way to go as well. But choosing another well-supported
encoding by default would be better for reprotest I think.

Cheers,

-- 
TiN
-------------- next part --------------
FROM debian:unstable

RUN apt-get update && apt-get install build-essential locales-all -y

# it works
RUN LC_ALL=kk_KZ.XXXXXX gcc -c /dev/null 2>/dev/null || echo $?

# it works
RUN LC_ALL=kk_KZ.RK1048 gcc -c /dev/null 2>/dev/null || echo $?

RUN apt-get update && apt-get install python3 python perl help2man -y

# it works
RUN LC_ALL=kk_KZ.XXXXXX help2man 2> /dev/null || echo $?

# it doesn't work
RUN LC_ALL=kk_KZ.RK1048 help2man > /dev/null || echo $?

# it doesn't work
RUN perl -e 'use Encode; encode("kk_KZ.RK1048", "hello world\n");' || echo $?

# it works, fixed on python3.4 https://bugs.python.org/issue22682
RUN python3 -c "print('hello world'.encode('RK1048'))" || echo $?

# it doesn't work
RUN python -c "print('hello world'.encode('RK1048'))" || echo $?
-------------- next part --------------
Sending build context to Docker daemon  15.87kB

Step 1/10 : FROM debian:unstable
 ---> 864e51f464f4
Step 2/10 : RUN apt-get update && apt-get install build-essential locales-all -y
 ---> Running in 5fd75b0b6023
[...]
 ---> f7cd7d5f8026
Removing intermediate container 5fd75b0b6023
Step 3/10 : RUN LC_ALL=kk_KZ.XXXXXX gcc -c /dev/null 2>/dev/null || echo $?
 ---> Running in 041590cf42cd
 ---> ff06bc434394
Removing intermediate container 041590cf42cd
Step 4/10 : RUN LC_ALL=kk_KZ.RK1048 gcc -c /dev/null 2>/dev/null || echo $?
 ---> Running in 29c2a9a8d230
 ---> 352d471f5ab0
Removing intermediate container 29c2a9a8d230
Step 5/10 : RUN apt-get update && apt-get install python3 python perl help2man -y
 ---> Running in 16cb0c41dcd5
 [...]
 ---> 41d072b8c303
Removing intermediate container 16cb0c41dcd5
Step 6/10 : RUN LC_ALL=kk_KZ.XXXXXX help2man 2> /dev/null || echo $?
 ---> Running in 7b285f543d96
255
 ---> 905fb19e162c
Removing intermediate container 7b285f543d96
Step 7/10 : RUN LC_ALL=kk_KZ.RK1048 help2man > /dev/null || echo $?
 ---> Running in 9cccccd2013a
Unknown encoding 'RK1048' at /usr/bin/help2man line 56.
255
 ---> 5cf6baff897f
Removing intermediate container 9cccccd2013a
Step 8/10 : RUN perl -e 'use Encode; encode("kk_KZ.RK1048", "hello world\n");' || echo $?
 ---> Running in 98d09def9c43
Unknown encoding 'kk_KZ.RK1048' at -e line 1.
255
 ---> 36e1ad70065b
Removing intermediate container 98d09def9c43
Step 9/10 : RUN python3 -c "print('hello world'.encode('RK1048'))" || echo $?
 ---> Running in 3ed9742f3e95
b'hello world'
 ---> 975b830d1e47
Removing intermediate container 3ed9742f3e95
Step 10/10 : RUN python -c "print('hello world'.encode('RK1048'))" || echo $?
 ---> Running in eb157dfe2e9b
Traceback (most recent call last):
  File "<string>", line 1, in <module>
LookupError: unknown encoding: RK1048
1
 ---> cc5944fb3a62
Removing intermediate container eb157dfe2e9b
Successfully built cc5944fb3a62
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL: <http://alioth-lists.debian.net/pipermail/reproducible-builds/attachments/20180418/2a53f7ae/attachment.sig>