Bug#547231: closed by Salvatore Bonaccorso <salvatore.bonaccorso at gmail.com> ([rt.cpan.org #59832] clobbers binmode layers on filehandles)

[Andrew Pimlott]

Salvatore, thanks for following up on this report.  I have some
questions below.

Excerpts from owner's message of Wed Aug 18 04:30:08 -0700 2010:
> This is an automatic notification regarding your Bug report
...
> Hi Andrew
> 
> I'm forwarding you the reply from upstream according to your report.
> 
> From: Hiroo_HAYASHI via RT <bug-Term-ReadLine-Gnu at rt.cpan.org>
> 
> > If I call
> > 
> >     Term::ReadLine->new('test', \*IN, \*OUT);
> > 
> > any encoding layers I have set on OUT seem to be removed.  I tracked
> 
> Term::Gnu::ReadLine "is an implementation of Term::ReadLine using the
> GNU Readline/History Library." as described in the document.
> The GNU Readline Library may call low level APIs directly.  So this is
> not a bug.

I realize that Readline has to do some low-level things.  But I don't
understand why it has to clobber the encoding layers on the filehandle.
Is it possible to preserve them?  Do you not agree that the test program
I sent is terribly confusing?  If Term::ReadLine::Gnu changes the
behavior of the filehandles like this, it should at least be documented.

> BTW if the reporter wants to handle multibyte (ex. UTF-8) characters,
> the GNU Readline Library support them and, as the result,
> Term::Gnu::ReadLine also support them. Refer the documents for the GNU
> Readline Library for details.

I couldn't find the documentation you are refering to, or how I'm
supposed to handle multibyte characters with Term::ReadLine::GNU in
Perl.  I notice that if I set the encoding layer back to UTF-8, it seems
to work.  In this test program, the first and third print statements
output the expected character, and only the second is broken:

    use Term::ReadLine;
    binmode(STDOUT, ':encoding(UTF-8)');
    print STDOUT ">", chr(0xf3), "<\n";
    $Term::ReadLine::Gnu::Attribs{outstream} = \*STDOUT;
    print STDOUT ">", chr(0xf3), "<\n";
    binmode(STDOUT, ':encoding(UTF-8)');
    print STDOUT ">", chr(0xf3), "<\n";

Will I have problems if I do it this way?  If this works, I don't see
why Term::ReadLine::GNU had to remove the encoding layer in the first
place.

> > it
> > down to the line in the perl source that sets $Attribs{outstream}.  I
> > looked briefly at the .xs source for _rl_store_iostream, but I can't
> > see
> > what's causing this.  I hope someone who knows perl internals can
> > figure
> > it out.
> 
> Here is a comment in Gnu.pm;
> ---------------------------------------
> # Each variable in the GNU Readline Library is tied to an entry of
> # this hash (%Attribs).  By accessing the hash entry, you can read
> # and/or write the variable in the GNU Readline Library.  See the
> # package definition of Term::ReadLine::Gnu::Var and following code
> # for more details.

I understand that part.  I traced the setting of $Attribs{outstream}
through the tie into _rl_store_iostream in Gnu.xs.  The code there looks
like

            rl_outstream = PerlIO_findFILE(stream);

Somehow this breaks my encoding layers, but I don't understand why.

Andrew