[Pkg-haskell-maintainers] Bug#748125: System.Posix.Directory.readDirStream can return strings that S.P.Files.getFileStatus cannot use

[Robert Bihlmeyer]

Package: ghc
Version: 7.4.1-4
Severity: normal

If the LANG/LC_ environment specifies an UTF-8 locale the following
program getFileStatus will sometimes fail with ENOENT. In my tests this
is the case for some (not all) filenames containing invalid UTF-8

Attached is a simple tar containing files with an all-ASCII name, an
UTF-8 name (both work) and a GB2312 name exhibiting the problem.

I don't think running a program with LC_CTYPE=*.UTF-8 means that all
filenames that it encounters have to be valid UTF-8.

  import System.Posix.Directory
  import System.Posix.Files

  isNull = null
  makeString x = x
  concat' = concat
  putStrLn' = putStrLn

  scanDir :: DirStream -> IO ()
  scanDir dir = do entry <- readDirStream dir
                   if isNull entry
                     then return ()
                     else do stat <- getFileStatus entry
                             putStrLn' $ 
                               concat' [makeString (if isDirectory stat
                                                    then "dir: "
                                                    else "file: "), entry]
                             scanDir dir

  main = do changeWorkingDirectory $ makeString "/tmp/dirtest"
            dir <- openDirStream $ makeString "."
            scanDir dir
            closeDirStream dir

Changing the program to use the ByteString variants (example below)
works as no conversion is ever done.

  import qualified Data.Char
  import qualified Data.ByteString
  import qualified Data.ByteString.Char8
  import System.Posix.Directory.ByteString
  import System.Posix.Files.ByteString

  isNull = Data.ByteString.null
  makeString x = Data.ByteString.pack $ map (toEnum . fromEnum) x
  concat' = Data.ByteString.concat
  putStrLn' = Data.ByteString.Char8.putStrLn

  scanDir :: DirStream -> IO ()
  scanDir dir = do entry <- readDirStream dir
                   if isNull entry
                     then return ()
                     else do stat <- getFileStatus entry
                             putStrLn' $ 
                               concat' [makeString (if isDirectory stat
                                                    then "dir: "
                                                    else "file: "), entry]
                             scanDir dir

  main = do changeWorkingDirectory $ makeString "/tmp/dirtest"
            dir <- openDirStream $ makeString "."
            scanDir dir
            closeDirStream dir

-------------- next part --------------
A non-text attachment was scrubbed...
Name: dirtest.tar
Type: application/x-tar
Size: 10240 bytes
Desc: not available
URL: <http://lists.alioth.debian.org/pipermail/pkg-haskell-maintainers/attachments/20140514/598103de/attachment.tar>
-------------- next part --------------

Regards,
-- 
Robert Bihlmeyer    ASSIST    Arrow ECS Internet Security AG