[Python-apps-team] Bug#766316: Unable to reproduce

Mon Oct 27 21:27:44 UTC 2014

Running a en_US locale (see below), I have created a folder called
~/Téléchargements/ and symlinked a file in that folder. The subtitle is
downloaded correctly without the mentioned Exception.

This is the relevant extract when running subdownloader in debug mode:

$ subdownloader -d
[snip]
[22:08] DEBUG::subdownloader.gui.main # Downloading to:
u'/home/emilien/T\xe9l\xe9chargements/DDLValley.rocks_The.Rise.and.Rise.of.Bitcoin.2014.HDRip.XviD.AC3-EVO.srt'
[22:08] DEBUG::subdownloader.gui.main # Trying to download subtitle
'/home/emilien/Téléchargements/DDLValley.rocks_The.Rise.and.Rise.of.Bitcoin.2014.HDRip.XviD.AC3-EVO.srt'
[22:08] DEBUG::subdownloader.gui.main # Downloading subtitle
'/home/emilien/Téléchargements/DDLValley.rocks_The.Rise.and.Rise.of.Bitcoin.2014.HDRip.XviD.AC3-EVO.srt'
[22:08] DEBUG::subdownloader.httpsearch.OSHttpSearch # Downloading
subtitle from url:
http://www.opensubtitles.org/en/download/file/1954435660.gz
[22:08] DEBUG::subdownloader.httpsearch.OSHttpSearch # Unpacking and
savinng to:
/home/emilien/Téléchargements/DDLValley.rocks_The.Rise.and.Rise.of.Bitcoin.2014.HDRip.XviD.AC3-EVO.srt

Sylvestre, do you have any idea how to reproduce this bug?
Could you try to run the program with the English locale and that folder
with an accent in its name, and tell us if the same Traceback occurs?

$ LANG=en_US.UTF-8 subdownloader -d

See below for further tests I've tried.

Note that when running the program as previously mentioned, I do see the
é character nicely in the folder tree structure on the left side of the
user interface.

I have performed a new test, this time "forcing" the locale to French
(while not having it installed on my dev system):

emilien at debiansid:~/Téléchargements$ LANG=fr_FR.UTF-8 subdownloader -d
[22:15] DEBUG::subdownloader.gui.main # Building main dialog
[22:15] DEBUG::subdownloader.gui.main # Showing main dialog
[22:15] DEBUG::subdownloader.gui.main # Scanning translation files .mo
in folder: locale
[22:15] DEBUG::subdownloader.gui.main # Found these translations
languages: []
[22:15] DEBUG::subdownloader.gui.main # Interface language: en
[22:15] DEBUG::subdownloader.gui.main # Current directory:
/home/emilien/Tlchargements
[22:15] DEBUG::subdownloader.SDService.SDService # Creating Server with
server = None and proxy = None
[22:15] DEBUG::subdownloader.SDService.SDService # Creating XMLRPC
server connection... to server http://api.opensubtitles.org/xml-rpc with
proxy None
[22:15] DEBUG::subdownloader.SDService.SDService # Connecting with
parameters ('http://api.opensubtitles.org/xml-rpc', None)
[22:15] DEBUG::subdownloader.WebService # successfully tested connection
[22:15] DEBUG::subdownloader.SDService.SDService # Trying direct
connection...
[22:15] DEBUG::subdownloader.SDService.SDService # ...connected
[22:15] DEBUG::subdownloader.SDService.SDService # connection connected
<modules.SDService.TimeoutFunction instance at 0xaeef61ac>
[22:15] DEBUG::subdownloader.SDService.SDService # Creating Server with
server = None and proxy = None
[22:15] DEBUG::subdownloader.SDService.SDService # Creating XMLRPC
server connection... to server http://sddb.subdownloader.net/xmlrpc/
with proxy None
[22:15] DEBUG::subdownloader.SDService.SDService # Connecting with
parameters ('http://sddb.subdownloader.net/xmlrpc/', None)
[22:15] DEBUG::subdownloader.WebService # successfully tested connection
[22:15] DEBUG::subdownloader.SDService.SDService # Trying direct
connection...
[22:15] DEBUG::subdownloader.SDService.SDService # ...connected
[22:15] DEBUG::subdownloader.SDService.SDService # connection connected
<modules.SDService.TimeoutFunction instance at 0xaeefe72c>
[22:15] DEBUG::subdownloader.SDService.SDService # ----------------
[22:15] DEBUG::subdownloader.SDService.SDService # Logging in (username:
)...
[22:15] DEBUG::subdownloader.SDService.SDService # Login ended in 0.035
with status: 200 OK
[22:15] DEBUG::subdownloader.SDService.SDService # Session ID:
vlpf67sdhldsunvmumo1loqm26
[22:15] DEBUG::subdownloader.SDService.SDService # ----------------

The program starts correctly, connects to the server, but:
- The user interface is in English
- The folder that is supposed to have the accent is displayed as
"Tlchargement" in the folder structure.

When clicking on that folder, an exception occurs:

[22:17] DEBUG::subdownloader.FileManagement.FileScan # Scanning:
/home/emilien/Tlchargements
Traceback (most recent call last):
  File "/usr/share/pyshared/subdownloader/gui/main.py", line 944, in
onFolderTreeClicked
    self.SearchVideos(folder_path)
  File "/usr/share/pyshared/subdownloader/gui/main.py", line 781, in
SearchVideos
    videos_found,subs_found = FileScan.ScanFilesFolders(path,recursively
= True,report_progress = self.progress)
  File "/usr/share/pyshared/subdownloader/FileManagement/FileScan.py",
line 50, in ScanFilesFolders
    all_subs_found +=
ScanSubtitlesFolder(os.path.dirname(path),recursively =
False,report_progress=report_progress, progress_end=progress_end)
  File "/usr/share/pyshared/subdownloader/FileManagement/FileScan.py",
line 124, in ScanSubtitlesFolder
    if os.path.isfile(os.path.join(folderpath, filename)) and
get_extension(filename).lower() in subtitlefile.SUBTITLES_EXT:
  File "/usr/lib/python2.7/posixpath.py", line 80, in join
    path += '/' + b
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 2:
ordinal not in range(128)

Note that when using the -l option of subdownloader, this issue doesn't
occur:
$ subdownloader -d -l fr

The following bits (shown in both attempts to use the French language)
make me think that the .mo files are not being looked in the folder they
are installed to.
[22:15] DEBUG::subdownloader.gui.main # Scanning translation files .mo
in folder: locale
[22:15] DEBUG::subdownloader.gui.main # Found these translations
languages: []
[22:15] DEBUG::subdownloader.gui.main # Interface language: en

I am investigating that specific issue further.

FYI This is the output of the `locale` command on my dev box:

$ locale
LANG=en_US.UTF-8
LANGUAGE=en_US:en
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=

    +Emilien

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://lists.alioth.debian.org/pipermail/python-apps-team/attachments/20141027/a9511c42/attachment.sig>