Mercurial > public > pelican-blog
comparison content/Coding/005-django-unicode-error-uploads.rst @ 4:7ce6393e6d30
Adding converted blog posts from old blog.
| author | Brian Neal <bgneal@gmail.com> |
|---|---|
| date | Thu, 30 Jan 2014 21:45:03 -0600 |
| parents | |
| children |
comparison
equal
deleted
inserted
replaced
| 3:c3115da3ff73 | 4:7ce6393e6d30 |
|---|---|
| 1 Django Uploads and UnicodeEncodeError | |
| 2 ##################################### | |
| 3 | |
| 4 :date: 2011-06-04 20:00 | |
| 5 :tags: Django, Python, Linux, Unicode | |
| 6 :slug: django-uploads-and-unicodeencodeerror | |
| 7 :author: Brian Neal | |
| 8 | |
| 9 Something strange happened that I wish to document in case it helps others. I | |
| 10 had to reboot my Ubuntu server while troubleshooting a disk problem. After the | |
| 11 reboot, I began receiving internal server errors whenever someone tried to view | |
| 12 a certain forum thread on my Django_ powered website. After some detective work, | |
| 13 I determined it was because a user that had posted in the thread had an avatar | |
| 14 image whose filename contained non-ASCII characters. The image file had been | |
| 15 there for months, and I still cannot explain why it just suddenly started | |
| 16 happening. | |
| 17 | |
| 18 The traceback I was getting ended with something like this: | |
| 19 | |
| 20 .. sourcecode:: python | |
| 21 | |
| 22 File "/django/core/files/storage.py", line 159, in _open | |
| 23 return File(open(self.path(name), mode)) | |
| 24 | |
| 25 UnicodeEncodeError: 'ascii' codec can't encode characters in position 72-79: ordinal not in range(128) | |
| 26 | |
| 27 So it appeared that the ``open()`` call was triggering the error. This led me on | |
| 28 a twisty Google search which had many dead ends. Eventually I found a suitable | |
| 29 explanation. Apparently, Linux filesystems don't enforce a particular Unicode | |
| 30 encoding for filenames. Linux applications must decide how to interpret | |
| 31 filenames all on their own. The Python OS library (on Linux) uses environment | |
| 32 variables to determine what locale you are in, and this chooses the encoding for | |
| 33 filenames. If these environment variables are not set, Python falls back to | |
| 34 ASCII (by default), and hence the source of my ``UnicodeEncodeError``. | |
| 35 | |
| 36 So how do you tell a Python instance that is running under Apache / ``mod_wsgi`` | |
| 37 about these environment variables? It turns out the answer is in the `Django | |
| 38 documentation`_, albeit in the ``mod_python`` integration section. | |
| 39 | |
| 40 So, to fix the issue, I added the following lines to my ``/etc/apache2/envvars`` | |
| 41 file: | |
| 42 | |
| 43 .. sourcecode:: bash | |
| 44 | |
| 45 export LANG='en_US.UTF-8' | |
| 46 export LC_ALL='en_US.UTF-8' | |
| 47 | |
| 48 Note that you must cold stop and re-start Apache for these changes to take | |
| 49 effect. I got tripped up at first because I did an ``apache2ctrl | |
| 50 graceful``, and that was not sufficient to create a new environment. | |
| 51 | |
| 52 .. _Django: http://djangoproject.com | |
| 53 .. _Django documentation: https://docs.djangoproject.com/en/1.3/howto/deployment/modpython/#if-you-get-a-unicodeencodeerror |