A newline-to-break Python-Markdown extension
############################################

:date: 2011-05-09 22:40
:tags: Markdown, Python
:slug: a-newline-to-break-python-markdown-extension
:author: Brian Neal

When I launched a new version of my website, I decided the new forums would use
Markdown_ instead of BBCode_ for the markup. This decision was mainly a personal
one for aesthetic reasons. I felt that Markdown was more natural to write compared
to the clunky square brackets of BBCode.

My new site is coded in Python_ using the Django_ framework. For a Markdown implementation
I chose `Python-Markdown`_.

My mainly non-technical users seemed largely ambivalent to the change from
BBCode to Markdown. This was probably because I gave them a nice Javascript editor 
(`MarkItUp!`_) which inserted the correct markup for them.

However, shortly after launch, one particular feature of Markdown really riled up
some users: the default line break behavior. In strict Markdown, to create a new
paragraph, you must insert a blank line between paragraphs. Hard returns (newlines)
are simply ignored, just like they are in HTML. You can, however, force a break by 
ending a line with two blank spaces. This isn't very intuitive, unlike the rest of
Markdown.

Now I agree the default behavior is useful if you are creating an online document, like a blog post.
However, non-technical users really didn't understand this behavior at all in the context
of a forum post. For example, many of my users post radio-show playlists, formatted with 
one song per line. When such a playlist was pasted into a forum post, Markdown made it 
all one giant run-together paragraph. This did not please my users. Arguably, they should 
have used a Markdown list. But it became clear teaching people the new syntax wasn't
going to work, especially when it used to work just fine in BBCode and they had created
their playlists in the same way for several years.

It turns out I am not alone in my observations (or on the receiving end of user wrath). Other,
much larger sites, like StackOverflow_ and GitHub_, have altered their Markdown parsers
to treat newlines as hard breaks. How can this be done with Python-Markdown?

It turns out this is really easy. Python-Markdown was designed with user customization
in mind by offering an extension facility. The `extension documentation`_ is good, 
and you can find extension writing help on the friendly `mailing list`_.

Here is a simple extension for Python-Markdown that turns newlines into HTML <br /> tags.

.. sourcecode:: python

   """
   A python-markdown extension to treat newlines as hard breaks; like
   StackOverflow and GitHub flavored Markdown do.

   """
   import markdown


   BR_RE = r'\n'

   class Nl2BrExtension(markdown.Extension):

       def extendMarkdown(self, md, md_globals):
           br_tag = markdown.inlinepatterns.SubstituteTagPattern(BR_RE, 'br')
           md.inlinePatterns.add('nl', br_tag, '_end')


   def makeExtension(configs=None):
       return Nl2BrExtension(configs)

I saved this code in a file called ``mdx_nl2br.py`` and put it on my ``PYTHONPATH``. You can then use 
it in a Django template like this:

.. sourcecode:: django

   {{ value|markdown:"nl2br" }}

To use the extension in Python code, something like this should do the trick:

.. sourcecode:: python

   import markdown
   md = markdown.Markdown(safe_mode=True, extensions=['nl2br'])
   converted_text = md.convert(text)

**Update (June 21, 2011):** This extension is now being distributed with
Python-Markdown! See `issue 13 on github`_ for the details. Thanks to Waylan
Limberg for the help in creating the extension and for including it with
Python-Markdown.


.. _Markdown: http://daringfireball.net/projects/markdown/
.. _BBCode: http://en.wikipedia.org/wiki/BBCode
.. _Python: http://python.org
.. _Django: http://djangoproject.com
.. _MarkItUp!: http://markitup.jaysalvat.com/home/
.. _StackOverflow: http://blog.stackoverflow.com/2009/10/markdown-one-year-later/
.. _GitHub: http://github.github.com/github-flavored-markdown/
.. _Python-Markdown: http://www.freewisdom.org/projects/python-markdown/
.. _extension documentation: http://www.freewisdom.org/projects/python-markdown/Writing_Extensions
.. _mailing list: http://lists.sourceforge.net/lists/listinfo/python-markdown-discuss
.. _issue 13 on github: https://github.com/waylan/Python-Markdown/issues/13