bgneal@4: A newline-to-break Python-Markdown extension
bgneal@4: ############################################
bgneal@4: 
bgneal@4: :date: 2011-05-09 22:40
bgneal@4: :tags: Markdown, Python
bgneal@4: :slug: a-newline-to-break-python-markdown-extension
bgneal@4: :author: Brian Neal
bgneal@4: 
bgneal@4: When I launched a new version of my website, I decided the new forums would use
bgneal@4: Markdown_ instead of BBCode_ for the markup. This decision was mainly a personal
bgneal@4: one for aesthetic reasons. I felt that Markdown was more natural to write compared
bgneal@4: to the clunky square brackets of BBCode.
bgneal@4: 
bgneal@4: My new site is coded in Python_ using the Django_ framework. For a Markdown implementation
bgneal@4: I chose `Python-Markdown`_.
bgneal@4: 
bgneal@4: My mainly non-technical users seemed largely ambivalent to the change from
bgneal@4: BBCode to Markdown. This was probably because I gave them a nice Javascript editor 
bgneal@4: (`MarkItUp!`_) which inserted the correct markup for them.
bgneal@4: 
bgneal@4: However, shortly after launch, one particular feature of Markdown really riled up
bgneal@4: some users: the default line break behavior. In strict Markdown, to create a new
bgneal@4: paragraph, you must insert a blank line between paragraphs. Hard returns (newlines)
bgneal@4: are simply ignored, just like they are in HTML. You can, however, force a break by 
bgneal@4: ending a line with two blank spaces. This isn't very intuitive, unlike the rest of
bgneal@4: Markdown.
bgneal@4: 
bgneal@4: Now I agree the default behavior is useful if you are creating an online document, like a blog post.
bgneal@4: However, non-technical users really didn't understand this behavior at all in the context
bgneal@4: of a forum post. For example, many of my users post radio-show playlists, formatted with 
bgneal@4: one song per line. When such a playlist was pasted into a forum post, Markdown made it 
bgneal@4: all one giant run-together paragraph. This did not please my users. Arguably, they should 
bgneal@4: have used a Markdown list. But it became clear teaching people the new syntax wasn't
bgneal@4: going to work, especially when it used to work just fine in BBCode and they had created
bgneal@4: their playlists in the same way for several years.
bgneal@4: 
bgneal@4: It turns out I am not alone in my observations (or on the receiving end of user wrath). Other,
bgneal@4: much larger sites, like StackOverflow_ and GitHub_, have altered their Markdown parsers
bgneal@4: to treat newlines as hard breaks. How can this be done with Python-Markdown?
bgneal@4: 
bgneal@4: It turns out this is really easy. Python-Markdown was designed with user customization
bgneal@4: in mind by offering an extension facility. The `extension documentation`_ is good, 
bgneal@4: and you can find extension writing help on the friendly `mailing list`_.
bgneal@4: 
bgneal@4: Here is a simple extension for Python-Markdown that turns newlines into HTML <br /> tags.
bgneal@4: 
bgneal@4: .. sourcecode:: python
bgneal@4: 
bgneal@4:    """
bgneal@4:    A python-markdown extension to treat newlines as hard breaks; like
bgneal@4:    StackOverflow and GitHub flavored Markdown do.
bgneal@4: 
bgneal@4:    """
bgneal@4:    import markdown
bgneal@4: 
bgneal@4: 
bgneal@4:    BR_RE = r'\n'
bgneal@4: 
bgneal@4:    class Nl2BrExtension(markdown.Extension):
bgneal@4: 
bgneal@4:        def extendMarkdown(self, md, md_globals):
bgneal@4:            br_tag = markdown.inlinepatterns.SubstituteTagPattern(BR_RE, 'br')
bgneal@4:            md.inlinePatterns.add('nl', br_tag, '_end')
bgneal@4: 
bgneal@4: 
bgneal@4:    def makeExtension(configs=None):
bgneal@4:        return Nl2BrExtension(configs)
bgneal@4: 
bgneal@4: I saved this code in a file called ``mdx_nl2br.py`` and put it on my ``PYTHONPATH``. You can then use 
bgneal@4: it in a Django template like this:
bgneal@4: 
bgneal@4: .. sourcecode:: django
bgneal@4: 
bgneal@4:    {{ value|markdown:"nl2br" }}
bgneal@4: 
bgneal@4: To use the extension in Python code, something like this should do the trick:
bgneal@4: 
bgneal@4: .. sourcecode:: python
bgneal@4: 
bgneal@4:    import markdown
bgneal@4:    md = markdown.Markdown(safe_mode=True, extensions=['nl2br'])
bgneal@4:    converted_text = md.convert(text)
bgneal@4: 
bgneal@4: **Update (June 21, 2011):** This extension is now being distributed with
bgneal@4: Python-Markdown! See `issue 13 on github`_ for the details. Thanks to Waylan
bgneal@4: Limberg for the help in creating the extension and for including it with
bgneal@4: Python-Markdown.
bgneal@4: 
bgneal@4: 
bgneal@4: .. _Markdown: http://daringfireball.net/projects/markdown/
bgneal@4: .. _BBCode: http://en.wikipedia.org/wiki/BBCode
bgneal@4: .. _Python: http://python.org
bgneal@4: .. _Django: http://djangoproject.com
bgneal@4: .. _MarkItUp!: http://markitup.jaysalvat.com/home/
bgneal@4: .. _StackOverflow: http://blog.stackoverflow.com/2009/10/markdown-one-year-later/
bgneal@4: .. _GitHub: http://github.github.com/github-flavored-markdown/
bgneal@4: .. _Python-Markdown: http://www.freewisdom.org/projects/python-markdown/
bgneal@4: .. _extension documentation: http://www.freewisdom.org/projects/python-markdown/Writing_Extensions
bgneal@4: .. _mailing list: http://lists.sourceforge.net/lists/listinfo/python-markdown-discuss
bgneal@4: .. _issue 13 on github: https://github.com/waylan/Python-Markdown/issues/13