--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/content/Coding/020-django-moinmoin.rst	Thu Jan 30 21:45:03 2014 -0600
@@ -0,0 +1,443 @@
+Integrating Django and MoinMoin with Redis
+##########################################
+
+:date: 2012-12-02 14:50
+:tags: Django, MoinMoin, Redis
+:slug: integrating-django-and-moinmoin-with-redis
+:author: Brian Neal
+
+We want a Wiki!
+===============
+
+Over at SurfGuitar101.com_, we decided we'd like to have a wiki to capture
+community knowledge. I briefly looked at candidate wiki engines with an eye
+towards integrating them with Django_, the framework that powers
+SurfGuitar101.com_. And of course I was biased towards a wiki solution that was
+written in Python_. I had tried a few wikis in the past, including the behemoth
+MediaWiki_. MediaWiki is a very powerful piece of software, but it is also
+quite complex, and I didn't want to have to maintain a PHP infrastructure to run
+it.
+
+Enter MoinMoin_. This is a mature wiki platform that is actively maintained and
+written in Python. It is full featured but did not seem overly complex to me.
+It stores its pages in flat files, which seemed appealing for our likely small
+wiki needs. It turns out I had been a user of MoinMoin for many years without
+really knowing it. The `Python.org wiki`_, `Mercurial wiki`_, and `omniORB
+wiki`_ are all powered by MoinMoin_. We'd certainly be in good company.
+
+Single Sign-On
+==============
+
+The feature that clinched it was MoinMoin's flexible `authentication system`_.
+It would be very desirable if my users did not have to sign into Django and
+then sign in again to the wiki with possibly a different username. Managing two
+different password databases would be a real headache. The ideal solution would
+mean signing into Django would log the user into MoinMoin with the same
+username automatically.
+
+MoinMoin supports this with their `external cookie authentication`_ mechanism.
+The details are provided in the previous link; basically a Django site needs to
+perform the following:
+
+#. Set an external cookie for MoinMoin to read whenever a user logs into Django.
+#. To prevent spoofing, the Django application should create a record that the
+   cookie was created in some shared storage area accessible to MoinMoin. This
+   allows MoinMoin to validate that the cookie is legitimate and not a fake.
+#. When the user logs out of the Django application, it should delete this
+   external cookie and the entry in the shared storage area.
+#. Periodically the Django application should expire entires in the shared
+   storage area for clean-up purposes. Otherwise this storage would grow and
+   grow if users never logged out. Deleting entries older than the external
+   cookie's age should suffice.
+
+My Django Implementation
+========================
+
+There are of course many ways to approach this problem. Here is what I came up
+with. I created a Django application called *wiki* to hold this integration
+code. There is quite a lot of code here, too much to conveniently show in this
+blog post. I will post snippets below, but you can refer to the complete code
+in my `Bitbucket repository`_. You can also view online the `wiki application in
+bitbucket`_ for convenience.
+
+Getting notified of when users log into or out of Django is made easy thanks to
+Django's `login and logout signals`_. By creating a signal handler I can be
+notified when a user logs in or out. The signal handler code looks like this:
+
+.. sourcecode:: python
+
+   import logging
+   from django.contrib.auth.signals import user_logged_in, user_logged_out
+   from wiki.constants import SESSION_SET_MEMBER
+
+   logger = logging.getLogger(__name__)
+
+   def login_callback(sender, request, user, **kwargs):
+       """Signal callback function for a user logging in.
+
+       Sets a flag for the middleware to create an external cookie.
+
+       """
+       logger.info('User login: %s', user.username)
+
+       request.wiki_set_cookie = True
+
+   def logout_callback(sender, request, user, **kwargs):
+       """Signal callback function for a user logging in.
+
+       Sets a flag for the middleware to delete the external cookie.
+
+       Since the user is about to logout, her session will be wiped out after
+       this function returns. This forces us to set an attribute on the request
+       object so that the response middleware can delete the wiki's cookie.
+
+       """
+       if user:
+           logger.info('User logout: %s', user.username)
+
+           # Remember what Redis set member to delete by adding an attribute to the
+           # request object:
+           request.wiki_delete_cookie = request.session.get(SESSION_SET_MEMBER)
+
+
+   user_logged_in.connect(login_callback, dispatch_uid='wiki.signals.login')
+   user_logged_out.connect(logout_callback, dispatch_uid='wiki.signals.logout')
+
+When a user logs in I want to create an external cookie for MoinMoin. But
+cookies can only be created on HttpResponse_ objects, and all we have access to
+here in the signal handler is the request object. The solution here is to set
+an attribute on the request object that a later piece of middleware_ will
+process. I at first resisted this approach, thinking it was kind of hacky.
+I initially decided to set a flag in the session, but then found out that in
+some cases the session is not always available. I then reviewed some of the
+Django supplied middleware classes and saw that they also set attributes on the
+request object, so this must be an acceptable practice.
+
+My middleware looks like this.
+
+.. sourcecode:: python
+
+   class WikiMiddleware(object):
+       """
+       Check for flags on the request object to determine when to set or delete an
+       external cookie for the wiki application. When creating a cookie, also
+       set an entry in Redis that the wiki application can validate to prevent
+       spoofing.
+
+       """
+
+       def process_response(self, request, response):
+
+           if hasattr(request, 'wiki_set_cookie'):
+               create_wiki_session(request, response)
+           elif hasattr(request, 'wiki_delete_cookie'):
+               destroy_wiki_session(request.wiki_delete_cookie, response)
+
+           return response
+
+The ``create_wiki_session()`` function creates the cookie for MoinMoin and
+stores a hash of the cookie in a shared storage area for MoinMoin to validate.
+In our case, Redis_ makes an excellent shared storage area. We create a sorted
+set in Redis to store our cookie hashes. The score for each hash is the
+timestamp of when the cookie was created. This allows us to easily delete
+expired cookies by score periodically.
+
+.. sourcecode:: python
+
+   def create_wiki_session(request, response):
+       """Sets up the session for the external wiki application.
+
+       Creates the external cookie for the Wiki.
+       Updates the Redis set so the Wiki can verify the cookie.
+
+       """
+       now = datetime.datetime.utcnow()
+       value = cookie_value(request.user, now)
+       response.set_cookie(settings.WIKI_COOKIE_NAME,
+               value=value,
+               max_age=settings.WIKI_COOKIE_AGE,
+               domain=settings.WIKI_COOKIE_DOMAIN)
+
+       # Update a sorted set in Redis with a hash of our cookie and a score
+       # of the current time as a timestamp. This allows us to delete old
+       # entries by score periodically. To verify the cookie, the external wiki
+       # application computes a hash of the cookie value and checks to see if
+       # it is in our Redis set.
+
+       h = hashlib.sha256()
+       h.update(value)
+       name = h.hexdigest()
+       score = time.mktime(now.utctimetuple())
+       conn = get_redis_connection()
+
+       try:
+           conn.zadd(settings.WIKI_REDIS_SET, score, name)
+       except redis.RedisError:
+           logger.error("Error adding wiki cookie key")
+
+       # Store the set member name in the session so we can delete it when the
+       # user logs out:
+       request.session[SESSION_SET_MEMBER] = name
+
+We store the name of the Redis set member in the user's session so we can
+delete it from Redis when the user logs out. During logout, this set member is
+retrieved from the session in the logout signal handler and stored on the
+request object. This is because the session will be destroyed after the logout
+signal handler runs and before the middleware can access it. The middleware
+can check for the existence of this attribute as its cue to delete the wiki
+session.
+
+.. sourcecode:: python
+
+   def destroy_wiki_session(set_member, response):
+       """Destroys the session for the external wiki application.
+
+       Delete the external cookie.
+       Deletes the member from the Redis set as this entry is no longer valid.
+
+       """
+       response.delete_cookie(settings.WIKI_COOKIE_NAME,
+                              domain=settings.WIKI_COOKIE_DOMAIN)
+
+       if set_member:
+           conn = get_redis_connection()
+           try:
+               conn.zrem(settings.WIKI_REDIS_SET, set_member)
+           except redis.RedisError:
+               logger.error("Error deleting wiki cookie set member")
+
+As suggested in the MoinMoin external cookie documentation, I create a cookie
+whose value consists of the username, email address, and a key separated by
+the ``#`` character. The key is just a string of stuff that makes it difficult
+for a spoofer to recreate.
+
+.. sourcecode:: python
+
+   def cookie_value(user, now):
+       """Creates the value for the external wiki cookie."""
+
+       # The key part of the cookie is just a string that would make things
+       # difficult for a spoofer; something that can't be easily made up:
+
+       h = hashlib.sha256()
+       h.update(user.username + user.email)
+       h.update(now.isoformat())
+       h.update(''.join(random.sample(string.printable, 64)))
+       h.update(settings.SECRET_KEY)
+       key = h.hexdigest()
+
+       parts = (user.username, user.email, key)
+       return '#'.join(parts)
+
+Finally on the Django side we should periodically delete expired Redis set
+members in case users do not log out. Since I am using Celery_ with my Django
+application, I created a Celery task that runs periodically to delete old set
+members. This function is a bit longer than it probably needs to be, but
+I wanted to log how big this set is before and after we cull the expired
+entries.
+
+.. sourcecode:: python
+
+   @task
+   def expire_cookies():
+       """
+       Periodically run this task to remove expired cookies from the Redis set
+       that is shared between this Django application & the MoinMoin wiki for
+       authentication.
+
+       """
+       now = datetime.datetime.utcnow()
+       cutoff = now - datetime.timedelta(seconds=settings.WIKI_COOKIE_AGE)
+       min_score = time.mktime(cutoff.utctimetuple())
+
+       conn = get_redis_connection()
+
+       set_name = settings.WIKI_REDIS_SET
+       try:
+           count = conn.zcard(set_name)
+       except redis.RedisError:
+           logger.error("Error getting zcard")
+           return
+
+       try:
+           removed = conn.zremrangebyscore(set_name, 0.0, min_score)
+       except redis.RedisError:
+           logger.error("Error removing by score")
+           return
+
+       total = count - removed
+       logger.info("Expire wiki cookies: removed %d, total is now %d",
+               removed, total)
+
+MoinMoin Implementation
+=======================
+
+As described in the MoinMoin external cookie documentation, you have to
+configure MoinMoin to use your external cookie authentication mechanism.
+It is also nice to disable the ability for the MoinMoin user to change their
+username and email address since that is being managed by the Django
+application. These changes to the MoinMoin ``Config`` class are shown below.
+
+.. sourcecode:: python
+
+   class Config(multiconfig.DefaultConfig):
+
+       # ...
+
+       # Use ExternalCookie method for integration authentication with Django:
+       auth = [ExternalCookie(autocreate=True)]
+
+       # remove ability to change username & email, etc.
+       user_form_disable = ['name', 'aliasname', 'email',]
+       user_form_remove = ['password', 'password2', 'css_url', 'logout', 'create',
+               'account_sendmail', 'jid']
+
+Next we create an ``ExternalCookie`` class and associated helper functions to
+process the cookie and verify it in Redis. This code is shown in its entirety
+below. It is based off the example in the MoinMoin external cookie
+documentation, but uses Redis as the shared storage area.
+
+.. sourcecode:: python
+
+   import hashlib
+   import Cookie
+   import logging
+
+   from MoinMoin.auth import BaseAuth
+   from MoinMoin.user import User
+   import redis
+
+   COOKIE_NAME = 'YOUR_COOKIE_NAME_HERE'
+
+   # Redis connection and database settings
+   REDIS_HOST = 'localhost'
+   REDIS_PORT = 6379
+   REDIS_DB = 0
+
+   # The name of the set in Redis that holds cookie hashes
+   REDIS_SET = 'wiki_cookie_keys'
+
+   logger = logging.getLogger(__name__)
+
+
+   def get_cookie_value(cookie):
+       """Returns the value of the Django cookie from the cookie.
+       None is returned if the cookie is invalid or the value cannot be
+       determined.
+
+       This function works around an issue with different Python versions.
+       In Python 2.5, if you construct a SimpleCookie with a dict, then
+           type(cookie[key]) == unicode
+       whereas in later versions of Python:
+           type(cookie[key]) == Cookie.Morsel
+       """
+       if cookie:
+           try:
+               morsel = cookie[COOKIE_NAME]
+           except KeyError:
+               return None
+
+           if isinstance(morsel, unicode):             # Python 2.5
+               return morsel
+           elif isinstance(morsel, Cookie.Morsel):     # Python 2.6+
+               return morsel.value
+
+       return None
+
+
+   def get_redis_connection(host=REDIS_HOST, port=REDIS_PORT, db=REDIS_DB):
+       """
+       Create and return a Redis connection using the supplied parameters.
+
+       """
+       return redis.StrictRedis(host=host, port=port, db=db)
+
+
+   def validate_cookie(value):
+       """Determines if cookie was created by Django. Returns True on success,
+       False on failure.
+
+       Looks up the hash of the cookie value in Redis. If present, cookie
+       is deemed legit.
+
+       """
+       h = hashlib.sha256()
+       h.update(value)
+       set_member = h.hexdigest()
+
+       conn = get_redis_connection()
+       success = False
+       try:
+           score = conn.zscore(REDIS_SET, set_member)
+           success = score is not None
+       except redis.RedisError:
+           logger.error('Could not check Redis for ExternalCookie auth')
+
+       return success
+
+
+   class ExternalCookie(BaseAuth):
+       name = 'external_cookie'
+
+       def __init__(self, autocreate=False):
+           self.autocreate = autocreate
+           BaseAuth.__init__(self)
+
+       def request(self, request, user_obj, **kwargs):
+           user = None
+           try_next = True
+
+           try:
+               cookie = Cookie.SimpleCookie(request.cookies)
+           except Cookie.CookieError:
+               cookie = None
+
+           val = get_cookie_value(cookie)
+           if val:
+               try:
+                   username, email, _ = val.split('#')
+               except ValueError:
+                   return user, try_next
+
+               if validate_cookie(val):
+                   user = User(request, name=username, auth_username=username,
+                           auth_method=self.name)
+
+                   changed = False
+                   if email != user.email:
+                       user.email = email
+                       changed = True
+
+                   if user:
+                       user.create_or_update(changed)
+                   if user and user.valid:
+                       try_next = False
+
+           return user, try_next
+
+Conclusion
+==========
+
+I've been running this setup for a month now and it is working great. My users
+and I are enjoying our shiny new MoinMoin wiki integrated with our Django
+powered community website. The single sign-on experience is quite seamless and
+eliminates the need for separate accounts.
+
+
+.. _SurfGuitar101.com: http://surfguitar101.com
+.. _Django: https://www.djangoproject.com
+.. _MediaWiki: http://www.mediawiki.org
+.. _MoinMoin: http://moinmo.in/
+.. _Python: http://www.python.org
+.. _Python.org wiki: http://wiki.python.org/moin/
+.. _Mercurial wiki: http://mercurial.selenic.com/wiki/
+.. _omniORB wiki: http://www.omniorb-support.com/omniwiki
+.. _authentication system: http://moinmo.in/HelpOnAuthentication
+.. _external cookie authentication: http://moinmo.in/HelpOnAuthentication/ExternalCookie
+.. _login and logout signals: https://docs.djangoproject.com/en/1.4/topics/auth/#login-and-logout-signals
+.. _HttpResponse: https://docs.djangoproject.com/en/1.4/ref/request-response/#httpresponse-objects
+.. _middleware: https://docs.djangoproject.com/en/1.4/topics/http/middleware/
+.. _Redis: http://redis.io/
+.. _Bitbucket repository: https://bitbucket.org/bgneal/sg101
+.. _wiki application in bitbucket: https://bitbucket.org/bgneal/sg101/src/a5b8f25e1752faf71ed429ec7f22ff6f3b3dc851/wiki?at=default
+.. _Celery: http://celeryproject.org/