pelican-blog: content/Coding/020-django-moinmoin.rst comparison

comparison content/Coding/020-django-moinmoin.rst @ 4:7ce6393e6d30

Adding converted blog posts from old blog.

author	Brian Neal <bgneal@gmail.com>
date	Thu, 30 Jan 2014 21:45:03 -0600
parents
children	49bebfa6f9d3

comparison

equal deleted inserted replaced

-:c3115da3ff73
+:7ce6393e6d30
+Integrating Django and MoinMoin with Redis
+##########################################
+:date: 2012-12-02 14:50
+:tags: Django, MoinMoin, Redis
+:slug: integrating-django-and-moinmoin-with-redis
+:author: Brian Neal
+We want a Wiki!
+===============
+Over at SurfGuitar101.com_, we decided we'd like to have a wiki to capture
+community knowledge. I briefly looked at candidate wiki engines with an eye
+towards integrating them with Django_, the framework that powers
+SurfGuitar101.com_. And of course I was biased towards a wiki solution that was
+written in Python_. I had tried a few wikis in the past, including the behemoth
+MediaWiki_. MediaWiki is a very powerful piece of software, but it is also
+quite complex, and I didn't want to have to maintain a PHP infrastructure to run
+it.
+Enter MoinMoin_. This is a mature wiki platform that is actively maintained and
+written in Python. It is full featured but did not seem overly complex to me.
+It stores its pages in flat files, which seemed appealing for our likely small
+wiki needs. It turns out I had been a user of MoinMoin for many years without
+really knowing it. The `Python.org wiki`_, `Mercurial wiki`_, and `omniORB
+wiki`_ are all powered by MoinMoin_. We'd certainly be in good company.
+Single Sign-On
+==============
+The feature that clinched it was MoinMoin's flexible `authentication system`_.
+It would be very desirable if my users did not have to sign into Django and
+then sign in again to the wiki with possibly a different username. Managing two
+different password databases would be a real headache. The ideal solution would
+mean signing into Django would log the user into MoinMoin with the same
+username automatically.
+MoinMoin supports this with their `external cookie authentication`_ mechanism.
+The details are provided in the previous link; basically a Django site needs to
+perform the following:
+#. Set an external cookie for MoinMoin to read whenever a user logs into Django.
+#. To prevent spoofing, the Django application should create a record that the
+cookie was created in some shared storage area accessible to MoinMoin. This
+allows MoinMoin to validate that the cookie is legitimate and not a fake.
+#. When the user logs out of the Django application, it should delete this
+external cookie and the entry in the shared storage area.
+#. Periodically the Django application should expire entires in the shared
+storage area for clean-up purposes. Otherwise this storage would grow and
+grow if users never logged out. Deleting entries older than the external
+cookie's age should suffice.
+My Django Implementation
+========================
+There are of course many ways to approach this problem. Here is what I came up
+with. I created a Django application called *wiki* to hold this integration
+code. There is quite a lot of code here, too much to conveniently show in this
+blog post. I will post snippets below, but you can refer to the complete code
+in my `Bitbucket repository`_. You can also view online the `wiki application in
+bitbucket`_ for convenience.
+Getting notified of when users log into or out of Django is made easy thanks to
+Django's `login and logout signals`_. By creating a signal handler I can be
+notified when a user logs in or out. The signal handler code looks like this:
+.. sourcecode:: python
+import logging
+from django.contrib.auth.signals import user_logged_in, user_logged_out
+from wiki.constants import SESSION_SET_MEMBER
+logger = logging.getLogger(__name__)
+def login_callback(sender, request, user, **kwargs):
+"""Signal callback function for a user logging in.
+Sets a flag for the middleware to create an external cookie.
+"""
+logger.info('User login: %s', user.username)
+request.wiki_set_cookie = True
+def logout_callback(sender, request, user, **kwargs):
+"""Signal callback function for a user logging in.
+Sets a flag for the middleware to delete the external cookie.
+Since the user is about to logout, her session will be wiped out after
+this function returns. This forces us to set an attribute on the request
+object so that the response middleware can delete the wiki's cookie.
+"""
+if user:
+logger.info('User logout: %s', user.username)
+# Remember what Redis set member to delete by adding an attribute to the
+# request object:
+request.wiki_delete_cookie = request.session.get(SESSION_SET_MEMBER)
+user_logged_in.connect(login_callback, dispatch_uid='wiki.signals.login')
+user_logged_out.connect(logout_callback, dispatch_uid='wiki.signals.logout')
+When a user logs in I want to create an external cookie for MoinMoin. But
+cookies can only be created on HttpResponse_ objects, and all we have access to
+here in the signal handler is the request object. The solution here is to set
+an attribute on the request object that a later piece of middleware_ will
+process. I at first resisted this approach, thinking it was kind of hacky.
+I initially decided to set a flag in the session, but then found out that in
+some cases the session is not always available. I then reviewed some of the
+Django supplied middleware classes and saw that they also set attributes on the
+request object, so this must be an acceptable practice.
+My middleware looks like this.
+.. sourcecode:: python
+class WikiMiddleware(object):
+"""
+Check for flags on the request object to determine when to set or delete an
+external cookie for the wiki application. When creating a cookie, also
+set an entry in Redis that the wiki application can validate to prevent
+spoofing.
+"""
+def process_response(self, request, response):
+if hasattr(request, 'wiki_set_cookie'):
+create_wiki_session(request, response)
+elif hasattr(request, 'wiki_delete_cookie'):
+destroy_wiki_session(request.wiki_delete_cookie, response)
+return response
+The ``create_wiki_session()`` function creates the cookie for MoinMoin and
+stores a hash of the cookie in a shared storage area for MoinMoin to validate.
+In our case, Redis_ makes an excellent shared storage area. We create a sorted
+set in Redis to store our cookie hashes. The score for each hash is the
+timestamp of when the cookie was created. This allows us to easily delete
+expired cookies by score periodically.
+.. sourcecode:: python
+def create_wiki_session(request, response):
+"""Sets up the session for the external wiki application.
+Creates the external cookie for the Wiki.
+Updates the Redis set so the Wiki can verify the cookie.
+"""
+now = datetime.datetime.utcnow()
+value = cookie_value(request.user, now)
+response.set_cookie(settings.WIKI_COOKIE_NAME,
+value=value,
+max_age=settings.WIKI_COOKIE_AGE,
+domain=settings.WIKI_COOKIE_DOMAIN)
+# Update a sorted set in Redis with a hash of our cookie and a score
+# of the current time as a timestamp. This allows us to delete old
+# entries by score periodically. To verify the cookie, the external wiki
+# application computes a hash of the cookie value and checks to see if
+# it is in our Redis set.
+h = hashlib.sha256()
+h.update(value)
+name = h.hexdigest()
+score = time.mktime(now.utctimetuple())
+conn = get_redis_connection()
+try:
+conn.zadd(settings.WIKI_REDIS_SET, score, name)
+except redis.RedisError:
+logger.error("Error adding wiki cookie key")
+# Store the set member name in the session so we can delete it when the
+# user logs out:
+request.session[SESSION_SET_MEMBER] = name
+We store the name of the Redis set member in the user's session so we can
+delete it from Redis when the user logs out. During logout, this set member is
+retrieved from the session in the logout signal handler and stored on the
+request object. This is because the session will be destroyed after the logout
+signal handler runs and before the middleware can access it. The middleware
+can check for the existence of this attribute as its cue to delete the wiki
+session.
+.. sourcecode:: python
+def destroy_wiki_session(set_member, response):
+"""Destroys the session for the external wiki application.
+Delete the external cookie.
+Deletes the member from the Redis set as this entry is no longer valid.
+"""
+response.delete_cookie(settings.WIKI_COOKIE_NAME,
+domain=settings.WIKI_COOKIE_DOMAIN)
+if set_member:
+conn = get_redis_connection()
+try:
+conn.zrem(settings.WIKI_REDIS_SET, set_member)
+except redis.RedisError:
+logger.error("Error deleting wiki cookie set member")
+As suggested in the MoinMoin external cookie documentation, I create a cookie
+whose value consists of the username, email address, and a key separated by
+the ``#`` character. The key is just a string of stuff that makes it difficult
+for a spoofer to recreate.
+.. sourcecode:: python
+def cookie_value(user, now):
+"""Creates the value for the external wiki cookie."""
+# The key part of the cookie is just a string that would make things
+# difficult for a spoofer; something that can't be easily made up:
+h = hashlib.sha256()
+h.update(user.username + user.email)
+h.update(now.isoformat())
+h.update(''.join(random.sample(string.printable, 64)))
+h.update(settings.SECRET_KEY)
+key = h.hexdigest()
+parts = (user.username, user.email, key)
+return '#'.join(parts)
+Finally on the Django side we should periodically delete expired Redis set
+members in case users do not log out. Since I am using Celery_ with my Django
+application, I created a Celery task that runs periodically to delete old set
+members. This function is a bit longer than it probably needs to be, but
+I wanted to log how big this set is before and after we cull the expired
+entries.
+.. sourcecode:: python
+@task
+def expire_cookies():
+"""
+Periodically run this task to remove expired cookies from the Redis set
+that is shared between this Django application & the MoinMoin wiki for
+authentication.
+"""
+now = datetime.datetime.utcnow()
+cutoff = now - datetime.timedelta(seconds=settings.WIKI_COOKIE_AGE)
+min_score = time.mktime(cutoff.utctimetuple())
+conn = get_redis_connection()
+set_name = settings.WIKI_REDIS_SET
+try:
+count = conn.zcard(set_name)
+except redis.RedisError:
+logger.error("Error getting zcard")
+return
+try:
+removed = conn.zremrangebyscore(set_name, 0.0, min_score)
+except redis.RedisError:
+logger.error("Error removing by score")
+return
+total = count - removed
+logger.info("Expire wiki cookies: removed %d, total is now %d",
+removed, total)
+MoinMoin Implementation
+=======================
+As described in the MoinMoin external cookie documentation, you have to
+configure MoinMoin to use your external cookie authentication mechanism.
+It is also nice to disable the ability for the MoinMoin user to change their
+username and email address since that is being managed by the Django
+application. These changes to the MoinMoin ``Config`` class are shown below.
+.. sourcecode:: python
+class Config(multiconfig.DefaultConfig):
+# ...
+# Use ExternalCookie method for integration authentication with Django:
+auth = [ExternalCookie(autocreate=True)]
+# remove ability to change username & email, etc.
+user_form_disable = ['name', 'aliasname', 'email',]
+user_form_remove = ['password', 'password2', 'css_url', 'logout', 'create',
+'account_sendmail', 'jid']
+Next we create an ``ExternalCookie`` class and associated helper functions to
+process the cookie and verify it in Redis. This code is shown in its entirety
+below. It is based off the example in the MoinMoin external cookie
+documentation, but uses Redis as the shared storage area.
+.. sourcecode:: python
+import hashlib
+import Cookie
+import logging
+from MoinMoin.auth import BaseAuth
+from MoinMoin.user import User
+import redis
+COOKIE_NAME = 'YOUR_COOKIE_NAME_HERE'
+# Redis connection and database settings
+REDIS_HOST = 'localhost'
+REDIS_PORT = 6379
+REDIS_DB = 0
+# The name of the set in Redis that holds cookie hashes
+REDIS_SET = 'wiki_cookie_keys'
+logger = logging.getLogger(__name__)
+def get_cookie_value(cookie):
+"""Returns the value of the Django cookie from the cookie.
+None is returned if the cookie is invalid or the value cannot be
+determined.
+This function works around an issue with different Python versions.
+In Python 2.5, if you construct a SimpleCookie with a dict, then
+type(cookie[key]) == unicode
+whereas in later versions of Python:
+type(cookie[key]) == Cookie.Morsel
+"""
+if cookie:
+try:
+morsel = cookie[COOKIE_NAME]
+except KeyError:
+return None
+if isinstance(morsel, unicode):             # Python 2.5
+return morsel
+elif isinstance(morsel, Cookie.Morsel):     # Python 2.6+
+return morsel.value
+return None
+def get_redis_connection(host=REDIS_HOST, port=REDIS_PORT, db=REDIS_DB):
+"""
+Create and return a Redis connection using the supplied parameters.
+"""
+return redis.StrictRedis(host=host, port=port, db=db)
+def validate_cookie(value):
+"""Determines if cookie was created by Django. Returns True on success,
+False on failure.
+Looks up the hash of the cookie value in Redis. If present, cookie
+is deemed legit.
+"""
+h = hashlib.sha256()
+h.update(value)
+set_member = h.hexdigest()
+conn = get_redis_connection()
+success = False
+try:
+score = conn.zscore(REDIS_SET, set_member)
+success = score is not None
+except redis.RedisError:
+logger.error('Could not check Redis for ExternalCookie auth')
+return success
+class ExternalCookie(BaseAuth):
+name = 'external_cookie'
+def __init__(self, autocreate=False):
+self.autocreate = autocreate
+BaseAuth.__init__(self)
+def request(self, request, user_obj, **kwargs):
+user = None
+try_next = True
+try:
+cookie = Cookie.SimpleCookie(request.cookies)
+except Cookie.CookieError:
+cookie = None
+val = get_cookie_value(cookie)
+if val:
+try:
+username, email, _ = val.split('#')
+except ValueError:
+return user, try_next
+if validate_cookie(val):
+user = User(request, name=username, auth_username=username,
+auth_method=self.name)
+changed = False
+if email != user.email:
+user.email = email
+changed = True
+if user:
+user.create_or_update(changed)
+if user and user.valid:
+try_next = False
+return user, try_next
+Conclusion
+==========
+I've been running this setup for a month now and it is working great. My users
+and I are enjoying our shiny new MoinMoin wiki integrated with our Django
+powered community website. The single sign-on experience is quite seamless and
+eliminates the need for separate accounts.
+.. _SurfGuitar101.com: http://surfguitar101.com
+.. _Django: https://www.djangoproject.com
+.. _MediaWiki: http://www.mediawiki.org
+.. _MoinMoin: http://moinmo.in/
+.. _Python: http://www.python.org
+.. _Python.org wiki: http://wiki.python.org/moin/
+.. _Mercurial wiki: http://mercurial.selenic.com/wiki/
+.. _omniORB wiki: http://www.omniorb-support.com/omniwiki
+.. _authentication system: http://moinmo.in/HelpOnAuthentication
+.. _external cookie authentication: http://moinmo.in/HelpOnAuthentication/ExternalCookie
+.. _login and logout signals: https://docs.djangoproject.com/en/1.4/topics/auth/#login-and-logout-signals
+.. _HttpResponse: https://docs.djangoproject.com/en/1.4/ref/request-response/#httpresponse-objects
+.. _middleware: https://docs.djangoproject.com/en/1.4/topics/http/middleware/
+.. _Redis: http://redis.io/
+.. _Bitbucket repository: https://bitbucket.org/bgneal/sg101
+.. _wiki application in bitbucket: https://bitbucket.org/bgneal/sg101/src/a5b8f25e1752faf71ed429ec7f22ff6f3b3dc851/wiki?at=default
+.. _Celery: http://celeryproject.org/

Mercurial > public > pelican-blog

comparison content/Coding/020-django-moinmoin.rst @ 4:7ce6393e6d30