Who's Online with Redis & Python, a slight return
#################################################

:date: 2011-12-17 19:05
:tags: Redis, Python
:slug: who-s-online-with-redis-python-a-slight-return
:author: Brian Neal

In a `previous post`_, I blogged about building a "Who's Online" feature using
Redis_ and Python_ with redis-py_.  I've been integrating Celery_ into my
website, and I stumbled across this old code. Since I made that post, I
discovered yet another cool feature in Redis: sorted sets. So here is an even
better way of implementing this feature using Redis sorted sets.

A sorted set in Redis is like a regular set, but each member has a numeric
score. When you add a member to a sorted set, you also specify the score for
that member. You can then retrieve set members if their score falls into a
certain range. You can also easily remove members outside a given score range.

For a "Who's Online" feature, we need a sorted set to represent the set
of all users online. Whenever we see a user, we insert that user into the set
along with the current time as their score. This is accomplished with the Redis
zadd_ command.  If the user is already in the set, zadd_ simply updates
their score with the current time.

To obtain the curret list of who's online, we use the zrangebyscore_ command to
retrieve the list of users who's score (time) lies between, say, 15 minutes ago,
until now.

Periodically, we need to remove stale members from the set. This can be
accomplished by using the zremrangebyscore_ command. This command will remove
all members that have a score between minimum and maximum values. In this case,
we can use the beginning of time for the minimum, and 15 minutes ago for the
maximum.

That's really it in a nutshell. This is much simpler than my previous
solution which used two sets.

So let's look at some code. The first problem we need to solve is how to
convert a Python ``datetime`` object into a score. This can be accomplished by
converting the ``datetime`` into a POSIX timestamp integer, which is the number
of seconds from the UNIX epoch of January 1, 1970.

.. sourcecode:: python

   import datetime
   import time

   def to_timestamp(dt):
       """
       Turn the supplied datetime object into a UNIX timestamp integer.

       """
       return int(time.mktime(dt.timetuple()))

With that handy function, here are some examples of the operations described
above.

.. sourcecode:: python

   import redis

   # Redis set keys:
   USER_SET_KEY = "whos_online:users"

   # the period over which we collect who's online stats:
   MAX_AGE = datetime.timedelta(minutes=15)

   # obtain a connection to redis:
   conn = redis.StrictRedis()

   # add/update a user to the who's online set:

   username = "sally"
   ts = to_timestamp(datetime.datetime.now())
   conn.zadd(USER_SET_KEY, ts, username)

   # retrieve the list of users who have been active in the last MAX_AGE minutes

   now = datetime.datetime.now()
   min = to_timestamp(now - MAX_AGE)
   max = to_timestamp(now)

   whos_online = conn.zrangebyscore(USER_SET_KEY, min, max)

   # e.g. whos_online = ['sally', 'harry', 'joe']

   # periodically remove stale members

   cutoff = to_timestamp(datetime.datetime.now() - MAX_AGE)
   conn.zremrangebyscore(USER_SET_KEY, 0, cutoff)

.. _previous post: http://deathofagremmie.com/2011/04/25/a-better-who-s-online-with-redis-python/
.. _Redis: http://redis.io/
.. _Python: http://www.python.org
.. _redis-py: https://github.com/andymccurdy/redis-py
.. _Celery: http://celeryproject.org
.. _zadd: http://redis.io/commands/zadd
.. _zrangebyscore: http://redis.io/commands/zrangebyscore
.. _zremrangebyscore: http://redis.io/commands/zremrangebyscore