diff content/Coding/010-redis-whos-online-return.rst @ 4:7ce6393e6d30

Adding converted blog posts from old blog.
author Brian Neal <bgneal@gmail.com>
date Thu, 30 Jan 2014 21:45:03 -0600
parents
children
line wrap: on
line diff
--- /dev/null	Thu Jan 01 00:00:00 1970 +0000
+++ b/content/Coding/010-redis-whos-online-return.rst	Thu Jan 30 21:45:03 2014 -0600
@@ -0,0 +1,100 @@
+Who's Online with Redis & Python, a slight return
+#################################################
+
+:date: 2011-12-17 19:05
+:tags: Redis, Python
+:slug: who-s-online-with-redis-python-a-slight-return
+:author: Brian Neal
+
+In a `previous post`_, I blogged about building a "Who's Online" feature using
+Redis_ and Python_ with redis-py_.  I've been integrating Celery_ into my
+website, and I stumbled across this old code. Since I made that post, I
+discovered yet another cool feature in Redis: sorted sets. So here is an even
+better way of implementing this feature using Redis sorted sets.
+
+A sorted set in Redis is like a regular set, but each member has a numeric
+score. When you add a member to a sorted set, you also specify the score for
+that member. You can then retrieve set members if their score falls into a
+certain range. You can also easily remove members outside a given score range.
+
+For a "Who's Online" feature, we need a sorted set to represent the set
+of all users online. Whenever we see a user, we insert that user into the set
+along with the current time as their score. This is accomplished with the Redis
+zadd_ command.  If the user is already in the set, zadd_ simply updates
+their score with the current time.
+
+To obtain the curret list of who's online, we use the zrangebyscore_ command to
+retrieve the list of users who's score (time) lies between, say, 15 minutes ago,
+until now.
+
+Periodically, we need to remove stale members from the set. This can be
+accomplished by using the zremrangebyscore_ command. This command will remove
+all members that have a score between minimum and maximum values. In this case,
+we can use the beginning of time for the minimum, and 15 minutes ago for the
+maximum.
+
+That's really it in a nutshell. This is much simpler than my previous
+solution which used two sets.
+
+So let's look at some code. The first problem we need to solve is how to
+convert a Python ``datetime`` object into a score. This can be accomplished by
+converting the ``datetime`` into a POSIX timestamp integer, which is the number
+of seconds from the UNIX epoch of January 1, 1970.
+
+.. sourcecode:: python
+
+   import datetime
+   import time
+
+   def to_timestamp(dt):
+       """
+       Turn the supplied datetime object into a UNIX timestamp integer.
+
+       """
+       return int(time.mktime(dt.timetuple()))
+
+With that handy function, here are some examples of the operations described
+above.
+
+.. sourcecode:: python
+
+   import redis
+
+   # Redis set keys:
+   USER_SET_KEY = "whos_online:users"
+
+   # the period over which we collect who's online stats:
+   MAX_AGE = datetime.timedelta(minutes=15)
+
+   # obtain a connection to redis:
+   conn = redis.StrictRedis()
+
+   # add/update a user to the who's online set:
+
+   username = "sally"
+   ts = to_timestamp(datetime.datetime.now())
+   conn.zadd(USER_SET_KEY, ts, username)
+
+   # retrieve the list of users who have been active in the last MAX_AGE minutes
+
+   now = datetime.datetime.now()
+   min = to_timestamp(now - MAX_AGE)
+   max = to_timestamp(now)
+
+   whos_online = conn.zrangebyscore(USER_SET_KEY, min, max)
+
+   # e.g. whos_online = ['sally', 'harry', 'joe']
+
+   # periodically remove stale members
+
+   cutoff = to_timestamp(datetime.datetime.now() - MAX_AGE)
+   conn.zremrangebyscore(USER_SET_KEY, 0, cutoff)
+
+.. _previous post: http://deathofagremmie.com/2011/04/25/a-better-who-s-online-with-redis-python/
+.. _Redis: http://redis.io/
+.. _Python: http://www.python.org
+.. _redis-py: https://github.com/andymccurdy/redis-py
+.. _Celery: http://celeryproject.org
+.. _zadd: http://redis.io/commands/zadd
+.. _zrangebyscore: http://redis.io/commands/zrangebyscore
+.. _zremrangebyscore: http://redis.io/commands/zremrangebyscore