bgneal@4: Who's Online with Redis & Python, a slight return
bgneal@4: #################################################
bgneal@4: 
bgneal@4: :date: 2011-12-17 19:05
bgneal@4: :tags: Redis, Python
bgneal@4: :slug: who-s-online-with-redis-python-a-slight-return
bgneal@4: :author: Brian Neal
bgneal@4: 
bgneal@4: In a `previous post`_, I blogged about building a "Who's Online" feature using
bgneal@4: Redis_ and Python_ with redis-py_.  I've been integrating Celery_ into my
bgneal@4: website, and I stumbled across this old code. Since I made that post, I
bgneal@4: discovered yet another cool feature in Redis: sorted sets. So here is an even
bgneal@4: better way of implementing this feature using Redis sorted sets.
bgneal@4: 
bgneal@4: A sorted set in Redis is like a regular set, but each member has a numeric
bgneal@4: score. When you add a member to a sorted set, you also specify the score for
bgneal@4: that member. You can then retrieve set members if their score falls into a
bgneal@4: certain range. You can also easily remove members outside a given score range.
bgneal@4: 
bgneal@4: For a "Who's Online" feature, we need a sorted set to represent the set
bgneal@4: of all users online. Whenever we see a user, we insert that user into the set
bgneal@4: along with the current time as their score. This is accomplished with the Redis
bgneal@4: zadd_ command.  If the user is already in the set, zadd_ simply updates
bgneal@4: their score with the current time.
bgneal@4: 
bgneal@4: To obtain the curret list of who's online, we use the zrangebyscore_ command to
bgneal@4: retrieve the list of users who's score (time) lies between, say, 15 minutes ago,
bgneal@4: until now.
bgneal@4: 
bgneal@4: Periodically, we need to remove stale members from the set. This can be
bgneal@4: accomplished by using the zremrangebyscore_ command. This command will remove
bgneal@4: all members that have a score between minimum and maximum values. In this case,
bgneal@4: we can use the beginning of time for the minimum, and 15 minutes ago for the
bgneal@4: maximum.
bgneal@4: 
bgneal@4: That's really it in a nutshell. This is much simpler than my previous
bgneal@4: solution which used two sets.
bgneal@4: 
bgneal@4: So let's look at some code. The first problem we need to solve is how to
bgneal@4: convert a Python ``datetime`` object into a score. This can be accomplished by
bgneal@4: converting the ``datetime`` into a POSIX timestamp integer, which is the number
bgneal@4: of seconds from the UNIX epoch of January 1, 1970.
bgneal@4: 
bgneal@4: .. sourcecode:: python
bgneal@4: 
bgneal@4:    import datetime
bgneal@4:    import time
bgneal@4: 
bgneal@4:    def to_timestamp(dt):
bgneal@4:        """
bgneal@4:        Turn the supplied datetime object into a UNIX timestamp integer.
bgneal@4: 
bgneal@4:        """
bgneal@4:        return int(time.mktime(dt.timetuple()))
bgneal@4: 
bgneal@4: With that handy function, here are some examples of the operations described
bgneal@4: above.
bgneal@4: 
bgneal@4: .. sourcecode:: python
bgneal@4: 
bgneal@4:    import redis
bgneal@4: 
bgneal@4:    # Redis set keys:
bgneal@4:    USER_SET_KEY = "whos_online:users"
bgneal@4: 
bgneal@4:    # the period over which we collect who's online stats:
bgneal@4:    MAX_AGE = datetime.timedelta(minutes=15)
bgneal@4: 
bgneal@4:    # obtain a connection to redis:
bgneal@4:    conn = redis.StrictRedis()
bgneal@4: 
bgneal@4:    # add/update a user to the who's online set:
bgneal@4: 
bgneal@4:    username = "sally"
bgneal@4:    ts = to_timestamp(datetime.datetime.now())
bgneal@4:    conn.zadd(USER_SET_KEY, ts, username)
bgneal@4: 
bgneal@4:    # retrieve the list of users who have been active in the last MAX_AGE minutes
bgneal@4: 
bgneal@4:    now = datetime.datetime.now()
bgneal@4:    min = to_timestamp(now - MAX_AGE)
bgneal@4:    max = to_timestamp(now)
bgneal@4: 
bgneal@4:    whos_online = conn.zrangebyscore(USER_SET_KEY, min, max)
bgneal@4: 
bgneal@4:    # e.g. whos_online = ['sally', 'harry', 'joe']
bgneal@4: 
bgneal@4:    # periodically remove stale members
bgneal@4: 
bgneal@4:    cutoff = to_timestamp(datetime.datetime.now() - MAX_AGE)
bgneal@4:    conn.zremrangebyscore(USER_SET_KEY, 0, cutoff)
bgneal@4: 
bgneal@4: .. _previous post: http://deathofagremmie.com/2011/04/25/a-better-who-s-online-with-redis-python/
bgneal@4: .. _Redis: http://redis.io/
bgneal@4: .. _Python: http://www.python.org
bgneal@4: .. _redis-py: https://github.com/andymccurdy/redis-py
bgneal@4: .. _Celery: http://celeryproject.org
bgneal@4: .. _zadd: http://redis.io/commands/zadd
bgneal@4: .. _zrangebyscore: http://redis.io/commands/zrangebyscore
bgneal@4: .. _zremrangebyscore: http://redis.io/commands/zremrangebyscore