|
bgneal@4
|
1 Who's Online with Redis & Python, a slight return
|
|
bgneal@4
|
2 #################################################
|
|
bgneal@4
|
3
|
|
bgneal@4
|
4 :date: 2011-12-17 19:05
|
|
bgneal@4
|
5 :tags: Redis, Python
|
|
bgneal@4
|
6 :slug: who-s-online-with-redis-python-a-slight-return
|
|
bgneal@4
|
7 :author: Brian Neal
|
|
bgneal@4
|
8
|
|
bgneal@4
|
9 In a `previous post`_, I blogged about building a "Who's Online" feature using
|
|
bgneal@4
|
10 Redis_ and Python_ with redis-py_. I've been integrating Celery_ into my
|
|
bgneal@4
|
11 website, and I stumbled across this old code. Since I made that post, I
|
|
bgneal@4
|
12 discovered yet another cool feature in Redis: sorted sets. So here is an even
|
|
bgneal@4
|
13 better way of implementing this feature using Redis sorted sets.
|
|
bgneal@4
|
14
|
|
bgneal@4
|
15 A sorted set in Redis is like a regular set, but each member has a numeric
|
|
bgneal@4
|
16 score. When you add a member to a sorted set, you also specify the score for
|
|
bgneal@4
|
17 that member. You can then retrieve set members if their score falls into a
|
|
bgneal@4
|
18 certain range. You can also easily remove members outside a given score range.
|
|
bgneal@4
|
19
|
|
bgneal@4
|
20 For a "Who's Online" feature, we need a sorted set to represent the set
|
|
bgneal@4
|
21 of all users online. Whenever we see a user, we insert that user into the set
|
|
bgneal@4
|
22 along with the current time as their score. This is accomplished with the Redis
|
|
bgneal@4
|
23 zadd_ command. If the user is already in the set, zadd_ simply updates
|
|
bgneal@4
|
24 their score with the current time.
|
|
bgneal@4
|
25
|
|
bgneal@4
|
26 To obtain the curret list of who's online, we use the zrangebyscore_ command to
|
|
bgneal@4
|
27 retrieve the list of users who's score (time) lies between, say, 15 minutes ago,
|
|
bgneal@4
|
28 until now.
|
|
bgneal@4
|
29
|
|
bgneal@4
|
30 Periodically, we need to remove stale members from the set. This can be
|
|
bgneal@4
|
31 accomplished by using the zremrangebyscore_ command. This command will remove
|
|
bgneal@4
|
32 all members that have a score between minimum and maximum values. In this case,
|
|
bgneal@4
|
33 we can use the beginning of time for the minimum, and 15 minutes ago for the
|
|
bgneal@4
|
34 maximum.
|
|
bgneal@4
|
35
|
|
bgneal@4
|
36 That's really it in a nutshell. This is much simpler than my previous
|
|
bgneal@4
|
37 solution which used two sets.
|
|
bgneal@4
|
38
|
|
bgneal@4
|
39 So let's look at some code. The first problem we need to solve is how to
|
|
bgneal@4
|
40 convert a Python ``datetime`` object into a score. This can be accomplished by
|
|
bgneal@4
|
41 converting the ``datetime`` into a POSIX timestamp integer, which is the number
|
|
bgneal@4
|
42 of seconds from the UNIX epoch of January 1, 1970.
|
|
bgneal@4
|
43
|
|
bgneal@4
|
44 .. sourcecode:: python
|
|
bgneal@4
|
45
|
|
bgneal@4
|
46 import datetime
|
|
bgneal@4
|
47 import time
|
|
bgneal@4
|
48
|
|
bgneal@4
|
49 def to_timestamp(dt):
|
|
bgneal@4
|
50 """
|
|
bgneal@4
|
51 Turn the supplied datetime object into a UNIX timestamp integer.
|
|
bgneal@4
|
52
|
|
bgneal@4
|
53 """
|
|
bgneal@4
|
54 return int(time.mktime(dt.timetuple()))
|
|
bgneal@4
|
55
|
|
bgneal@4
|
56 With that handy function, here are some examples of the operations described
|
|
bgneal@4
|
57 above.
|
|
bgneal@4
|
58
|
|
bgneal@4
|
59 .. sourcecode:: python
|
|
bgneal@4
|
60
|
|
bgneal@4
|
61 import redis
|
|
bgneal@4
|
62
|
|
bgneal@4
|
63 # Redis set keys:
|
|
bgneal@4
|
64 USER_SET_KEY = "whos_online:users"
|
|
bgneal@4
|
65
|
|
bgneal@4
|
66 # the period over which we collect who's online stats:
|
|
bgneal@4
|
67 MAX_AGE = datetime.timedelta(minutes=15)
|
|
bgneal@4
|
68
|
|
bgneal@4
|
69 # obtain a connection to redis:
|
|
bgneal@4
|
70 conn = redis.StrictRedis()
|
|
bgneal@4
|
71
|
|
bgneal@4
|
72 # add/update a user to the who's online set:
|
|
bgneal@4
|
73
|
|
bgneal@4
|
74 username = "sally"
|
|
bgneal@4
|
75 ts = to_timestamp(datetime.datetime.now())
|
|
bgneal@4
|
76 conn.zadd(USER_SET_KEY, ts, username)
|
|
bgneal@4
|
77
|
|
bgneal@4
|
78 # retrieve the list of users who have been active in the last MAX_AGE minutes
|
|
bgneal@4
|
79
|
|
bgneal@4
|
80 now = datetime.datetime.now()
|
|
bgneal@4
|
81 min = to_timestamp(now - MAX_AGE)
|
|
bgneal@4
|
82 max = to_timestamp(now)
|
|
bgneal@4
|
83
|
|
bgneal@4
|
84 whos_online = conn.zrangebyscore(USER_SET_KEY, min, max)
|
|
bgneal@4
|
85
|
|
bgneal@4
|
86 # e.g. whos_online = ['sally', 'harry', 'joe']
|
|
bgneal@4
|
87
|
|
bgneal@4
|
88 # periodically remove stale members
|
|
bgneal@4
|
89
|
|
bgneal@4
|
90 cutoff = to_timestamp(datetime.datetime.now() - MAX_AGE)
|
|
bgneal@4
|
91 conn.zremrangebyscore(USER_SET_KEY, 0, cutoff)
|
|
bgneal@4
|
92
|
|
bgneal@4
|
93 .. _previous post: http://deathofagremmie.com/2011/04/25/a-better-who-s-online-with-redis-python/
|
|
bgneal@4
|
94 .. _Redis: http://redis.io/
|
|
bgneal@4
|
95 .. _Python: http://www.python.org
|
|
bgneal@4
|
96 .. _redis-py: https://github.com/andymccurdy/redis-py
|
|
bgneal@4
|
97 .. _Celery: http://celeryproject.org
|
|
bgneal@4
|
98 .. _zadd: http://redis.io/commands/zadd
|
|
bgneal@4
|
99 .. _zrangebyscore: http://redis.io/commands/zrangebyscore
|
|
bgneal@4
|
100 .. _zremrangebyscore: http://redis.io/commands/zremrangebyscore
|
