Redis: fan out news feeds in list or sorted set? -

i'm caching fan-out news feeds redis in following way:

each feed activity key/value, activity:id value json string of data.

each news feed list, key feed:user:user_id , list contains keys of relevant activities.

to retrieve news feed use example: 'sort feed:user:user_id nosort * limit 0 40'

i'm considering changing feed sorted set score activity's timestamp, way feed sorted time.

i read http://arindam.quora.com/redis-sorted-sets-and-lists-pertaining-to-newsfeed recommend using lists because of time complexity of sorted sets, keep using lists have take care of insert order, inserting past story requires iterate through list , finding right index push to. (which can cause new problems in distributed environments).

should keep using lists or go sorted sets?

is there way retrieve news feed instantly sorted set, (like sort ... * command list) or have zrange , iterating through results , getting each value?

yes, sorted sets fast , powerful. seem better match requirements sort operations. time complexity misunderstood. o(log(n)) fast, , scales fine. use tens of millions of members in 1 sorted set. retrieval , insertion sub-millisecond.

use zrangebyscore key min max withscores [limit offset count] results.

depending on how store timestamps 'scores', zrevrangebyscore might better.

a small remark timestamps: sorted set scores don't need decimal part should using 15 digits or less. score has stay in range -999999999999999 999999999999999. note: these limits exist because redis server stores score (float) redis-string representation internally.

i therefore recommend format, converted zulu time: -20140313122802 second-precision. may add 1 digit 100ms-precision, no more if want no loss in precision. it's still float64 way, loss of precision fine in scenarios, case fits in 'perfect precision' range, that's recommend.

if data expires within 10 years, can skip 3 first digits (ccy of ccyy), achieve .0001 second precision.

i suggest negative scores here, can use simpler zrangebyscore instead of rev one. can use -inf start score (minus infinity) , limit 0 100 top 100 results.

two sorted set members (or 'keys' that's ambiguous since sorted set key in itself) may share score, that's no problem, results within identical score alphabetical.

hope helps, tw

edit after chat

the op wanted collect data (using zset) different keys (get/set or hget/hset keys). join can you, zrangebyscore can't. preferred way of doing this, simple lua script. lua script executed on server. in example below use eval simplicity, in production use script exists, script load , evalsha. client libraries have bookkeeping logic built-in, don't upload script each time.

here's example.lua:

local r={} local zkey=keys[1] local a=redis.call('zrangebyscore', zkey, keys[2], keys[3], 'withscores', 'limit', 0, keys[4]) i=1,#a,2   r[i]=a[i+1]   r[i+1]=redis.call('get', a[i]) end return r 

you use (raw example, not coded performance):

redis-cli -p 14322 set activity:1 act1json redis-cli -p 14322 set activity:2 act2json redis-cli -p 14322 zadd feed 1 activity:1 redis-cli -p 14322 zadd feed 2 activity:2   redis-cli -p 14322 eval '$(cat example.lua)' 4 feed '-inf' '+inf' 100 

result:

1) "1" 2) "act1json" 3) "2" 4) "act2json"