Mikey S. Mikey S. - 1 month ago 21
Ruby Question

Ruby - Redis based mutex with expiration implementation

I'm trying to implement a memory based, multi process shared mutex, which supports timeout, using Redis.

I need the mutex to be non-blocking, meaning that I just need to be able to know if I was able to fetch the mutex or not, and if not - simply continue with execution of fallback code.

something along these lines:

if lock('my_lock_key', timeout: 1.minute)
# Do some job
else
# exit
end


An un-expiring mutex could be implemented using redis's
setnx mutex 1
:

if redis.setnx('#{mutex}', '1')
# Do some job
redis.delete('#{mutex}')
else
# exit
end


But what if I need a mutex with a timeout mechanism (In order to avoid a situation where the ruby code fails before the
redis.delete
command, resulting the mutex being locked forever, for example, but not for this reason only).

Doing something like this obviously doesn't work:

redis.multi do
redis.setnx('#{mutex}', '1')
redis.expire('#{mutex}', key_timeout)
end


since I'm re-setting an expiration to the mutex EVEN if I wasn't able to set the mutex (
setnx
returns 0).

Naturally, I would've expected to have something like
setnxex
which atomically sets a key's value with an expiration time, but only if the key does not exist already. Unfortunately, Redis does not support this as far as I know.

I did however, find
renamenx key otherkey
, which lets you rename a key to some other key, only if the other key does not already exist.

I came up with something like this (for demonstration purposes, I wrote it down monolithically, and didn't break it down to methods):

result = redis.multi do
dummy_key = "mutex:dummy:#{Time.now.to_f}#{key}"
redis.setex dummy_key, key_timeout, 0
redis.renamenx dummy_key, key
end
if result.length > 1 && result.second == 1
# do some job
redis.delete key
else
# exit
end


Here, i'm setting an expiration for a dummy key, and try to rename it to the real key (in one transaction).

If the
renamenx
operation fails, then we weren't able to obtain the mutex, but no harm done: the dummy key will expire (it can be optionally deleted immediately by adding one line of code) and the real key's expiration time will remain intact.

If the
renamenx
operation succeeded, then we were able to obtain the mutex, and the mutex will get the desired expiration time.

Can anyone see any flaw with the above solution? Is there a more standard solution for this problem? I would really hate using an external gem in order to solve this problem...

Answer

If you're using Redis 2.6+, you can do this much more simply with the Lua scripting engine. The Redis documentation says:

A Redis script is transactional by definition, so everything you can do with a Redis transaction, you can also do with a script, and usually the script will be both simpler and faster.

Implementing it is trivial:

LUA_ACQUIRE = "return redis.call('setnx', KEYS[1], 1) == 1 and redis.call('expire', KEYS[1], KEYS[2]) and 1 or 0"
def lock(key, timeout = 3600)
  if redis.eval(LUA_ACQUIRE, key, timeout) == 1
    begin
      yield
    ensure
      r.del key
    end
  end
end

Usage:

lock("somejob") { do_exclusive_job }