David Moles David Moles - 4 months ago 10
Java Question

Per-key blocking Map in Java

I'm dealing with some third-party library code that involves creating expensive objects and caching them in a

Map
. The existing implementation is something like

lock.lock()
try {
Foo result = cache.get(key);
if (result == null) {
result = createFooExpensively(key);
cache.put(key, result);
}
return result;
} finally {
lock.unlock();
}


Obviously this is not the best design when
Foos
for different
keys
can be created independently.

My current hack is to use a
Map
of
Futures
:

lock.lock();
Future<Foo> future;
try {
future = allFutures.get(key);
if (future == null) {
future = executorService.submit(new Callable<Foo>() {
public Foo call() {
return createFooExpensively(key);
}
});
allFutures.put(key, future);
}
} finally {
lock.unlock();
}

try {
return future.get();
} catch (InterruptedException e) {
throw new MyRuntimeException(e);
} catch (ExecutionException e) {
throw new MyRuntimeException(e);
}


But this seems... a little hacky, for two reasons:


  1. The work is done on an arbitrary pooled thread. I'd be happy to have the work
    done on the first thread that tries to get that particular key, especially since
    it's going to be blocked anyway.

  2. Even when the
    Map
    is fully populated, we still go through
    Future.get()
    to get
    the results. I expect this is pretty cheap, but it's ugly.



What I'd like is to replace
cache
with a
Map
that will block gets for a given key until that key has a value, but allow other gets meanwhile. Does any such thing exist? Or does someone have a cleaner alternative to the
Map
of
Futures
?

Answer

Creating a lock per key sounds tempting, but it may not be what you want, especially when the number of keys is large.

As you would probably need to create a dedicated (read-write) lock for each key, it has impact on your memory usage. Also, that fine granularity may hit a point of diminishing returns given a finite number of cores if concurrency is truly high.

ConcurrentHashMap is oftentimes a good enough solution in a situation like this. It provides normally full reader concurrency (normally readers do not block), and updates can be concurrent up to the level of concurrency level desired. This gives you pretty good scalability. The above code may be expressed with ConcurrentHashMap like the following:

ConcurrentMap<Key,Foo> cache = new ConcurrentHashMap<>();
...
Foo result = cache.get(key);
if (result == null) {
  result = createFooExpensively(key);
  Foo old = cache.putIfAbsent(key, result);
  if (old != null) {
    result = old;
  }
}

The straightforward use of ConcurrentHashMap does have one drawback, which is that multiple threads may find that the key is not cached, and each may invoke createFooExpensively(). As a result, some threads may do throw-away work. To avoid this, you would want to use the memoizer pattern that's mentioned in "Java Concurrency in Practice".

But then again, the nice folks at Google already solved these problems for you in the form of CacheBuilder: http://docs.guava-libraries.googlecode.com/git-history/release/javadoc/com/google/common/cache/CacheBuilder.html

LoadingCache<Key,Foo> cache = CacheBuilder.newBuilder().
  concurrencyLevel(32).
  build(new CacheLoader<Key,Foo>() {
    public Foo load(Key key) {
      return createFooExpensively(key);
    }
  });

...
Foo result = cache.get(key);