Ben Ben - 1 year ago 140
Apache Configuration Question

Distribute partitioned IgniteCache on selected nodes only along deployed Service

I am running an Apache Ignite cluster with various nodes which provide services. Each node is assigned a certain service group (custom node attributes) describing the service which is being provided by this node, e.g. Authorization, Payment, ...

Some of these services are utilizing the Ignite Cache but I want these service specific caches deployed on the associated service nodes only. So I am adding a node filter to my cache configuration:

// configuration with custom attribute is provided at node start up
IgniteConfiguration nodeConfig = new IgniteConfiguration();
Map<String,String> nodeAttributes = Collections.singletonMap("role", "MyService");

CacheConfiguration cfg = new CacheConfiguration<>("MyServiceCache");
cfg.setNodeFilter((node) -> node.attribute("role") == nodeConfig .getUserAttributes().get("role"));

The cache deployment is working as expected (at least no errors are shown). However when I add a 2nd node which provides the same service (for reasons of scaling) a log entry is shown:

No server nodes found for cache client: MyServiceCache

As soon as I stop the 2nd node another log message is showing up:

[17:26:29] Topology snapshot [ver=9, servers=1, clients=0, CPUs=4, heap=1.7GB]
[17:26:29] All server nodes for the following caches have left the cluster: 'MyServiceCache'
[17:26:29] Must have server nodes for caches to operate.

As far as I understand the 1st node should still be serving the service and cache. So these messages do not make much sense to me. Can someone elaborate please?

Here's a concrete example:

  • Run
    twice or more

  • Stop 1st node

  • Watch log output of 2nd node

public class NodeConfig {

public static IgniteConfiguration myServiceNode()
IgniteConfiguration nodeConfig = new IgniteConfiguration();
Map<String,String> nodeAttributes = Collections.singletonMap("role", "myService");
return nodeConfig;


public class CacheConfig {

public static CacheConfiguration<Long, String> myServiceCache() {

CacheConfiguration<Long, String> cfg = new CacheConfiguration<>("MyServiceCache");


cfg.setNodeFilter((node) -> node.attribute("role") == NodeConfig.myServiceNode().getUserAttributes().get("role"));

return cfg;


public class MyService implements Service {

private Ignite ignite;
private IgniteCache cache;

public void cancel(ServiceContext serviceContext) {
System.out.println("Service " + + " cancelled.");

public void init(ServiceContext serviceContext) throws Exception {
System.out.println("Service " + + " initialized.");

public void execute(ServiceContext serviceContext) throws Exception {

CacheConfiguration config = CacheConfig.myServiceCache();
cache = ignite.getOrCreateCache(config).withExpiryPolicy(new CreatedExpiryPolicy(Duration.ONE_MINUTE));

System.out.println("Service " + + " executing.");

public static void main(String[] args) {

Ignite ignite = Ignition.start(NodeConfig.myServiceNode());

IgniteServices svcs ="role", NodeConfig.myServiceNode().getUserAttributes().get("role")));

svcs.deployNodeSingleton("MyService", new MyService());



Answer Source

You should not use == to compare attribute values. This condition works fine:

cfg.setNodeFilter((node) -> Objects.equals(node.attribute("role"), NodeConfig.myServiceNode().getUserAttributes().get("role")));
Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download