Ben Ben - 1 month ago 33
Apache Configuration Question

Distribute partitioned IgniteCache on selected nodes only along deployed Service

I am running an Apache Ignite cluster with various nodes which provide services. Each node is assigned a certain service group (custom node attributes) describing the service which is being provided by this node, e.g. Authorization, Payment, ...

Some of these services are utilizing the Ignite Cache but I want these service specific caches deployed on the associated service nodes only. So I am adding a node filter to my cache configuration:

// configuration with custom attribute is provided at node start up
IgniteConfiguration nodeConfig = new IgniteConfiguration();
Map<String,String> nodeAttributes = Collections.singletonMap("role", "MyService");
nodeConfig.setUserAttributes(nodeAttributes);

CacheConfiguration cfg = new CacheConfiguration<>("MyServiceCache");
cfg.setNodeFilter((node) -> node.attribute("role") == nodeConfig .getUserAttributes().get("role"));


The cache deployment is working as expected (at least no errors are shown). However when I add a 2nd node which provides the same service (for reasons of scaling) a log entry is shown:

No server nodes found for cache client: MyServiceCache


As soon as I stop the 2nd node another log message is showing up:

[17:26:29] Topology snapshot [ver=9, servers=1, clients=0, CPUs=4, heap=1.7GB]
[17:26:29] All server nodes for the following caches have left the cluster: 'MyServiceCache'
[17:26:29] Must have server nodes for caches to operate.


As far as I understand the 1st node should still be serving the service and cache. So these messages do not make much sense to me. Can someone elaborate please?

Here's a concrete example:


  • Run
    MyService.java
    twice or more

  • Stop 1st node

  • Watch log output of 2nd node



NodeConfig.java

public class NodeConfig {

public static IgniteConfiguration myServiceNode()
{
IgniteConfiguration nodeConfig = new IgniteConfiguration();
Map<String,String> nodeAttributes = Collections.singletonMap("role", "myService");
nodeConfig.setUserAttributes(nodeAttributes);
return nodeConfig;
}

}


CacheConfig.java

public class CacheConfig {

public static CacheConfiguration<Long, String> myServiceCache() {

CacheConfiguration<Long, String> cfg = new CacheConfiguration<>("MyServiceCache");

cfg.setBackups(2);

cfg.setNodeFilter((node) -> node.attribute("role") == NodeConfig.myServiceNode().getUserAttributes().get("role"));

return cfg;
}

}


MyService.java

public class MyService implements Service {

@IgniteInstanceResource
private Ignite ignite;
private IgniteCache cache;

@Override
public void cancel(ServiceContext serviceContext) {
System.out.println("Service " + serviceContext.name() + " cancelled.");
}

@Override
public void init(ServiceContext serviceContext) throws Exception {
System.out.println("Service " + serviceContext.name() + " initialized.");
}

@Override
public void execute(ServiceContext serviceContext) throws Exception {

CacheConfiguration config = CacheConfig.myServiceCache();
cache = ignite.getOrCreateCache(config).withExpiryPolicy(new CreatedExpiryPolicy(Duration.ONE_MINUTE));

System.out.println("Service " + serviceContext.name() + " executing.");
}

public static void main(String[] args) {

Ignite ignite = Ignition.start(NodeConfig.myServiceNode());

IgniteServices svcs = ignite.services(ignite.cluster().forAttribute("role", NodeConfig.myServiceNode().getUserAttributes().get("role")));

svcs.deployNodeSingleton("MyService", new MyService());

}

}

Answer

You should not use == to compare attribute values. This condition works fine:

cfg.setNodeFilter((node) -> Objects.equals(node.attribute("role"), NodeConfig.myServiceNode().getUserAttributes().get("role")));
Comments