Arman Arman - 2 months ago 11
Java Question

Spring XD transform and route message in one module

How to transform and route a message in 1 module.

I created a module like this:

xd-ta-core.xml

<int:channel id="input"/>
<int:channel id="output"/>
<beans:bean id="taCore" class="com.company.threatanalyzer.xd.plugin.XdTaCore">
<constructor-arg value="${inputLogsType}"/>
<constructor-arg value="${zookeeperHost}"/>
<property name="persistChannel" value="${persistChannel}"/>
<property name="correlateChannel" value="${correlateChannel}"/>
</beans:bean>

<int:transformer input-channel="input" output-channel="output" ref="taCore" method="transform"/>
<int:router input-channel="output" ref="taCore" method="route"/>


XdTaCore.java

public Event transform(String payload) {
Event event = tupleDeserializer.fromJson(payload);
event.setId(UUID.randomUUID().toString());
logger.warn("=======TA Core Transform======= {} ===========", event.getId());
return event;
}

public List<String> route(Event payload) {
List<String> outChannels = new ArrayList<>();
List<PolicyAction> policyActions = policyEnforcer.enforce(payload);

if (policyActions.contains(PolicyAction.PERSIST)) {
outChannels.add(persistChannel);
}
if (policyActions.contains(PolicyAction.CORRELATE)) {
outChannels.add(correlateChannel);
}
logger.warn("=======TA Core Route======= {} ========== {} ===============", payload.getId(), policyActions);

return outChannels;
}


but in logs i get like this:

2016-09-10T14:35:26+0430 1.3.1.RELEASE WARN pool-27-thread-1 plugin.XdTaCore - =======TA Core Transform======= dc6209e2-79ff-4b88-a245-4b12a603058c ========== {}
2016-09-10T14:35:26+0430 1.3.1.RELEASE WARN pool-27-thread-1 plugin.XdTaCore - =======TA Core Route======= dc6209e2-79ff-4b88-a245-4b12a603058c ========== [CORRELATE, PERSIST] ===========
2016-09-10T14:35:26+0430 1.3.1.RELEASE WARN pool-27-thread-1 plugin.XdTaCore - =======TA Core Transform======= fabcf574-32b1-4510-96c5-8be44ae8b757 ========== {}
2016-09-10T14:35:27+0430 1.3.1.RELEASE WARN pool-27-thread-1 plugin.XdTaCore - =======TA Core Transform======= 4f07e5e8-7f0e-4e48-a6a0-813e57dd081c ========== {}
2016-09-10T14:35:27+0430 1.3.1.RELEASE WARN pool-27-thread-1 plugin.XdTaCore - =======TA Core Route======= 4f07e5e8-7f0e-4e48-a6a0-813e57dd081c ========== [CORRELATE, PERSIST] ===========
2016-09-10T14:35:27+0430 1.3.1.RELEASE WARN pool-27-thread-1 plugin.XdTaCore - =======TA Core Transform======= 6393be33-6188-405d-b0b3-8e9f87ec4af0 ========== {}
2016-09-10T14:35:28+0430 1.3.1.RELEASE WARN pool-27-thread-1 plugin.XdTaCore - =======TA Core Transform======= 58b17358-58b8-46f3-894a-b7b5811564bc ========== {}
2016-09-10T14:35:28+0430 1.3.1.RELEASE WARN pool-27-thread-1 plugin.XdTaCore - =======TA Core Route======= 58b17358-58b8-46f3-894a-b7b5811564bc ========== [CORRELATE, PERSIST] ===========
2016-09-10T14:35:28+0430 1.3.1.RELEASE WARN pool-27-thread-1 plugin.XdTaCore - =======TA Core Transform======= e92e153d-8f39-4ae5-9c82-14a7f141aef5 ========== {}


As you can see the transform function runs for each message but route function just run for half of them.

And another question is which one is better for performance:


  • one module that do 10 line of code

  • 5 module that each one do 2 line of code


Answer

There are two subscribers to channel output - the message bus and your router. In that scenario, messages will be alternately sent to each.

You need to use a third, intermediate, channel between your transformer and router (e.g. toRouter)...

<int:transformer input-channel="input" output-channel="toRouter" ref="taCore" method="transform"/>

<int:router input-channel="toRouter" ref="taCore" method="route"/>

The transformer output channel is the router's input channel.

And another question is which one is better for performance:

It depends; for a simple component like this, it's better to use a single module. For more complicated scenarios, where you might want to scale the components independently, separate modules might be better.