annedroiid annedroiid - 1 year ago 155
Java Question

KafkaConsumer never exits .poll method - GroupCoordinatorNotAvailableException

I have an implementation of a

in java, and currently it is never exiting the
method. When I drill down into the source code in debug mode I've found that it is getting stuck in the while loop in
, as the coordinator is never found.

The future returned from
in the loop fails the first time with
, and then will fail every subsequent time with
org.apache.kafka.common.errors.GroupCoordinatorNotAvailableException: The group coordinator is not available.
. Does anyone know why this might happen?

If I use the console producer/consumer I am able to successfully send and receive messages, it is only when I use my implementation of the KafkaConsumer. Additionally, the consumer does work on two of my servers so I know it is not the implementation of the consumer.

Here are the properties my consumer is created with:

Properties props = new Properties();
props.put("bootstrap.servers", "myserver:9000);
props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
props.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
props.put("", groupId);
props.put("", "true");
props.put("", "1000");
props.put("", "30000");


The topic is definitely created before the consumer starts.

Edit 2:
I deleted all of the brokers in my cluster and recreated them, and now I'm failing at a different point. In
while trying to rejoin, the future returned from
repeatedly fails with
org.apache.kafka.common.errors.NotCoordinatorForGroupException: This is not the correct coordinator for this group.
. Still not sure what is going on.

Edit 3:
I deleted the brokers and recreated them with a different id and now the
method is returning and it's successfully consuming messages. I'd still like to know why it failed in the first place though so I can make sure it doesn't happen again.

Answer Source

Deleting the brokers and creating new ones fixed the problem. Still not sure went wrong with the brokers though.