ethrbunny ethrbunny - 3 months ago 30
Java Question

Stop a Kafka Streams app

Is it possible to have a Kafka Streams app that runs through all the data in a topic and then exits?

Example I'm producing data into topics based on date. The consumer gets kicked off by cron, runs through all the available data and then .. does what? I don't want it to sit and wait for more data. Just assume it's all there and then exit gracefully.

Possible?

Answer

You can create a consumer and then once it stops pulling up data, you can have call consumer.close(). Or if you want to poll again in the future just call consumer.pause() and call .resume later.

One way to do this is within the consumer poll block. Such as

data = consumer.poll()
if (!data.next()) {
   consumer.close()
}

Keep in mind that poll returns ConsumerRecord<K,V> and conforms to the Iterable interface.