ethrbunny ethrbunny - 1 year ago 118
Java Question

Stop a Kafka KStreams app

Is it possible to have a Kafka KStreams app that runs through all the data in a topic and then exits?

Example Im producing data into topics based on date. The consumer gets kicked off by cron, runs through all the available data and then .. does what? I don't want it to sit and wait for more data. Just assume it's all there and then exit gracefully.


Answer Source

You can create a consumer and then once it stops pulling up data, you can have call consumer.close(). Or if you want to poll again in the future just call consumer.pause() and call .resume later.

One way to do this is within the consumer poll block. Such as

data = consumer.poll()
if (! {

Keep in mind that poll returns ConsumerRecord<K,V> and conforms to the Iterable interface.

Recommended from our users: Dynamic Network Monitoring from WhatsUp Gold from IPSwitch. Free Download