kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael D. Coon" <mdco...@yahoo.com.INVALID>
Subject KStream Close Processor
Date Fri, 08 Apr 2016 19:23:41 GMT
   I'm seeing my processor's "close" method being called AFTER my downstream producer has
been closed. I had assumed that on close I would be able to flush whatever I had been buffering
up to send to kafka topic. In other words, we've seen significant performance differences
in building flows with small messages and large messages in/out of kafka. So my processor
buffers up messages to a threshold and flushes those as a composite message bundle to improve
downstream processing. But if this close method is called AFTER the producer has already been
closed, I would have no way to actually flush the final composite bundles to my topic on shutdown.
Is there some way to get a call BEFORE producer shutdown occurs?

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message