apex-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ananth Gundabattula <agundabatt...@gmail.com>
Subject A few questions about the Kafka 0.9 operator
Date Tue, 14 Jun 2016 19:41:48 GMT
Hello Siyuan/All,

I have a couple of questions regarding the Kafka 0.9 operator. Could you
please help me in understanding this operator a bit better?

   - As stated in
   http://www.slideshare.net/ApacheApex/apache-apex-kafka-input-operator ,
   kafka 0.9 operator stores it "check-pointed offsets" in Kafka itself using
   the App name ? It sounds like -originalAppID is not used by this operator
   at all - In other words, I cant force an app to process starting from the
   beginning until I change the App name if the App is based on the Kafka 0.9
   operator as the input operator
   - How does the kafka 0.9 operator handle downstream operators failure ?
   By this I mean, an Apex downstream operator fails, and is brought back up
   by STRAM. However this operator was significantly lagging behind the
   current window of the kafka 0.9 operator window. Does the buffer server
   within the Kafka 0.9 operator buffer many windows to handle this situation
   ? ( and hence replays accordingly ? ) . I ask this to fine tune the buffer
   memory property.
   - Is EXACTLY_ONCE processing supported in this operator ? if yes, is it
   fair to assume that HDFS would be used to manage this type of configuration
   - Is EXACTLY_ONCE based off the streaming window or the Application
   Window in Apex ?



View raw message