apex-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Raja.Aravapalli <Raja.Aravapa...@target.com>
Subject kafka input is processing records in a jumbled order
Date Tue, 07 Jun 2016 22:34:53 GMT


I have built a DAG, that reads from kafka and in the next operators, does lookup to a hbase
table and update hbase table based on some business logic.

Some times my operator which does hbase lookup and update in the same operator(Custom written),
is processing the records it receives from kafka in a jumbled order, which is causing, many
records being ignored from processing!!

I am not using any parallel partitions/instance, and with KafkaInputOperator I am using only
partition strategy ONE_TO_MANY.

I am very new to Apex. I expected, Apex will guarantee the ordering.

Can someone pls share your knowledge on the issue…?

Thanks a lot in advance…

View raw message