flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From M Singh <mans2si...@yahoo.com>
Subject Flink - Once and once processing
Date Fri, 29 Jul 2016 17:26:02 GMT
I have a use case where we need to update a counter in a db and for this need to guarantee
once only processing.  If we have some entries in a batch and it partially updates the counters
and then fails, if Flink retries the processing for that batch, some of the counters will
be updated twice (the ones which succeeded in the first batch).
I think in order to guarantee once only processing, I will have to set the buffer size to
zero (ie, send one item at a time). 

Is there any alternative configuration or suggestion on how I can achieve once only updates
using a batch mode with partial failures ?

View raw message