ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Valentin Kulichenko <valentin.kuliche...@gmail.com>
Subject Continuous queries changes
Date Thu, 23 Jul 2015 05:53:45 GMT
Igniters,

Based on discussions with our users I came to conclusion that our
continuous query implementation is not good enough for use cases with
strong consistency requirements, because there is a possibility to lose
updates in case of topology change.

So I started working on https://issues.apache.org/jira/browse/IGNITE-426
and I hope to finish in in couple of weeks so that we can include it in
next release.

I have the following design in mind:

   - Maintain updates queue on backup node(s) in addition to primary node.
   - If primary node crushes, this queue is flushed to listening clients.
   - To avoid notification duplicates we will have a per-partition update
   counter. Once an entry in some partition is updated, counter for this
   partition is incremented on both primary and backups. The value of this
   counter is also sent along with the update to the client, which also
   maintains the copy of this mapping. If at some moment it receives an update
   with the counter less than in its local map, this update is a duplicate and
   can be discarded.
   - Also need to figure out the best way to clean the backup queue if
   topology is stable. Will be happy to hear any suggestions :)

To make all this work we also need to implement thread-per-partition mode
in atomic cache, because now updates order on backup nodes can differ from
the primary node: https://issues.apache.org/jira/browse/IGNITE-104. I'm
already working on this.

Feel free to share your thoughts!

-Val

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message