ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alexey Goncharuk <alexey.goncha...@gmail.com>
Subject Re: listening for events on transactions
Date Wed, 01 Jun 2016 22:13:35 GMT
David,

Have you considered using continuous queries for your use-case [1]? Even if
there were such a thing as a transaction event, I do not see how you can
reliably (read - in a proper order) publish this information to Kafka.

Say, you have 2 clients and 2 server nodes. First client executes a
transaction on keys (1, 2) and second client executes a transaction on keys
(2, 3). These events would have been fired on client nodes because only
client knows when the whole transaction is committed on all nodes. Now you
get two unordered events (1, 2) and (2, 3) which gives you an undefined
state for key (2), or you need to add some sort of custom logic to order
these updates.

The solution would be to listen for updates on primary nodes, which is
exactly what continuous queries do. They guarantee that:

   - Updates for each key will be received in a proper order
   - Events are fault-tolerant, meaning that even if primary node crashed,
   you would still receive an update from backup node


[1] https://apacheignite.readme.io/docs/continuous-queries

2016-06-01 13:16 GMT-07:00 Denis Magda <dmagda@gridgain.com>:

> Hi David,
>
> I would implement this using custom events in the following way:
>
> - introduce your custom event - TransactionEvent and send it every time to
> subscribers a transaction succeeds using IgniteEvents.recordLocal(…)
> method. The event can contain any info you need.
>
> - the separate process can subscribe for your custom TransactionEvents,
> process them and publish to Kafka.
>
> This will let you to decouple Ignite logic from Kafka logic.
>
> Below is a code snippet that demonstrates how to use custom events in
> general:
> https://gist.github.com/dmagda/f1b93a9aa4774a360465bc9f63cc58c9
>
> Will this work for you?
>
> —
> Denis
>
> On Jun 1, 2016, at 9:09 PM, David Robinson <drobin1437@gmail.com> wrote:
>
> I can - but apparently not with a separate process that gets notifications
> - which is the design point I was seeking.
> Thanks again.
>
> On Wed, Jun 1, 2016 at 12:55 PM, Alexei Scherbakov <
> alexey.scherbakoff@gmail.com> wrote:
>
>> You can easily implement such a thing as a part of transaction logic.
>>
>>
>> 2016-06-01 14:52 GMT+03:00 limabean <drobin1437@gmail.com>:
>>
>>> Hi Alexi,
>>>
>>> Thank you for the clarification.
>>>
>>> My final goal is to notify other processes not related to the grid
>>> application about changes to the data caches.  For example, it would be
>>> nice
>>> to have a Kafka publisher registered as a transaction listener and then
>>> when
>>> it gets transaction events, publish this information to a Kafka topic.
>>>
>>> Ignite is good at pulling data in, but it needs to be equally good at
>>> sharing data to work well in my environment.
>>>
>>> What do you think ?
>>>
>>>
>>>
>>>
>>>
>>> --
>>> View this message in context:
>>> http://apache-ignite-users.70518.x6.nabble.com/listening-for-events-on-transactions-tp5346p5357.html
>>> Sent from the Apache Ignite Users mailing list archive at Nabble.com
>>> <http://nabble.com>.
>>>
>>
>>
>>
>> --
>>
>> Best regards,
>> Alexei Scherbakov
>>
>
>
>

Mime
View raw message