ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bosko Devetak <bosko.deve...@booking.com>
Subject Re: [External] Re: Triggering cache updates in Ignite
Date Wed, 28 Jun 2017 11:34:14 GMT
Hi Denis,

On Wed, Jun 28, 2017 at 12:13 AM, Denis Magda <dmagda@gridgain.com> wrote:

Hello Daniel,
>
> That looks promising!
>
> So, in case with Apache Ignite we need to subscribe to Kafka to get all
> the updates while the Kafka will receive them from the replicator, right?
> Is there any way to provide a direct support for Ignite by implementing a
> dedicating replicator?
>

Short answer: Yes, that is relatively easy to achieve by implementing
Applier interface.

A bit of details: Basically replicator pipeline looks like this:

    [binlog parser] -> [schema decorator] -> [applier]

Currently there are three appliers implemented:

1. HBase
2. Kafka
3. STDOUT

They all implement the Applier interface. STDOUT applier is a simple
implementation used for debugging, but it is also a good place to start
when writing a new applier. We are open for pull requests and we can help
with advice.

The Applier Interface:

https://github.com/mysql-time-machine/replicator/blob/master/src/main/java/com/booking/replication/applier/Applier.java

STDOUT Applier Implementation:

https://github.com/mysql-time-machine/replicator/blob/master/src/main/java/com/booking/replication/applier/StdoutJsonApplier.java

When you run the replicator you can specify the applier type in startup
parameters:

java -jar mysql-replicator.jar \
    --applier STDOUT \
    --schema $schema \
    --binlog-filename $binlog-filename \
    --config-path $config-path


> In general, may I ask you to send this message to Ignite dev list or I can
> forward it there copying you? It makes sense to discuss the project with
> the whole community.
>

I have added  dev@ignite.apache.org  to cc.

I am out of the office now, but we can talk more once I am back from 10th
of July.

Cheers,
Bosko.



>
> —
> Denis
>
> On Jun 22, 2017, at 6:08 AM, Daniël van Eeden <daniel.vaneeden@booking.com>
> wrote:
>
> Hello Denis,
>
>
> This is the project I mentioned at the MySQL meetup:
> https://mysql-time-machine.github.io/
>
> This allows you to consume a MySQL binlog stream, which then can be used
> to update the cache entries in Ignite when they are created, updated or
> removed in MySQL directly.
> This is opensource and *much* cheaper than Golden Gate.
>
>
> Cheers,
>
> --
> Daniël van Eeden
> Database Administrator
>
> Booking.com B.V.
> Vijzelstraat 66-80 Amsterdam 1017HL Netherlands
> Direct +31207033812 <+31%2020%20703%203812>
> [image: Booking.com] <http://www.booking.com/>
> The world's #1 accommodation site
> 43 languages, 187+ offices worldwide, 96,000+ global destinations,
> 1,200,000+ room nights booked every day
> No booking fees, best price always guaranteed
> Subsidiary of the Priceline Group (NASDAQ: PCLN)
>
>
>


-- 
Bosko Devetak
Senior Developer

Booking.com B.V.
Vijzelstraat 66-80 Amsterdam 1017HL Netherlands
Direct +31207158399
[image: Booking.com] <http://www.booking.com/>
The world's #1 accommodation site
43 languages, 199+ offices worldwide, 96,000+ global destinations,
1,400,000+ room nights booked every day
No booking fees, best price always guaranteed
Subsidiary of the Priceline Group (NASDAQ: PCLN)

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message