ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Johannes Lichtenberger <johannes.lichtenber...@unitedplanet.com>
Subject Re: Ignite Partitioned Cache / Use for an in-memory transaction-log
Date Wed, 07 Aug 2019 10:29:38 GMT

if I want to replicate data from one node to N-nodes (that is only a 
subset of nodes in the whole cluster), could I also specify that not 
only the primary is updated in synchronous mode, but some of the backup 
nodes, too?

I also would like to somehow guarantee read-your-own-writes semantics. 
My idea is for replicating a storage / database system to have a single 
writer through which all writes are propagated.

I'd probably use something like oldest node is the master (which can be 
done using Apache Ignite).

Then let's say someone decides to create a database and a first 
resource. I'd have to send an event round robin to create the resource 
on let's say three of 10 cluster nodes. Maybe I could use a distributed 
semaphore which just accepts 3-times acquiring the semaphore (name is 
database + resource concatenated). Once the initial bootstrapping of 
creating directories and files happened on all 3 nodes a 
write-transaction can be opened, which writes into a distributed cache. 
However as I'm thinking about it the cache for this resource must be 
shared on only the same 3 cluster-nodes. That's maybe already where I 
have no clue how to achieve this.

Maybe I'm looking into how other data stores are replicated and 
partitioned, first ;)

There's also BookKeeper for distributing a transaction-log. But I'm not 
sure if it's overhead if I don't have to persist the changes before 
applying it to the real data-files (to provide consistency -- I don't 
need persisted write-ahead logging).

kind regards


On 06.08.19 15:20, Ilya Kasnacheev wrote:
> Hello!
> You can use ignite.affinity(cacheName).mapKeyToNode(key): it returns 
> ClusterNode which is primary for this key.
> Not sure that I understand you about the quorum.
> Regards,
> -- 
> Ilya Kasnacheev
> вт, 6 авг. 2019 г. в 15:47, Johannes Lichtenberger 
> <johannes.lichtenberger@unitedplanet.com 
> <mailto:johannes.lichtenberger@unitedplanet.com>>:
>     Hi,
>     can I somehow query on which cluster-nodes a partitioned cache stores
>     values for specific keys? I might want to use a cache for
>     replicating an
>     in-memory transaction log (which does not have to be persisted) to
>     replicate a document of a NoSQL document store to a few nodes.
>     Thus, instead of a local cache I'd simply write into an ignite
>     cache and
>     then would like to query which cluster-nodes have stored a
>     specific key
>     (for instance the document name). So for instance I could tell a load
>     balancer for reads to read documents from one of the backup replicas.
>     During a transaction commit I would also need this information to
>     know
>     where to send an event to... to commit the transaction. And
>     somehow I'd
>     love to wait for a quorum of nodes if the transaction really has
>     committed or it needs to roll back.
>     kind regards
>     Johannes

View raw message