ignite-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ilya.kasnacheev" <ilya.kasnach...@gmail.com>
Subject Re: Ignite cache partition size
Date Mon, 04 Dec 2017 16:28:50 GMT
Hello!

* 8192 partitions have been tried with success, as far as I know. I think
you can give it a shot, see where it gets you. It might slow down partition
exchange a bit, but it should not hang just from increased amount of
information (as opposed to stuck operations)

* Yes,they need to have the same number of partitions for caches with
collocated keys. Come to think of it, if all of 2N partitions were
collocated with 1N partitions, what you will really have is just 1N
partitions. Partitions should be able to part in order to work.

For optimal number of partitions, consider 256 primary partitions per node.
This means 8192 is OK for you until you move past 32 nodes.

Some additional hints:
* Increasing number of partitions may increase number of open files if using
persistence. Plan accordingly, keep ulimits in mind.
* You can broadcast ignite.affinity(cacheName).primaryPartitions(localNode)
and ignite.cache(cacheName).localSize(CachePeekMode.PRIMARY) to data nodes
to get hold of how even partition distribution is in your case (are keys
distributed well between partitions? are partitions between nodes?
* There was a research on how well partitions are distributed between nodes
with Rendevouz affinity, see non-"balanced" PNGs (that's per # of nodes) in
https://issues.apache.org/jira/browse/IGNITE-3018 - I think you can observe
serious deviations from average number of partitions in those images, so
it's possible that your 2x difference is no accident.
* The community is currently working on faster and fairer affinity functions
:)

Regards,



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Mime
View raw message