ignite-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Semen Boikov (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (IGNITE-4154) Optimize amount of data stored in discovery history
Date Fri, 11 Nov 2016 12:48:58 GMT

    [ https://issues.apache.org/jira/browse/IGNITE-4154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15656992#comment-15656992
] 

Semen Boikov edited comment on IGNITE-4154 at 11/11/16 12:48 PM:
-----------------------------------------------------------------

Done these optimizations as part of this issue:

Discovery history optimizations:
- remove discarded message for discovery pending messages
- remove duplicated data from TcpDiscoveryNodeAddedMessage.oldNodesDiscoData
- do not store unnecessary data in discovery EnsuredMessageHistory
- use special property for EnsuredMessageHistory size instead of IGNITE_DISCOVERY_HISTORY_SIZE

Affinity history optimizations:
- do not store calculated primary/backup maps in history
- try save the same assignments instance for caches with similar affinity

Exchange messages optimizations:
- do not send duplicated partition state maps for caches with similar affinity
- use zip compression for data sent in exchange messages



was (Author: sboikov):
Done these optimizations as part of this issue:

Discovery history optimizations:
- remove discarded message for discovery pending messages
- remove duplicated data from TcpDiscoveryNodeAddedMessage.oldNodesDiscoData
- do not store unnecessary data in discovery EnsuredMessageHistory
- use special property for EnsuredMessageHistory size instead of IGNITE_DISCOVERY_HISTORY_SIZE
Affinity history optimizations:
- do not store calculated primary/backup maps in history
- try save the same assignments instance for caches with similar affinity
Exchange messages optimizations:
- do not send duplicated partition state maps for caches with similar affinity
- use zip compression for data sent in exchange messages


> Optimize amount of data stored in discovery history
> ---------------------------------------------------
>
>                 Key: IGNITE-4154
>                 URL: https://issues.apache.org/jira/browse/IGNITE-4154
>             Project: Ignite
>          Issue Type: Task
>          Components: general
>            Reporter: Semen Boikov
>            Assignee: Semen Boikov
>             Fix For: 1.8
>
>
> Currently on large clusters with lots of caches amount of data stored in tcp discovery
history can grow too much.
> What can be improved:
> - TcpDiscoveryNodeAddedMessage.oldNodesDiscoData contains per-node discovery data. For
cache component essentially this is list of configured caches, and for joined node at the
time when 'collectDiscoveryData' is called this list should be the same on all nodes, so it
can be stored once instead of per-node map
> - even after message is discarded it is still stored in PendingMessages collection, looks
like it can be safely removed after discarding
> - now system property IGNITE_DISCOVERY_HISTORY_SIZE controls both size of events needed
for client reconnects and number of topology snapshots stored in GridDiscoveryManager, need
have two properies
> - for clients TcpDiscoveryNodeAddFinishedMessage.clientDiscoData can be initialized only
when message is sent to joining client 



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message