JosephGao created CASSANDRA-10402:
-------------------------------------
Summary: reduce the memory usage of Metadata.tokenMap.tokenToHost
Key: CASSANDRA-10402
URL: https://issues.apache.org/jira/browse/CASSANDRA-10402
Project: Cassandra
Issue Type: Improvement
Components: Drivers (now out of tree)
Reporter: JosephGao
Fix For: 2.1.x
My application uses 2000+ keyspaces, and will dynamically create keyspaces and tables. And
then in java client, the Metadata.tokenMap.tokenToHost would use about 1g memory. so this
will cause a lot of full gc.
As I see, the key of the tokenToHost is keyspace, and the value is a tokenId_to_replicateNodes
map.
When I try to solve this problem, I find something not sure: all keyspaces have same 'tokenId_to_replicateNodes'
map.
My replication strategy of all keyspaces is : simpleStrategy and replicationFactor is
3
So would it be possible if keyspaces use same strategy, the value of tokenToHost map use
a same map. So it would extremely reduce the memory usage
ps： the following is effected code:
for (KeyspaceMetadata keyspace : keyspaces)
{
ReplicationStrategy strategy = keyspace.replicationStrategy();
Map<Token, Set<Host>> ksTokens = (strategy == null)
? makeNonReplicatedMap(tokenToPrimary)
: strategy.computeTokenToReplicaMap(tokenToPrimary, ring);
tokenToHosts.put(keyspace.getName(), ksTokens);
}
tokenToPrimary is all same, ring is all same, and if strategy is all same , strategy.computeTokenToReplicaMap
would return 'same' map but different object( cause every calling returns a new HashMap)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
|