cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "JosephGao (JIRA)" <>
Subject [jira] [Created] (CASSANDRA-10402) reduce the memory usage of Metadata.tokenMap.tokenToHost
Date Mon, 28 Sep 2015 02:20:04 GMT
JosephGao created CASSANDRA-10402:

             Summary: reduce the memory usage of Metadata.tokenMap.tokenToHost
                 Key: CASSANDRA-10402
             Project: Cassandra
          Issue Type: Improvement
          Components: Drivers (now out of tree)
            Reporter: JosephGao
             Fix For: 2.1.x

My application uses 2000+ keyspaces, and will dynamically create keyspaces and tables. And
then in java client, the Metadata.tokenMap.tokenToHost would use about 1g memory. so this
will cause a lot of  full gc.
   As I see, the key of the tokenToHost is keyspace, and the value is a tokenId_to_replicateNodes

   When I try to solve this problem, I find something not sure: all keyspaces have same 'tokenId_to_replicateNodes'
    My replication strategy of all keyspaces is : simpleStrategy and replicationFactor is

    So would it be possible if keyspaces use same strategy, the value of tokenToHost map use
a same map. So it would extremely reduce the memory usage

     ps: the following is effected code:
for (KeyspaceMetadata keyspace : keyspaces)
    ReplicationStrategy strategy = keyspace.replicationStrategy();
    Map<Token, Set<Host>> ksTokens = (strategy == null)
        ? makeNonReplicatedMap(tokenToPrimary)
        : strategy.computeTokenToReplicaMap(tokenToPrimary, ring);

    tokenToHosts.put(keyspace.getName(), ksTokens);

tokenToPrimary is all same, ring is all same, and if strategy is all same , strategy.computeTokenToReplicaMap
would return 'same' map but different object( cause every calling returns a new HashMap)

This message was sent by Atlassian JIRA

View raw message