cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From horschi <hors...@gmail.com>
Subject Re: memory usage problem of Metadata.tokenMap.tokenToHost
Date Tue, 22 Sep 2015 08:09:32 GMT
Hi Joseph,

I think 2000 keyspaces might be just too much. Fewer keyspaces (and CFs)
will probably work much better.

kind regards,
Christian


On Tue, Sep 22, 2015 at 9:29 AM, joseph gao <gaojf.bokecc@gmail.com> wrote:

> Hi, anybody could help me?
>
> 2015-09-21 0:47 GMT+08:00 joseph gao <gaojf.bokecc@gmail.com>:
>
>> ps : that's the code in java drive , in MetaData.TokenMap.build:
>>
>> for (KeyspaceMetadata keyspace : keyspaces)
>> {
>>     ReplicationStrategy strategy = keyspace.replicationStrategy();
>>     Map<Token, Set<Host>> ksTokens = (strategy == null)
>>         ? makeNonReplicatedMap(tokenToPrimary)
>>         : strategy.computeTokenToReplicaMap(tokenToPrimary, ring);
>>
>>     tokenToHosts.put(keyspace.getName(), ksTokens);
>>
>> tokenToPrimary is all same, ring is all same, and if strategy is all
>> same , strategy.computeTokenToReplicaMap would return 'same' map but
>> different object( cause every calling returns a new HashMap
>>
>> 2015-09-21 0:22 GMT+08:00 joseph gao <gaojf.bokecc@gmail.com>:
>>
>>> cassandra: 2.1.7
>>> java driver: datastax java driver 2.1.6
>>>
>>> Here is the problem:
>>>    My application uses 2000+ keyspaces, and will dynamically create
>>> keyspaces and tables. And then in java client, the
>>> Metadata.tokenMap.tokenToHost would use about 1g memory. so this will cause
>>> a lot of  full gc.
>>>    As I see, the key of the tokenToHost is keyspace, and the value is a
>>> tokenId_to_replicateNodes map.
>>>
>>>    When I try to solve this problem, I find something not sure: all
>>> keyspaces have same 'tokenId_to_replicateNodes' map.
>>>     My replication strategy of all keyspaces is : simpleStrategy and
>>> replicationFactor is 3
>>>
>>>     So would it be possible if keyspaces use same strategy, the value of
>>> tokenToHost map use a same map. So it would extremely reduce the memory
>>> usage
>>>
>>>      thanks a lot
>>>
>>> --
>>> ------
>>> Joseph Gao
>>> PhoneNum:15210513582
>>> QQ: 409343351
>>>
>>
>>
>>
>> --
>> ------
>> Joseph Gao
>> PhoneNum:15210513582
>> QQ: 409343351
>>
>
>
>
> --
> ------
> Joseph Gao
> PhoneNum:15210513582
> QQ: 409343351
>

Mime
View raw message