cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From joseph gao <gaojf.bok...@gmail.com>
Subject Re: memory usage problem of Metadata.tokenMap.tokenToHost
Date Tue, 15 Dec 2015 02:02:55 GMT
That's great. Thanks, Alex!

2015-10-02 0:36 GMT+08:00 Alexandre Dutra <alexandre.dutra@datastax.com>:

> I confirm that the optional schema metadata feature (JAVA-828
> <https://datastax-oss.atlassian.net/browse/JAVA-828>) has been merged
> upstream and is included in driver 3.0.0-alpha3 and will as well be
> included in the forthcoming 2.1.8 version.
>
> On a side note, JAVA-664
> <https://datastax-oss.atlassian.net/browse/JAVA-664> has been open for
> quite a while now and aims to fix the problem you describe. It will
> hopefully make it into 2.1.8 as well, although I cannot guarantee that at
> this point.
>
> Thanks,
>
> Alexandre Dutra
>
> On Mon, Sep 28, 2015 at 7:55 PM Alex Popescu <alexp@datastax.com> wrote:
>
>> Besides the others' advice that 2000+ keyspaces might be too much, the
>> latest Java driver (2.0.11) includes an option to disable the Metadata API
>> http://www.datastax.com/dev/blog/datastax-java-driver-2-0-11-released.
>> I'm not sure at this moment if this has been merged into 2.1 already.
>>
>> On Sun, Sep 20, 2015 at 9:22 AM, joseph gao <gaojf.bokecc@gmail.com>
>> wrote:
>>
>>> cassandra: 2.1.7
>>> java driver: datastax java driver 2.1.6
>>>
>>> Here is the problem:
>>>    My application uses 2000+ keyspaces, and will dynamically create
>>> keyspaces and tables. And then in java client, the
>>> Metadata.tokenMap.tokenToHost would use about 1g memory. so this will cause
>>> a lot of  full gc.
>>>    As I see, the key of the tokenToHost is keyspace, and the value is a
>>> tokenId_to_replicateNodes map.
>>>
>>>    When I try to solve this problem, I find something not sure: all
>>> keyspaces have same 'tokenId_to_replicateNodes' map.
>>>     My replication strategy of all keyspaces is : simpleStrategy and
>>> replicationFactor is 3
>>>
>>>     So would it be possible if keyspaces use same strategy, the value of
>>> tokenToHost map use a same map. So it would extremely reduce the memory
>>> usage
>>>
>>>      thanks a lot
>>>
>>> --
>>> ------
>>> Joseph Gao
>>> PhoneNum:15210513582
>>> QQ: 409343351
>>>
>>
>>
>>
>> --
>> Bests,
>>
>> Alex Popescu | @al3xandru
>> Sen. Product Manager @ DataStax
>>
>>
>>
>> <http://cassandrasummit-datastax.com/?utm_campaign=summit15&utm_medium=summiticon&utm_source=emailsignature>
>>
> --
> Alexandre Dutra
> Driver & Tools Engineer @ DataStax
>



-- 
------
Joseph Gao
PhoneNum:15210513582
QQ: 409343351

Mime
View raw message