cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From joseph gao <gaojf.bok...@gmail.com>
Subject Re: memory usage problem of Metadata.tokenMap.tokenToHost
Date Mon, 04 Jan 2016 03:57:39 GMT
By the way, when would be the version 2.1.10 published?

2015-12-15 10:02 GMT+08:00 joseph gao <gaojf.bokecc@gmail.com>:

> That's great. Thanks, Alex!
>
> 2015-10-02 0:36 GMT+08:00 Alexandre Dutra <alexandre.dutra@datastax.com>:
>
>> I confirm that the optional schema metadata feature (JAVA-828
>> <https://datastax-oss.atlassian.net/browse/JAVA-828>) has been merged
>> upstream and is included in driver 3.0.0-alpha3 and will as well be
>> included in the forthcoming 2.1.8 version.
>>
>> On a side note, JAVA-664
>> <https://datastax-oss.atlassian.net/browse/JAVA-664> has been open for
>> quite a while now and aims to fix the problem you describe. It will
>> hopefully make it into 2.1.8 as well, although I cannot guarantee that at
>> this point.
>>
>> Thanks,
>>
>> Alexandre Dutra
>>
>> On Mon, Sep 28, 2015 at 7:55 PM Alex Popescu <alexp@datastax.com> wrote:
>>
>>> Besides the others' advice that 2000+ keyspaces might be too much, the
>>> latest Java driver (2.0.11) includes an option to disable the Metadata API
>>> http://www.datastax.com/dev/blog/datastax-java-driver-2-0-11-released.
>>> I'm not sure at this moment if this has been merged into 2.1 already.
>>>
>>> On Sun, Sep 20, 2015 at 9:22 AM, joseph gao <gaojf.bokecc@gmail.com>
>>> wrote:
>>>
>>>> cassandra: 2.1.7
>>>> java driver: datastax java driver 2.1.6
>>>>
>>>> Here is the problem:
>>>>    My application uses 2000+ keyspaces, and will dynamically create
>>>> keyspaces and tables. And then in java client, the
>>>> Metadata.tokenMap.tokenToHost would use about 1g memory. so this will cause
>>>> a lot of  full gc.
>>>>    As I see, the key of the tokenToHost is keyspace, and the value is a
>>>> tokenId_to_replicateNodes map.
>>>>
>>>>    When I try to solve this problem, I find something not sure: all
>>>> keyspaces have same 'tokenId_to_replicateNodes' map.
>>>>     My replication strategy of all keyspaces is : simpleStrategy and
>>>> replicationFactor is 3
>>>>
>>>>     So would it be possible if keyspaces use same strategy, the value
>>>> of tokenToHost map use a same map. So it would extremely reduce the memory
>>>> usage
>>>>
>>>>      thanks a lot
>>>>
>>>> --
>>>> ------
>>>> Joseph Gao
>>>> PhoneNum:15210513582
>>>> QQ: 409343351
>>>>
>>>
>>>
>>>
>>> --
>>> Bests,
>>>
>>> Alex Popescu | @al3xandru
>>> Sen. Product Manager @ DataStax
>>>
>>>
>>>
>>> <http://cassandrasummit-datastax.com/?utm_campaign=summit15&utm_medium=summiticon&utm_source=emailsignature>
>>>
>> --
>> Alexandre Dutra
>> Driver & Tools Engineer @ DataStax
>>
>
>
>
> --
> ------
> Joseph Gao
> PhoneNum:15210513582
> QQ: 409343351
>



-- 
------
Joseph Gao
PhoneNum:15210513582
QQ: 409343351

Mime
View raw message