Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
MIME-Version: 1.0
In-Reply-To: 
 <CAFYKJoo4CPxEV_+YeirzE+u+xVCBPJBDMS8OySLy82ejRtwByg@mail.gmail.com>
References: 
 <CAFYKJoo4CPxEV_+YeirzE+u+xVCBPJBDMS8OySLy82ejRtwByg@mail.gmail.com>
From: Alex Popescu <alexp@datastax.com>
Date: Mon, 28 Sep 2015 10:54:45 -0700
Message-ID: 
 <CAJ8Xi+R_0u_fA_oz4CCTRJ+T2uoCytboGGPuux7LbZvXxX=fUg@mail.gmail.com>
Subject: Re: memory usage problem of Metadata.tokenMap.tokenToHost
To: user <user@cassandra.apache.org>
Content-Type: multipart/alternative; boundary=001a11413648afe3f80520d26513

--001a11413648afe3f80520d26513
Content-Type: text/plain; charset=UTF-8

Besides the others' advice that 2000+ keyspaces might be too much, the
latest Java driver (2.0.11) includes an option to disable the Metadata API
http://www.datastax.com/dev/blog/datastax-java-driver-2-0-11-released. I'm
not sure at this moment if this has been merged into 2.1 already.

On Sun, Sep 20, 2015 at 9:22 AM, joseph gao <gaojf.bokecc@gmail.com> wrote:

> cassandra: 2.1.7
> java driver: datastax java driver 2.1.6
>
> Here is the problem:
>    My application uses 2000+ keyspaces, and will dynamically create
> keyspaces and tables. And then in java client, the
> Metadata.tokenMap.tokenToHost would use about 1g memory. so this will cause
> a lot of  full gc.
>    As I see, the key of the tokenToHost is keyspace, and the value is a
> tokenId_to_replicateNodes map.
>
>    When I try to solve this problem, I find something not sure: all
> keyspaces have same 'tokenId_to_replicateNodes' map.
>     My replication strategy of all keyspaces is : simpleStrategy and
> replicationFactor is 3
>
>     So would it be possible if keyspaces use same strategy, the value of
> tokenToHost map use a same map. So it would extremely reduce the memory
> usage
>
>      thanks a lot
>
> --
> ------
> Joseph Gao
> PhoneNum:15210513582
> QQ: 409343351
>


-- 
Bests,

Alex Popescu | @al3xandru
Sen. Product Manager @ DataStax


<http://cassandrasummit-datastax.com/?utm_campaign=summit15&utm_medium=summiticon&utm_source=emailsignature>

--001a11413648afe3f80520d26513
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Besides the others&#39; advice that 2000+ keyspaces might =
be too much, the latest Java driver (2.0.11) includes an option to disable =
the Metadata API=C2=A0<a href=3D"http://www.datastax.com/dev/blog/datastax-=
java-driver-2-0-11-released">http://www.datastax.com/dev/blog/datastax-java=
-driver-2-0-11-released</a>. I&#39;m not sure at this moment if this has be=
en merged into 2.1 already.</div><div class=3D"gmail_extra"><br><div class=
=3D"gmail_quote">On Sun, Sep 20, 2015 at 9:22 AM, joseph gao <span dir=3D"l=
tr">&lt;<a href=3D"mailto:gaojf.bokecc@gmail.com" target=3D"_blank">gaojf.b=
okecc@gmail.com</a>&gt;</span> wrote:<br><blockquote class=3D"gmail_quote" =
style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><di=
v dir=3D"ltr">cassandra: 2.1.7<div>java driver: datastax java driver 2.1.6<=
/div><div><br></div><div>Here is the problem:</div><div>=C2=A0 =C2=A0My app=
lication uses 2000+ keyspaces, and will dynamically create keyspaces and ta=
bles. And then in java client, the Metadata.tokenMap.tokenToHost would use =
about 1g memory. so this will cause a lot of =C2=A0full gc.</div><div>=C2=
=A0 =C2=A0As I see, the key of the tokenToHost is keyspace, and the value i=
s a tokenId_to_replicateNodes map.</div><div><br></div><div>=C2=A0 =C2=A0Wh=
en I try to solve this problem, I find something not sure: all keyspaces ha=
ve same &#39;tokenId_to_replicateNodes&#39; map.</div><div>=C2=A0 =C2=A0 My=
 replication strategy of all keyspaces is : simpleStrategy and replicationF=
actor is 3</div><div><br></div><div>=C2=A0 =C2=A0 So would it be possible i=
f keyspaces use same strategy, the value of tokenToHost map use a same map.=
 So it would extremely reduce the memory usage</div><div><br></div><div>=C2=
=A0 =C2=A0 =C2=A0thanks a lot<span class=3D"HOEnZb"><font color=3D"#888888"=
><br clear=3D"all"><div><br></div>-- <br><div>------<div>Joseph Gao</div><d=
iv>PhoneNum:<span style=3D"font-family:arial,sans-serif;font-size:13px;bord=
er-collapse:collapse">15210513582</span></div><div><font face=3D"arial, san=
s-serif"><span style=3D"border-collapse:collapse">QQ: 409343351</span></fon=
t></div></div>
</font></span></div></div>
</blockquote></div><br><br clear=3D"all"><div><br></div>-- <br><div class=
=3D"gmail_signature"><div dir=3D"ltr"><div><div dir=3D"ltr"><div><div dir=
=3D"ltr"><div><div dir=3D"ltr"><div><div>Bests,</div><div><br>Alex Popescu =
|=C2=A0<span style=3D"line-height:1.6em">@al3xandru</span></div><div dir=3D=
"ltr">Sen. Product Manager @ DataStax<br><br></div></div><div dir=3D"ltr"><=
br></div><div dir=3D"ltr"><a href=3D"http://cassandrasummit-datastax.com/?u=
tm_campaign=3Dsummit15&amp;utm_medium=3Dsummiticon&amp;utm_source=3Demailsi=
gnature" target=3D"_blank"><img src=3D"http://datastax.com/images/Summit_Em=
ail.png" width=3D"420" height=3D"58"></a><span></span><span></span><a href=
=3D"http:///" target=3D"_blank"></a><br></div></div></div></div></div></div=
></div></div></div>
</div>

--001a11413648afe3f80520d26513--