cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Harvey <alie...@gmail.com>
Subject Re: Bizarre side-effect of increasing read concurrency
Date Sat, 02 Apr 2011 05:45:24 GMT
Ah, that would probably explain it. Thanks!

On Apr 1, 8:49 pm, Edward Capriolo <edlinuxg...@gmail.com> wrote:
> On Fri, Apr 1, 2011 at 11:27 PM, Jason Harvey <alie...@gmail.com> wrote:
> > On further analysis, it looks like this behavior occurs when a node is
> > simply restarted. Is that normal behavior? If mark-and-sweep becomes
> > less and less effective over time, does that suggest an issue with GC,
> > or an issue with memory use?
>
> > On Apr 1, 8:21 pm, Jason Harvey <alie...@gmail.com> wrote:
> >> After increasing read concurrency from 8 to 64, GC mark-and-sweep was
> >> suddenly able to reclaim much more memory than it previously did.
>
> >> Previously, mark-and-sweep would run around 5.5GB, and would cut heap
> >> usage to 4GB. Now, it still runs at 5.5GB, but it shrinks all the way
> >> down to 2GB used. This behavior was consistent in every machine I
> >> increased read-concurrent on.
>
> >> Any thoughts on why this behavior changed? No other diagnostics
> >> appeared to correlate to the concurrency change, besides thread count.
>
> Jason,
>
> First you do not need to restart to adjust concurrent readers. It can
> be done from JMX without restart.
>
> As for the memory, after you restart you may have drained your caches
> and memtables which explains why less memory is used.
>
> Java also enjoys using all the memory your allocate and the Garbage
> collection does not give it back unless it needs to.
>
> Edward

Mime
View raw message