cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Edward Capriolo <edlinuxg...@gmail.com>
Subject Re: Bizarre side-effect of increasing read concurrency
Date Sat, 02 Apr 2011 03:49:29 GMT
On Fri, Apr 1, 2011 at 11:27 PM, Jason Harvey <alienth@gmail.com> wrote:
> On further analysis, it looks like this behavior occurs when a node is
> simply restarted. Is that normal behavior? If mark-and-sweep becomes
> less and less effective over time, does that suggest an issue with GC,
> or an issue with memory use?
>
> On Apr 1, 8:21 pm, Jason Harvey <alie...@gmail.com> wrote:
>> After increasing read concurrency from 8 to 64, GC mark-and-sweep was
>> suddenly able to reclaim much more memory than it previously did.
>>
>> Previously, mark-and-sweep would run around 5.5GB, and would cut heap
>> usage to 4GB. Now, it still runs at 5.5GB, but it shrinks all the way
>> down to 2GB used. This behavior was consistent in every machine I
>> increased read-concurrent on.
>>
>> Any thoughts on why this behavior changed? No other diagnostics
>> appeared to correlate to the concurrency change, besides thread count.
>

Jason,

First you do not need to restart to adjust concurrent readers. It can
be done from JMX without restart.

As for the memory, after you restart you may have drained your caches
and memtables which explains why less memory is used.

Java also enjoys using all the memory your allocate and the Garbage
collection does not give it back unless it needs to.

Edward

Mime
View raw message