incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From vineet daniel <vineetdan...@gmail.com>
Subject Re: 0.7 memory usage problem
Date Sat, 18 Sep 2010 16:36:33 GMT
Hi
Even I would like to add here something and correct me if I am wrong, I
downloaded 0.7 beta and ran it, just by chance I checked 'top' to see how
the new version is doing and there were 64 processes running though
Cassandra was on single node with default configuration options ( ran it as
is, as soon as I downloaded). No inserts done, no selects done nothing. I
don't think this is normal.

In anticipation of feedbacks on this.

Regards
Vineet Daniel
Cell          : +918106217121
Websites :
Blog <http://vinetedaniel.blogspot.com>   |
Linkedin<http://in.linkedin.com/in/vineetdaniel>
|  Twitter <https://twitter.com/vineetdaniel>





On Sat, Sep 18, 2010 at 7:34 PM, Peter Schuller <peter.schuller@infidyne.com
> wrote:

> > I see a spike in heap memory usage on Node 2 where it goes from around 1G
> to
> > 6GB (max) in less than an hour, and then goes our of memory.
> > There are some errors in the log file that are reported by other people,
> but
> > I don't think that these errors are the reason, because it use to happen
> > even before I have seen them.
> >
> > Can someone help me understand whats going on??
>
> Only partially:
>
> 54 seconds ParNew GC:s are wild and crazy:
>
>   INFO [GC inspection] 2010-09-17 14:53:59,403 GCInspector.java (line
> 129) GC for ParNew: 54095 ms, 53297952 reclaimed leaving 4712568360
> used; max is 6563430400
>
> Is the machine swapping?
>
> I noticed there is hinted hand-off activity going on. Maybe that is a
> result of nodes dropping in and out due to swapping. In any case, you
> definitely don't want to have the machine swapping to death. I'm not
> sure what the best way is to avoid this on Windows, other than
> decreasing heap size.
>
> The repeated exceptions in your log aren't normal as far as I know. I
> don't remember, but IIRC the UTF-8 encoding issues can be caused by
> changing partitioner after inserting data (but I'm not sure).
>
> With respect to memory use, you don't seem to be inserting so much
> data for bloom filters and sstable index samples to be a problem.
> Memtable flushing could cause problems if they're happening too slowly
> (maybe plausible with swapping) - except that the stage statistics
> don't indicate there are lots of memtables in memory waiting to be
> flushed, so that shouldn't be it.
>
> Hinted handoff maybe, but I don't remember whether hinted handoff has
> the potential to accumulate data in RAM (no time to check now).
>
> Regardless, I'd recommend fixing any swapping issues you have before
> trying to draw conclusions about performance. And you don't want those
> exceptions in your logs.
>
> --
> / Peter Schuller
>

Mime
View raw message