cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Victor Kabdebon <victor.kabde...@gmail.com>
Subject Re: Cassandra memory consumption
Date Wed, 09 Feb 2011 03:30:38 GMT
Yes I have, but I have to add that this is a server where there is so little
data (2.0 Mo of text, rougly a book) that even if there were an overhead due
to those things it would be minimal.
I don't understand what's eating up all that memory, is it because of Linux
that has difficulty getting rid of used memory ... I really am puzzled. (by
the way it is not a Amazon EC2 server this is a dedicated server).

Regards,
Victor K.

2011/2/8 Edward Capriolo <edlinuxguru@gmail.com>

> On Tue, Feb 8, 2011 at 4:56 PM, Victor Kabdebon
> <victor.kabdebon@gmail.com> wrote:
> > I will do that in the future and I will post my results here ( I upgraded
> > the server to debian 6 to see if there is any change, so memory is back
> to
> > normal). I will report in a few days.
> > In the meantime I am open to any suggestion...
> >
> > 2011/2/8 Aaron Morton <aaron@thelastpickle.com>
> >>
> >> When you attach to the JVM with JConsole how much non heap memory and
> how
> >> much heap memory is reported on the memory tab?
> >> Xmx controls the total size of the heap memory, which excludes the
> >> permanent generation.
> >> see
> >>
> >>
> http://www.oracle.com/technetwork/java/javase/gc-tuning-6-140523.html#generation_sizing
> >> and
> >>
> >>
> http://blogs.suncom/jonthecollector/entry/presenting_the_permanent_generation
> >> Total non-heap memory on a 0.7 box I have is around 27M. You numbers
> seem
> >> large but it would be interesting to know what the JVM is reporting.
> >> Aaron
> >> On 09 Feb, 2011,at 05:57 AM, Victor Kabdebon <victor.kabdebon@gmail.com
> >
> >> wrote:
> >>
> >> Information on the system :
> >>
> >> Debian 5
> >> Jvm :
> >> victor@testhost:~/database/apache-cassandra-0.6.6$ java -version
> >> java version "1.6.0_22"
> >> Java(TM) SE Runtime Environment (build 1.6.0_22-b04)
> >> Java HotSpot(TM) 64-Bit Server VM (build 17.1-b03, mixed mode)
> >>
> >> RAM : 2Go
> >>
> >>
> >> 2011/2/8 Victor Kabdebon <victor.kabdebon@gmail.com>
> >>>
> >>> Sorry Jonathan :
> >>>
> >>> So most of these informations were taken using the command :
> >>>
> >>> sudo ps aux | grep cassandra
> >>>
> >>> For the nodetool information it is :
> >>>
> >>> /bin/nodetool --host localhost --port 8081 info
> >>>
> >>>
> >>> Regars,
> >>>
> >>> Victor K.
> >>>
> >>>
> >>> 2011/2/8 Jonathan Ellis <jbellis@gmail.com>
> >>>
> >>>> I missed the part where you explained where you're getting your
> numbers
> >>>> from.
> >>>>
> >>>>
> >>>> On Tue, Feb 8, 2011 at 9:32 AM, Victor Kabdebon
> >>>> <victor.kabdebon@gmail.com> wrote:
> >>>> > It is really weird that I am the only one to have this issue.
> >>>> > I restarted Cassandra today and already the memory compution is
over
> >>>> > the
> >>>> > limit :
> >>>> >
> >>>> > root      1739  4.0 24.5 664968 494996 pts/4   SLl  15:51   0:12
> >>>> > /usr/bin/java -ea -Xms128M -Xmx256M -XX:+UseParNewGC
> >>>> > -XX:+UseConcMarkSweepGC
> >>>> > -XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8
> >>>> > -XX:MaxTenuringThreshold=1
> >>>> > -XX:CMSInitiatingOccupancyFraction=75
> >>>> > -XX:+UseCMSInitiatingOccupancyOnly
> >>>> > -XX:+HeapDumpOnOutOfMemoryError
> >>>> > -Dcom.sun.management.jmxremote.port=8081
> >>>> > -Dcom.sun.management.jmxremotessl=false
> >>>> > -Dcom.sun.management.jmxremote.authenticate=false
> >>>> > -Dstorage-config=bin/../conf -cp
> >>>> >
> >>>> >
> bin/../conf:bin/../build/classes:bin/../lib/antlr-3.1.3.jar:bin/../lib/apache-cassandra-06.6.jar:bin/../lib/avro-1.2.0-dev.jar:bin/../lib/cassandra-javautils.jar:bin/../lib/clhm-production.jar:bin/../lib/commons-cli-1.1.jar:bin/../lib/commons-codec-1.2.jar:bin/../lib/commons-collections-3.2.1.jar:bin/../lib/commons-io-1.4.jar:bin/../lib/commons-lang-2.4.jar:bin/../lib/commons-pool-1.5.4.jar:bin/../lib/google-collections-1.0.jar:bin/../lib/hadoop-core-0.20.1.jar:bin/../lib/hector-0.6.0-14.jar:bin/../lib/high-scale-lib.jar:bin/./lib/ivy-2.1.0.jar:bin/../lib/jackson-core-asl-1.4.0.jar:bin/../lib/jackson-mapper-asl-1.4.0.jar:bin/../lib/jline-0.9.94.jar:bin/../lib/jna.jar:bin/../lib/json-simple-1.1.jar:bin/../lib/libthrift-r917130.jar:bin/../lib/log4j-1.2.14.jar:bin/../lib/perf4j-0.9.12.jar:bin/../lib/slf4j-api-1.5.8.jar:bin/../lib/slf4j-log4j12-1.5.8.jar:bin/../lib/uuid-3.1.jar
> >>>> > org.apache.cassandra.thrift.CassandraDaemon
> >>>> >
> >>>> > It is really an annoying problem if we cannot really foresee memory
> >>>> > consumption.
> >>>> >
> >>>> > Best regards,
> >>>> > Victor K
> >>>> >
> >>>> > 2011/2/8 Victor Kabdebon <victor.kabdebon@gmail.com>
> >>>> >>
> >>>> >> Dear all,
> >>>> >>
> >>>> >> Sorry to come back again to this point but I am really worried
> about
> >>>> >> Cassandra memory consumption. I have a single machine that
runs one
> >>>> >> Cassandra server. There is almost no data on it but I see a
crazy
> >>>> >> memory
> >>>> >> consumption and it doesn't care at all about the instructions...
> >>>> >> Note that I am not using mmap, but "Standard", I use also JNA
> (inside
> >>>> >> lib
> >>>> >> folder), i am running on debian 5 64 bits, so a pretty normal
> >>>> >> configuration.
> >>>> >> I also use Cassandra 0.6.8.
> >>>> >>
> >>>> >>
> >>>> >> Here are the informations I gathered on Cassandra :
> >>>> >>
> >>>> >> 105      16765  0.1 34.1 1089424 687476 ?      Sl   Feb02 
14:58I
> >>>> >> think you are
> >>>> >> /usr/bin/java -ea -Xms128M -Xmx256M -XX:+UseParNewGC
> >>>> >> -XX:+UseConcMarkSweepGC
> >>>> >> -XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8
> >>>> >> -XX:MaxTenuringThreshold=1
> >>>> >> -XX:CMSInitiatingOccupancyFraction=75
> >>>> >> -XX:+UseCMSInitiatingOccupancyOnly
> >>>> >> -XX:+HeapDumpOnOutOfMemoryError
> >>>> >> -Dcom.sunmanagement.jmxremote.port=8081
> >>>> >> -Dcom.sun.management.jmxremote.ssl=false
> >>>> >> -Dcom.sun.management.jmxremote.authenticate=false
> >>>> >> -Dstorage-config=bin/../conf -Dcassandra-foreground=yes -cp
> >>>> >>
> >>>> >>
> bin/../conf:bin/../build/classes:bin/../lib/antlr-3.1.3.jar:bin/../lib/apache-cassandra-0.6.6.jar:bin/../lib/avro-1.20-dev.jar:bin/../lib/cassandra-javautils.jar:bin/../lib/clhm-production.jar:bin/../lib/commons-cli-1.1.jar:bin/../lib/commons-codec-1.2.jar:bin/../lib/commons-collections-3.2.1.jar:bin/../lib/commons-io-1.4.jar:bin/../lib/commons-lang-2.4.jar:bin/../lib/commons-pool-1.5.4.jar:bin/../lib/google-collections-1.0.jar:bin/../lib/hadoop-core-0.20.1.jar:bin/../lib/hector-0.6.0-14.jar:bin/../lib/high-scale-lib.jar:bin/../lib/ivy-2.1.0.jar:bin/../lib/jackson-core-asl-1.4.0.jar:bin/../lib/jackson-mapper-asl-1.4.0.jar:bin/./lib/jline-0.9.94.jar:bin/../lib/jna.jar:bin/../lib/json-simple-1.1.jar:bin/../lib/libthrift-r917130.jar:bin/../lib/log4j-1.2.14.jar:bin/../lib/perf4j-0.9.12.jar:bin/../lib/slf4j-api-1.5.8.jar:bin/../lib/slf4j-log4j12-1.5.8.jar:bin/../lib/uuid-3.1.jar
> >>>> >> org.apache.cassandra.thrift.CassandraDaemon
> >>>> >>
> >>>> >> result of nodetool info :
> >>>> >>
> >>>> >> 116024732779488843382476400091948985708
> >>>> >> Load             : 1,94 MB
> >>>> >> Generation No    : 1296673772
> >>>> >> Uptime (seconds) : 467550
> >>>> >> Heap Memory (MB) : 120,26 / 253,94
> >>>> >>
> >>>> >>
> >>>> >> I have about 21 column families, none of them have a lot of
> >>>> >> information (
> >>>> >> as you see I have 2 Mb of text which is really small). Even
if I
> set
> >>>> >> Xmx at
> >>>> >> 256 there is 687M of memory used. Where does this memory come
from
> ?
> >>>> >> Bad
> >>>> >> garbage collection ? Something that I ignore ?
> >>>> >> Thank you for your help I really need to get rid of that problem.
> >>>> >>
> >>>> >> Best regards,
> >>>> >> Victor Kabdebon
> >>>> >
> >>>> >
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>> Jonathan Ellis
> >>>> Project Chair, Apache Cassandra
> >>>> co-founder of DataStax, the source for professional Cassandra support
> >>>> http://www.datastax.com
> >>>
> >>
> >
> >
>
> Each of your 21 column families will have its own memtable if you have
> the default memtable settings your memory usage will grow quite large
> over time. Have you tuned down your memtable size?
>

Mime
View raw message