incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Victor Kabdebon <victor.kabde...@gmail.com>
Subject Re: Cassandra memory consumption
Date Tue, 08 Feb 2011 21:56:24 GMT
I will do that in the future and I will post my results here ( I upgraded
the server to debian 6 to see if there is any change, so memory is back to
normal). I will report in a few days.
In the meantime I am open to any suggestion...

2011/2/8 Aaron Morton <aaron@thelastpickle.com>

> When you attach to the JVM with JConsole how much non heap memory and how
> much heap memory is reported on the memory tab?
>
> Xmx controls the total size of the heap memory, which excludes the
> permanent generation.
> see
>
> http://www.oracle.com/technetwork/java/javase/gc-tuning-6-140523.html#generation_sizing
> and
>
> http://blogs.suncom/jonthecollector/entry/presenting_the_permanent_generation<http://blogs.sun.com/jonthecollector/entry/presenting_the_permanent_generation>
>
> <http://blogs.sun.com/jonthecollector/entry/presenting_the_permanent_generation>
> Total non-heap memory on a 0.7 box I have is around 27M. You numbers seem
> large but it would be interesting to know what the JVM is reporting.
>
> Aaron
>
> On 09 Feb, 2011,at 05:57 AM, Victor Kabdebon <victor.kabdebon@gmail.com>
> wrote:
>
> Information on the system :
>
> *Debian 5*
> *Jvm :*
> victor@testhost:~/database/apache-cassandra-0.6.6$ java -version
> java version "1.6.0_22"
> Java(TM) SE Runtime Environment (build 1.6.0_22-b04)
> Java HotSpot(TM) 64-Bit Server VM (build 17.1-b03, mixed mode)
>
> *RAM :* 2Go
>
>
> 2011/2/8 Victor Kabdebon <victor.kabdebon@gmail.com>
>
>> Sorry Jonathan :
>>
>> So most of these informations were taken using the command :
>>
>> sudo ps aux | grep cassandra
>>
>> For the nodetool information it is :
>>
>> /bin/nodetool --host localhost --port 8081 info
>>
>>
>> Regars,
>>
>> Victor K.
>>
>>
>> 2011/2/8 Jonathan Ellis <jbellis@gmail.com>
>>
>>
>> I missed the part where you explained where you're getting your numbers
>>> from.
>>>
>>>
>>> On Tue, Feb 8, 2011 at 9:32 AM, Victor Kabdebon
>>> <victor.kabdebon@gmail.com> wrote:
>>> > It is really weird that I am the only one to have this issue.
>>> > I restarted Cassandra today and already the memory compution is over
>>> the
>>> > limit :
>>> >
>>> > root      1739  4.0 24.5 664968 494996 pts/4   SLl  15:51   0:12
>>> > /usr/bin/java -ea -Xms128M -Xmx256M -XX:+UseParNewGC
>>> -XX:+UseConcMarkSweepGC
>>> > -XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8
>>> -XX:MaxTenuringThreshold=1
>>> > -XX:CMSInitiatingOccupancyFraction=75
>>> -XX:+UseCMSInitiatingOccupancyOnly
>>> > -XX:+HeapDumpOnOutOfMemoryError
>>> -Dcom.sun.management.jmxremote.port=8081
>>> > -Dcom.sun.management.jmxremotessl=false
>>>
>>> > -Dcom.sun.management.jmxremote.authenticate=false
>>> > -Dstorage-config=bin/../conf -cp
>>> >
>>> bin/../conf:bin/../build/classes:bin/../lib/antlr-3.1.3.jar:bin/../lib/apache-cassandra-06.6.jar:bin/../lib/avro-1.2.0-dev.jar:bin/../lib/cassandra-javautils.jar:bin/../lib/clhm-production.jar:bin/../lib/commons-cli-1.1.jar:bin/../lib/commons-codec-1.2.jar:bin/../lib/commons-collections-3.2.1.jar:bin/../lib/commons-io-1.4.jar:bin/../lib/commons-lang-2.4.jar:bin/../lib/commons-pool-1.5.4.jar:bin/../lib/google-collections-1.0.jar:bin/../lib/hadoop-core-0.20.1.jar:bin/../lib/hector-0.6.0-14.jar:bin/../lib/high-scale-lib.jar:bin/./lib/ivy-2.1.0.jar:bin/../lib/jackson-core-asl-1.4.0.jar:bin/../lib/jackson-mapper-asl-1.4.0.jar:bin/../lib/jline-0.9.94.jar:bin/../lib/jna.jar:bin/../lib/json-simple-1.1.jar:bin/../lib/libthrift-r917130.jar:bin/../lib/log4j-1.2.14.jar:bin/../lib/perf4j-0.9.12.jar:bin/../lib/slf4j-api-1.5.8.jar:bin/../lib/slf4j-log4j12-1.5.8.jar:bin/../lib/uuid-3.1.jar
>>>
>>> > org.apache.cassandra.thrift.CassandraDaemon
>>> >
>>> > It is really an annoying problem if we cannot really foresee memory
>>> > consumption.
>>> >
>>> > Best regards,
>>> > Victor K
>>> >
>>> > 2011/2/8 Victor Kabdebon <victor.kabdebon@gmail.com>
>>> >>
>>> >> Dear all,
>>> >>
>>> >> Sorry to come back again to this point but I am really worried about
>>> >> Cassandra memory consumption. I have a single machine that runs one
>>> >> Cassandra server. There is almost no data on it but I see a crazy
>>> memory
>>> >> consumption and it doesn't care at all about the instructions...
>>> >> Note that I am not using mmap, but "Standard", I use also JNA (inside
>>> lib
>>> >> folder), i am running on debian 5 64 bits, so a pretty normal
>>> configuration.
>>> >> I also use Cassandra 0.6.8.
>>> >>
>>> >>
>>> >> Here are the informations I gathered on Cassandra :
>>> >>
>>> >> 105      16765  0.1 34.1 1089424 687476 ?      Sl   Feb02  14:58I
>>> think you are
>>>
>>> >> /usr/bin/java -ea -Xms128M -Xmx256M -XX:+UseParNewGC
>>> -XX:+UseConcMarkSweepGC
>>> >> -XX:+CMSParallelRemarkEnabled -XX:SurvivorRatio=8
>>> -XX:MaxTenuringThreshold=1
>>> >> -XX:CMSInitiatingOccupancyFraction=75
>>> -XX:+UseCMSInitiatingOccupancyOnly
>>> >> -XX:+HeapDumpOnOutOfMemoryError
>>> -Dcom.sunmanagement.jmxremote.port=8081
>>>
>>> >> -Dcom.sun.management.jmxremote.ssl=false
>>> >> -Dcom.sun.management.jmxremote.authenticate=false
>>> >> -Dstorage-config=bin/../conf -Dcassandra-foreground=yes -cp
>>> >>
>>> bin/../conf:bin/../build/classes:bin/../lib/antlr-3.1.3.jar:bin/../lib/apache-cassandra-0.6.6.jar:bin/../lib/avro-1.20-dev.jar:bin/../lib/cassandra-javautils.jar:bin/../lib/clhm-production.jar:bin/../lib/commons-cli-1.1.jar:bin/../lib/commons-codec-1.2.jar:bin/../lib/commons-collections-3.2.1.jar:bin/../lib/commons-io-1.4.jar:bin/../lib/commons-lang-2.4.jar:bin/../lib/commons-pool-1.5.4.jar:bin/../lib/google-collections-1.0.jar:bin/../lib/hadoop-core-0.20.1.jar:bin/../lib/hector-0.6.0-14.jar:bin/../lib/high-scale-lib.jar:bin/../lib/ivy-2.1.0.jar:bin/../lib/jackson-core-asl-1.4.0.jar:bin/../lib/jackson-mapper-asl-1.4.0.jar:bin/./lib/jline-0.9.94.jar:bin/../lib/jna.jar:bin/../lib/json-simple-1.1.jar:bin/../lib/libthrift-r917130.jar:bin/../lib/log4j-1.2.14.jar:bin/../lib/perf4j-0.9.12.jar:bin/../lib/slf4j-api-1.5.8.jar:bin/../lib/slf4j-log4j12-1.5.8.jar:bin/../lib/uuid-3.1.jar
>>>
>>> >> org.apache.cassandra.thrift.CassandraDaemon
>>> >>
>>> >> result of nodetool info :
>>> >>
>>> >> 116024732779488843382476400091948985708
>>> >> Load             : 1,94 MB
>>> >> Generation No    : 1296673772
>>> >> Uptime (seconds) : 467550
>>> >> Heap Memory (MB) : 120,26 / 253,94
>>> >>
>>> >>
>>> >> I have about 21 column families, none of them have a lot of
>>> information (
>>> >> as you see I have 2 Mb of text which is really small). Even if I set
>>> Xmx at
>>> >> 256 there is 687M of memory used. Where does this memory come from ?
>>> Bad
>>> >> garbage collection ? Something that I ignore ?
>>> >> Thank you for your help I really need to get rid of that problem.
>>> >>
>>> >> Best regards,
>>> >> Victor Kabdebon
>>> >
>>> >
>>>
>>>
>>>
>>> --
>>> Jonathan Ellis
>>> Project Chair, Apache Cassandra
>>> co-founder of DataStax, the source for professional Cassandra support
>>> http://www.datastax.com
>>>
>>
>>
>

Mime
View raw message