incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hiller, Dean" <Dean.Hil...@nrel.gov>
Subject Re: disabling bloomfilter not working? memory numbers don't add up?
Date Thu, 28 Feb 2013 19:05:36 GMT
Hmmmm so "nodetool info" heap memory only reports what JVM has allocated.  I had mistakingly
assumed it was using some of the JVM memory calls, no?

Thanks,
Dean

From: aaron morton <aaron@thelastpickle.com<mailto:aaron@thelastpickle.com>>
Reply-To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" <user@cassandra.apache.org<mailto:user@cassandra.apache.org>>
Date: Thursday, February 28, 2013 11:30 AM
To: "user@cassandra.apache.org<mailto:user@cassandra.apache.org>" <user@cassandra.apache.org<mailto:user@cassandra.apache.org>>
Subject: Re: disabling bloomfilter not working? memory numbers don't add up?


1. Can I stop the node, delete the *Filter.db files and restart the node(is this safe)???
No.

2. Why do I have 5 gig being eaten up by cassandra?  "nodetool info" memory 5.2Gig, key cache:11
meg and row cache 0 bytes.   All bloomfilters are also small <1meg.
If this is the Heap memory reported by the JVM that all you can say is "since the server was
started it has allocated at least 5.2 GB of memory" it's not "there is 5.2GB of live memory
in use"

Cheers
-----------------
Aaron Morton
Freelance Cassandra Developer
New Zealand

@aaronmorton
http://www.thelastpickle.com

On 25/02/2013, at 9:32 AM, "Hiller, Dean" <Dean.Hiller@nrel.gov<mailto:Dean.Hiller@nrel.gov>>
wrote:

Hmmmm, my upgrade completed and then I added node back in and ran my repair.  What is weird
is that my nreldata column family still shows 156Meg of memory still(down from 2 gig though!!)
in use and a false positive ratio of .99576 when I have the filter completely disabled(ie.
Set to 1.0).  I see the *Filter.db files on disk(and size approximately matches the in-memory
size).  I tried restarting the node as well.

1. Can I stop the node, delete the *Filter.db files and restart the node(is this safe)???
2. Why do I have 5 gig being eaten up by cassandra?  "nodetool info" memory 5.2Gig, key cache:11
meg and row cache 0 bytes.   All bloomfilters are also small <1meg.

Exception to #2 is I have nreldata still using 156MB for some reason but still no where close
to 5.2 gig that nodetool shows in use.

Thanks,
Dean




Bloom Filter Space Used: 2318392048<tel:2318392048>
Just to be sane do a quick check of the -Filter.db files on disk for this CF.
If they are very small try a restart on the node.

Number of Keys (estimate): 1249133696
Hey a billion rows on a node, what an age we live in :)

Cheers

-----------------
Aaron Morton
Freelance Cassandra Developer
New Zealand

@aaronmorton
http://www.thelastpickle.com<http://www.thelastpickle.com/>

On 23/02/2013, at 4:35 AM, "Hiller, Dean" <Dean.Hiller@nrel.gov<mailto:Dean.Hiller@nrel.gov><mailto:Dean.Hiller@nrel.gov><mailto:Dean.Hiller@nrel.gov<mailto:Dean.Hiller@nrel.gov>>>
wrote:

So in the cli, I ran

update column family nreldata with bloom_filter_fp_chance=1.0;

Then I ran

nodetool upgradesstables databus5 nreldata;

But my bloom filter size is still around 2gig(and I want to free up this heap)!!!! According
to nodetool cfstats command…

Column Family: nreldata
SSTable count: 10
Space used (live): 96841497731
Space used (total): 96841497731
Number of Keys (estimate): 1249133696
Memtable Columns Count: 7066
Memtable Data Size: 4286174
Memtable Switch Count: 924
Read Count: 19087150
Read Latency: 0.595 ms.
Write Count: 21281994
Write Latency: 0.013 ms.
Pending Tasks: 0
Bloom Filter False Postives: 974393
Bloom Filter False Ratio: 0.99998
Bloom Filter Space Used: 2318392048
Compacted row minimum size: 73
Compacted row maximum size: 446
Compacted row mean size: 143








Mime
View raw message