incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From aaron morton <aa...@thelastpickle.com>
Subject Re: OOM opening bloom filter
Date Tue, 13 Mar 2012 09:10:02 GMT
Thanks for the update. 

How much smaller did the BF get to ? 

A

-----------------
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com

On 13/03/2012, at 8:24 AM, Mick Semb Wever wrote:

> 
>>>>> It's my understanding then for this use case that bloom filters are of
>>>>> little importance and that i can
> 
> 
> Ok. To summarise our actions to get us out of this situation, in hope
> that it may help others one day, we did the following actions:
> 
> 1) upgrade to 1.0.7
> 2) set fp_ratio=0.99
> 3) set index_interval=4096
> 4) restarted the node with Xmx30G
> 5) run `nodetool scrub` 
>      and monitor total size of bf files
>      using `du -hc *-Filter.db | grep total`
> 6) restart node with original Xmx setting once total bf size is under
>      (scrub was running for >12hrs)
>      (remaining bloom filters can be rebuilt later from normal compact)
> 
> Hopefully it will also eventuate that this cluster can run with a more
> normal Xmx4G rather than the previous Xmx12G.
> 
> (2) and (3) are very much dependent on our set up using hadoop where all
> reads are get_range_slice with 16k rows per request. Both could be tuned
> correctly but they're the numbers that worked first up.
> 
> ~mck
> 
> -- 
> "When there is no enemy within, the enemies outside can't hurt you."
> African proverb 
> 
> | http://github.com/finn-no | http://tech.finn.no |


Mime
View raw message