incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Héctor Izquierdo Seliva <izquie...@strands.com>
Subject Re: How to warm up a cold node
Date Tue, 19 Apr 2011 20:20:11 GMT
El mié, 20-04-2011 a las 07:59 +1200, aaron morton escribió:
> The dynamic snitch only reduces the chance that a node used in a read operation, it depends
on the RF, the CL for the operation, the partitioner and possibly the network topology. Dropping
read messages is ok, so long as your operation completes at the requested CL.  
> 
> Are you using either a key_cache or a row_cache ? If so have you enabled the background
save for those as part of the column definition?
> 
> If this is about getting the OS caches warmed up you should see the pending count on
the READ stage backup up and the io stats start to show longer queues. In that case Peters
suggestion below is prob the best option.
> 
> Another hacky way to warm things would be to use get_count() on rows the app thiks it
may need to use. This will cause all the columns to be read from disk, but not sent over the
network. 
> 
> Aaron

The rows that need to be readed change over time and they are only read
for short amounts of time, so it's a bit tricky. I'll try to figure out
a way to use your suggestions.

In another thread I asked if it would be feasible to somehow store what
parts of the sstables are hot on shutdown and re read those parts of the
files on startup. Could it be done in a similar way to the work that's
being done on page migrations? What do you think?

Thanks for your time!


>   
> On 20 Apr 2011, at 00:41, Héctor Izquierdo Seliva wrote:
> 
> > Shouldn't the dynamic snitch take into account response times and ask a
> > slow node for less requests? It seems that at node startup, only a
> > handfull of requests arrive to the node and it keeps up well, but
> > there's moment where there's more than it can handle with a cold cache
> > and starts droping messages like crazy.
> > 
> > Could it be the transition from slow node to ok node is to steep?
> > 
> > 
> > El vie, 15-04-2011 a las 16:19 +0200, Peter Schuller escribió:
> >>> Hi everyone, is there any recommended procedure to warm up a node before
> >>> bringing it up?
> >> 
> >> Currently the only out-of-the-box support for warming up caches is
> >> that implied by the key cache and row cache, which will pre-heat on
> >> start-up. Indexes will be indirectly preheated by index sampling, to
> >> the extent that they operating system retains them in page cache.
> >> 
> >> If you're wanting to pre-heat sstables there's currently no way to do
> >> that (but it's a useful feature to have). Pragmatically, you can
> >> script something that e.g. does "cat path/to/keyspace/* > /dev/null"
> >> or similar. But that only works if the total database size fits
> >> reasonably well in page cache.
> >> 
> >> Pre-heating sstables on a per-cf basis on start-up would be a nice
> >> feature to have.
> >> 
> > 
> > 
> 



Mime
View raw message