incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From aaron morton <aa...@thelastpickle.com>
Subject Re: How to make compaction run faster?
Date Thu, 18 Apr 2013 07:50:39 GMT
> I believe that compaction occurs on the data directories and not in the commitlog. 
Yes, compaction only works on the data files. 

> When I ran iostat; I see "await" 26ms to 30 ms for my commit log disk. My CPU is less
than 18% used.
> 
> How I reduce the disk latency for my commit log disk. They are SSDs.
That does not sound right. Can you include the output from iostat for the commit log and data
volumes. Also some information on how many writes you are processing the the size of rows
as well. 

Cheers

-----------------
Aaron Morton
Freelance Cassandra Consultant
New Zealand

@aaronmorton
http://www.thelastpickle.com

On 18/04/2013, at 11:58 AM, Alexis Rodríguez <arodriguez@inconcertcc.com> wrote:

> Jay, 
> 
> I believe that compaction occurs on the data directories and not in the commitlog. 
> 
> http://wiki.apache.org/cassandra/MemtableSSTable
> 
> 
> 
> 
> On Wed, Apr 17, 2013 at 7:58 PM, Jay Svc <jaytechgeek@gmail.com> wrote:
> Hi Alexis,
> 
> Thank you for your response. 
> 
> My commit log is on SSD. which shows me 30 to 40 ms of disk latency.  
> 
> When I ran iostat; I see "await" 26ms to 30 ms for my commit log disk. My CPU is less
than 18% used.
> 
> How I reduce the disk latency for my commit log disk. They are SSDs.
> 
> Thank you in advance,
> Jay
> 
> 
> On Wed, Apr 17, 2013 at 3:58 PM, Alexis Rodríguez <arodriguez@inconcertcc.com>
wrote:
> :D
> 
> Jay, check if your disk(s) utilization allows you to change the configuration the way
Edward suggest. iostat -xkcd 1 will show you how much of your disk(s) are in use.
> 
> 
> 
> 
> On Wed, Apr 17, 2013 at 5:26 PM, Edward Capriolo <edlinuxguru@gmail.com> wrote:
> three things:
> 1) compaction throughput is fairly low (yaml nodetool)
> 2) concurrent compactions is fairly low (yaml)
> 3) multithreaded compaction might be off in your version
> 
> Try raising these things. Otherwise consider option 4.
> 
> 4)$$$$$$$$$$$$$$$$$$$$$$$ RAID,RAM<CPU$$$$$$$$$$$$$$
> 
> 
> On Wed, Apr 17, 2013 at 4:01 PM, Jay Svc <jaytechgeek@gmail.com> wrote:
> Hi Team,
> 
> 
> I have a high write traffic to my Cassandra cluster. I experience a very high number
of pending compactions. As I expect higher writes, The pending compactions keep increasing.
Even when I stop my writes it takes several hours to finishing pending compactions.
> 
> My CF is configured with LCS, with sstable_size_mb=20M. My CPU is below 20%, JVM memory
usage is between 45%-55%. I am using Cassandra 1.1.9.
> 
> How can I increase the compaction rate so it will run bit faster to match my write speed?

> 
> Your inputs are appreciated.
> 
> Thanks,
> Jay
> 
> 
> 
> 
> 


Mime
View raw message