incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From aaron morton <aa...@thelastpickle.com>
Subject Re: performance is drastically degraded after 0.7.8 --> 1.0.11 upgrade
Date Fri, 31 Aug 2012 04:09:50 GMT
>> we are running somewhat queue-like with aggressive write-read patterns.
We'll need some more details…

How much data ?
How many machines ?
What is the machine spec ?
How many clients ?
Is there an example of a slow request ? 
How are you measuring that it's slow ? 
Is there anything unusual in the log ? 

Cheers

-----------------
Aaron Morton
Freelance Developer
@aaronmorton
http://www.thelastpickle.com

On 31/08/2012, at 3:30 AM, Edward Capriolo <edlinuxguru@gmail.com> wrote:

> If you move from 7.X to 0.8X or 1.0X you have to rebuild sstables as
> soon as possible. If you have large bloomfilters you can hit a bug
> where the bloom filters will not work properly.
> 
> 
> On Thu, Aug 30, 2012 at 9:44 AM, Илья Шипицин <chipitsine@gmail.com>
wrote:
>> we are running somewhat queue-like with aggressive write-read patterns.
>> I was looking for scripting queries from live Cassandra installation, but I
>> didn't find any.
>> 
>> is there something like thrift-proxy or other query logging/scripting engine
>> ?
>> 
>> 2012/8/30 aaron morton <aaron@thelastpickle.com>
>>> 
>>> in terms of our high-rate write load cassandra1.0.11 is about 3 (three!!)
>>> times slower than cassandra-0.7.8
>>> 
>>> We've not had any reports of a performance drop off. All tests so far have
>>> show improvements in both read and write performance.
>>> 
>>> I agree, such digests save some network IO, but they seem to be very bad
>>> in terms of CPU and disk IO.
>>> 
>>> The sha1 is created so we can diagnose corruptions in the -Data component
>>> of the SSTables. They are not used to save network IO.
>>> It is calculated while streaming the Memtable to disk so has no impact on
>>> disk IO. While not the fasted algorithm I would assume it's CPU overhead in
>>> this case is minimal.
>>> 
>>> there's already relatively small Bloom filter file, which can be used for
>>> saving network traffic instead of sha1 digest.
>>> 
>>> Bloom filters are used to test if a row key may exist in an SSTable.
>>> 
>>> any explanation ?
>>> 
>>> If you can provide some more information on your use case we may be able
>>> to help.
>>> 
>>> Cheers
>>> 
>>> 
>>> -----------------
>>> Aaron Morton
>>> Freelance Developer
>>> @aaronmorton
>>> http://www.thelastpickle.com
>>> 
>>> On 30/08/2012, at 5:18 AM, Илья Шипицин <chipitsine@gmail.com>
wrote:
>>> 
>>> in terms of our high-rate write load cassandra1.0.11 is about 3 (three!!)
>>> times slower than cassandra-0.7.8
>>> after some investigation carried out I noticed files with "sha1" extension
>>> (which are missing for Cassandra-0.7.8)
>>> 
>>> in maybeWriteDigest() function I see no option fot switching sha1 digests
>>> off.
>>> 
>>> I agree, such digests save some network IO, but they seem to be very bad
>>> in terms of CPU and disk IO.
>>> why to use one more digest (which have to be calculated), there's already
>>> relatively small Bloom filter file, which can be used for saving network
>>> traffic instead of sha1 digest.
>>> 
>>> any explanation ?
>>> 
>>> Ilya Shipitsin
>>> 
>>> 
>> 


Mime
View raw message