incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jake Luciani <jak...@gmail.com>
Subject Re: Overwhelming a cluster with writes?
Date Tue, 06 Apr 2010 09:17:58 GMT
Hi Ilya,

You will always blow up if you use consistancy level zero to write  
gigs of data. The safe minimum for writes is ONE.  Zero is meant for  
small non batched writes.

Also look into batch_mutation call to write lots of data at once, in a  
series of  chunks.  this helps save on network back and forth.

-Jake



On Apr 6, 2010, at 3:18 AM, Ilya Maykov <ivmaykov@gmail.com> wrote:

> Right, I meant 4GB heap vs. the standard 1GB. And all other options in
> cassandra.in.sh at their defaults.
>
> Sorry I am a bit new to JVM tuning, and very new to Cassandra :)
>
> -- Ilya
>
> On Tue, Apr 6, 2010 at 12:16 AM, Benjamin Black <b@b3k.us> wrote:
>> I am specifically suggesting you NOT use a heap that large with your
>> 8GB machines.  Please test with 4GB first.
>>
>> On Tue, Apr 6, 2010 at 12:13 AM, Ilya Maykov <ivmaykov@gmail.com>  
>> wrote:
>>> That does sound similar. It's possible that the difference I'm  
>>> seeing
>>> between ConsistencyLevel.ZERO and ConsistencyLevel.ALL is simply due
>>> to the fact that using ALL slows down the writers enough that the GC
>>> can keep up. I could do a test with multiple clients writing at  
>>> ALL in
>>> parallel tomorrow. If there are still no problems writing at ALL  
>>> even
>>> with extra load from additional clients, that might point to  
>>> problems
>>> in how async writes are handled vs. sync writes.
>>>
>>> I will also do some profiling of the server processes with both ZERO
>>> and ALL writer behaviors and report back.
>>>
>>> RE: JVM_OPTS, I will try running with the "more sane" options (but a
>>> larger heap) as well.
>>>
>>> -- Ilya
>>>
>>> On Mon, Apr 5, 2010 at 11:59 PM, Rob Coli <rcoli@digg.com> wrote:
>>>> On 4/5/10 11:48 PM, Ilya Maykov wrote:
>>>>>
>>>>> No, the disks on all nodes have about 750GB free space. Also as
>>>>> mentioned in my follow-up email, writing with ConsistencyLevel.ALL
>>>>> makes the slowdowns / crashes go away.
>>>>
>>>> I am not sure if the above is consistent with the cause of #896,  
>>>> but the
>>>> other symptoms ("I inserted a bunch of data really fast via  
>>>> Thrift and GC
>>>> melted my machine!") sound like it..
>>>>
>>>> https://issues.apache.org/jira/browse/CASSANDRA-896
>>>>
>>>> =Rob
>>>>
>>>
>>

Mime
View raw message