Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@cassandra.apache.org
Received-SPF: pass (nike.apache.org: domain of
 gcdcu-cassandra-user-1@m.gmane.org designates 80.91.229.12 as permitted
 sender)
To: user@cassandra.apache.org
From: Oleg Anastastasyev <oleganas@gmail.com>
Subject: Re: 15 seconds to increment 17k keys?
Date: Tue, 6 Sep 2011 06:01:02 +0000 (UTC)
Lines: 19
Message-ID: <loom.20110906T075333-833@post.gmane.org>
References: 
 <CACE81MjFjhCyKWzSOm-ePA+nuusGeat3bRiYwz_mWYHuLd5agQ@mail.gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
User-Agent: Loom/3.14 (http://gmane.org/)

> in the family. There are millions of rows. Each operation consists of
> doing a batch_insert through pycassa, which increments ~17k keys. A
> majority of these keys are new in each batch.
> 
>  Each operation is taking up to 15 seconds. For our system this is a
> significant bottleneck.
> 

Try to split your batch to smaller pieces and launch them in parallel. This way
you may get better performance, because all cores are employed and there will be
less copying/rebuilding of large structures inside thrift & cassandra. I found
that 1k rows in a batch is behaving better than 10k.

It is also a good idea to split batch to slices according to replication
strategy and communicate appropriate slice directly to its natural endpoint.
This will reduce neccessary intercommunication between nodes.