From user-return-23897-apmail-cassandra-user-archive=cassandra.apache.org@cassandra.apache.org Thu Feb 2 17:53:28 2012 Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 644D99CDC for ; Thu, 2 Feb 2012 17:53:28 +0000 (UTC) Received: (qmail 49253 invoked by uid 500); 2 Feb 2012 17:53:26 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 49115 invoked by uid 500); 2 Feb 2012 17:53:25 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 49107 invoked by uid 99); 2 Feb 2012 17:53:25 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 02 Feb 2012 17:53:24 +0000 X-ASF-Spam-Status: No, hits=-2.3 required=5.0 tests=RCVD_IN_DNSWL_MED,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of forsberg@opera.com designates 213.236.208.81 as permitted sender) Received: from [213.236.208.81] (HELO smtp.opera.com) (213.236.208.81) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 02 Feb 2012 17:53:15 +0000 Received: from [10.30.1.71] (oslo.jvpn.opera.com [213.236.208.46]) (authenticated bits=0) by smtp.opera.com (8.14.3/8.14.3/Debian-5+lenny1) with ESMTP id q12Hqsr9032324 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT) for ; Thu, 2 Feb 2012 17:52:54 GMT Message-ID: <4F2ACD75.6080106@opera.com> Date: Thu, 02 Feb 2012 18:52:53 +0100 From: Erik Forsberg User-Agent: Mozilla/5.0 (X11; Linux i686; rv:9.0) Gecko/20111220 Thunderbird/9.0 MIME-Version: 1.0 To: user@cassandra.apache.org Subject: Recommended configuration for good streaming performance? Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Virus-Checked: Checked by ClamAV on apache.org Hi! We're experimenting with streaming from Hadoop to Cassandra using BulkoutputFormat, on cassandra-1.1 branch. Are there any specific settings we should tune on the Cassandra servers in order to get the best streaming performance? Our Cassandra hardware are 16 core (including HT cores) with 24GiB of RAM. They have two disks each. So far we've configured them with commitlog on one disk and sstables on the other, but with streaming not using commitlog (correct?) maybe it makes sense to have sstables on both disks, doubling available I/O? Thoughts on number of parallel streaming clients? Thanks, \EF