From user-return-16438-apmail-cassandra-user-archive=cassandra.apache.org@cassandra.apache.org Wed May 4 17:02:53 2011 Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 72F513836 for ; Wed, 4 May 2011 17:02:53 +0000 (UTC) Received: (qmail 45740 invoked by uid 500); 4 May 2011 17:02:50 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 45723 invoked by uid 500); 4 May 2011 17:02:50 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 45715 invoked by uid 99); 4 May 2011 17:02:50 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 04 May 2011 17:02:50 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [216.32.181.183] (HELO ch1outboundpool.messaging.microsoft.com) (216.32.181.183) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 04 May 2011 17:02:41 +0000 Received: from mail142-ch1-R.bigfish.com (216.32.181.172) by CH1EHSOBE012.bigfish.com (10.43.70.62) with Microsoft SMTP Server id 14.1.225.8; Wed, 4 May 2011 17:02:19 +0000 Received: from mail142-ch1 (localhost.localdomain [127.0.0.1]) by mail142-ch1-R.bigfish.com (Postfix) with ESMTP id 06A3D1BE84DF for ; Wed, 4 May 2011 17:02:20 +0000 (UTC) X-SpamScore: -15 X-BigFish: VPS-15(zz14e0M9371O98dKzz1202hzz8275bhz2dh2a8h668h839h61h) X-Spam-TCS-SCL: 0:0 X-Forefront-Antispam-Report: KIP:(null);UIP:(null);IPVD:NLI;H:AAPQHTCAS04.proque.st;RD:none;EFVD:NLI Received: from mail142-ch1 (localhost.localdomain [127.0.0.1]) by mail142-ch1 (MessageSwitch) id 1304528539680255_3357; Wed, 4 May 2011 17:02:19 +0000 (UTC) Received: from CH1EHSMHS011.bigfish.com (snatpool1.int.messaging.microsoft.com [10.43.68.248]) by mail142-ch1.bigfish.com (Postfix) with ESMTP id A145017004C for ; Wed, 4 May 2011 17:02:19 +0000 (UTC) Received: from AAPQHTCAS04.proque.st (165.215.94.2) by CH1EHSMHS011.bigfish.com (10.43.70.11) with Microsoft SMTP Server (TLS) id 14.1.225.8; Wed, 4 May 2011 17:02:17 +0000 Received: from AAPQMAILBX02V.proque.st ([fe80::a5a3:d005:a27c:9314]) by AAPQHTCAS04.proque.st ([fe80::6883:b760:3780:b2db%10]) with mapi; Wed, 4 May 2011 13:02:16 -0400 From: "Serediuk, Adam" To: "user@cassandra.apache.org" Date: Wed, 4 May 2011 13:02:12 -0400 Subject: Re: Node setup - recommended hardware Thread-Topic: Node setup - recommended hardware Thread-Index: AcwKfQUHTIVNGSDFQD+SDwl4SjHhKA== Message-ID: References: In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: acceptlanguage: en-US Content-Type: multipart/alternative; boundary="_000_FF1F2B8CABC941A9A6921F59C780E376serialssolutionscom_" MIME-Version: 1.0 X-OriginatorOrg: serialssolutions.com --_000_FF1F2B8CABC941A9A6921F59C780E376serialssolutionscom_ Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Having a well known node configuration that is trivial (one step) to create= is your best maintenance bet. We are using 4 disk nodes in the following c= onfiguration: disk1: boot_raid1 os_raid1 cassandra_commit_log disk2: boot_raid1 os_raid1 cassandra_data_dir_raid0 disk3: cassandra_data_dir_raid0 disk4: cassandra_data_dir_raid0 This gives us a solid stable foundation for the OS and the recommended conf= iguration for cassandra commitlog and data dirs. Every node in the ring can= be replaced with a single command via cobbler to have a replacement provis= ioned from bare metal to take over for a node if it fails. We will never bo= ther with repairing a node - we will replace it entirely upon failure from= bare metal. The node with the issue will be taken out of service, the issu= e resolved and put back into a pool of spares. On May 4, 2011, at 9:52 AM, Anthony Ikeda wrote: I wouldn't be concerned more about the performance with this configuration = I'm looking more form a maintenance perspective - I have to draft some main= tenance for our infrastructure team whom are used to a standard NAS storage= setup which Cassandra obviously breaks. Ultimately, would keeping the cassandra service separate from the data and/= or commit logs benefit from a recovery perspective where if we lose the pri= mary partition, we could restore that from the data that is still on the se= condary? What it considered best practice? What kind of routine health checks are best to look for daily? monthly? ann= ually? Basically how do you up-skill a technical infrastructure team to be able to= maintain a Cassandra node ring? Anthony On Wed, May 4, 2011 at 9:39 AM, Eric tamme > wrote: On Wed, May 4, 2011 at 12:25 PM, Anthony Ikeda > wrote: > I just want to ask, when setting up nodes in a Node ring is it worthwhile > using a 2 partition setup? i.e. Cassandra on the Primary, data directorie= s > etc on the second partition or does it really not make a difference? > Anthony > I don't think it makes much difference from a performance perspective at all. You might want to create a separate LVM for your data, or entire /var --_000_FF1F2B8CABC941A9A6921F59C780E376serialssolutionscom_ Content-Type: text/html; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Having a well known node c= onfiguration that is trivial (one step) to create is your best maintenance = bet. We are using 4 disk nodes in the following configuration:

disk1: boot_raid1 os_raid1 cassandra_commit_log
disk2= : boot_raid1 os_raid1 cassandra_data_dir_raid0
disk3: c= assandra_data_dir_raid0
disk4: cassandra_data_dir_raid0

This gives us a solid stable foundation for the OS and the = recommended configuration for cassandra commitlog and data dirs. Every node= in the ring can be replaced with a single command via cobbler to have a re= placement provisioned from bare metal to take over for a node if it fails. = We will never bother with repairing a node -  we will replace it entir= ely upon failure from bare metal. The node with the issue will be taken out= of service, the issue resolved and put back into a pool of spares.


On May 4, 2011, at 9:52 AM, Anthony Ikeda wrote:

I wouldn't be co= ncerned more about the performance with this configuration I'm looking more= form a maintenance perspective - I have to draft some maintenance for our = infrastructure team whom are used to a standard NAS storage setup which Cas= sandra obviously breaks.

Ultimately, would keeping the cassandra service separate fro= m the data and/or commit logs benefit from a recovery perspective where if = we lose the primary partition, we could restore that from the data that is = still on the secondary?

What it considered best practice?
What kind o= f routine health checks are best to look for daily? monthly? annually?

Basically how do you up-skill a technical infrastructu= re team to be able to maintain a Cassandra node ring?

Anthony



On Wed, May 4, 2011 at 9:39 AM, Eric tamme &= lt;etamme@gmail.com> wrot= e:
On Wed, May 4, 2011 at 12= :25 PM, Anthony Ikeda
<anthony.ikeda.dev@gmail.= com> wrote:
> I just want to ask, when setti= ng up nodes in a Node ring is it worthwhile
> using a 2 partition setup? i.e. Cassandra on the Primary, data directo= ries
> etc on the second partition or does it really not make a difference? > Anthony
>


I don't think it makes much difference from a performance persp= ective
at all.  You might want to create a separate LVM for your data, or
entire /var


= --_000_FF1F2B8CABC941A9A6921F59C780E376serialssolutionscom_--