Return-Path: X-Original-To: apmail-cassandra-user-archive@www.apache.org Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 122DA10B6F for ; Thu, 29 Aug 2013 23:39:41 +0000 (UTC) Received: (qmail 96202 invoked by uid 500); 29 Aug 2013 23:39:38 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 96157 invoked by uid 500); 29 Aug 2013 23:39:38 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 96148 invoked by uid 99); 29 Aug 2013 23:39:38 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 29 Aug 2013 23:39:38 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of jonathan.haddad@gmail.com designates 209.85.128.182 as permitted sender) Received: from [209.85.128.182] (HELO mail-ve0-f182.google.com) (209.85.128.182) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 29 Aug 2013 23:39:32 +0000 Received: by mail-ve0-f182.google.com with SMTP id m1so879001ves.27 for ; Thu, 29 Aug 2013 16:39:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:sender:in-reply-to:references:date:message-id:subject :from:to:content-type; bh=OABjdrkiVy90e/qa41VoXJMVXuR2m+ONKZBQDYBJBe0=; b=kbpwYKo1jxBYuHEw9crPLWttB8Naq3E5xPvLgd3HfbpIqGUxIqXhh/mPOmTLrZYf1l gR+SJGs4ZOra67Wyz2Ta6eUNaHKgQgqMzKGJRR5LMJ2//Hyvg8gS8zPl740uqnoPtvZM fXTsi2lHzh8d3BN5DN0qh3f+NzWCocju4gxrEWLKRm2ldknGDPVutg0YflB8qFGRgOot jZoc+FKLxYhn+pxhh6Xrp5eIH5IruBvFGeWLqPYnyljeSzhMuC1988MzQ3JbCR0FjFAE Vjso/KqG+di532NzeYgw/DUDKmcJoLrxgcqtIv480bPgxE492F4IY9NlSCUZlaLEmfbo MmBg== MIME-Version: 1.0 X-Received: by 10.220.145.132 with SMTP id d4mr5328366vcv.9.1377819552004; Thu, 29 Aug 2013 16:39:12 -0700 (PDT) Sender: jonathan.haddad@gmail.com Received: by 10.58.100.16 with HTTP; Thu, 29 Aug 2013 16:39:11 -0700 (PDT) In-Reply-To: References: Date: Thu, 29 Aug 2013 16:39:11 -0700 X-Google-Sender-Auth: Is-3ttVXpYZOS9H6yV3Ur2p68Fk Message-ID: Subject: Re: Cluster Management From: Jonathan Haddad To: user@cassandra.apache.org Content-Type: multipart/alternative; boundary=047d7b34347cf63fa104e51e9c9d X-Virus-Checked: Checked by ClamAV on apache.org --047d7b34347cf63fa104e51e9c9d Content-Type: text/plain; charset=ISO-8859-1 An alternative to cssh is fabric. It's very flexible in that you can automate almost any repetitive task that you'd send to machines in a cluster, and it's written in python, meaning if you're in AWS you can mix it with boto to automate pretty much anything you want. On Thu, Aug 29, 2013 at 4:25 PM, Anthony Grasso wrote: > Hi Particia, > > Thank you for the feedback. It has been helpful. > > > On Tue, Aug 27, 2013 at 12:02 AM, Patricia Gorla > wrote: > >> Anthony, >> >> We use a number of tools to manage our Cassandra cluster. >> >> * Datastax OpsCenter [0] for at a glance information, and trending >> statistics. You can also run operations through here, though I prefer >> to use nodetool for any mutative operation. >> * nodetool for ad hoc status checks, and day-to-day node management. >> * puppet for setup and initialization >> >> > For example, if I want to make some changes to the configuration file >> that resides on each node, is there a tool that will propagate the change >> to each node? >> >> For this, we use puppet to manage any changes to the configurations >> (which are stored in git). We initially had Cassandra auto-restart >> when the configuration changed, but you might not want the node to >> automatically join a cluster, so we turned this off. >> > > Puppet was the first thing that came to mind for us as well. In addition, > we had the same thought about auto-restarting nodes when the configuration > is changed. If a configuration on all the nodes is changed, we would want > to restart one node at a time and wait for it to rejoin before restarting > the next one. I am assuming in a case like this, you then manually perform > the restart operation for each node? > > >> >> > Another example is if I want to have a rolling repair (nodetool repair >> -pr) and clean up running on my cluster, is there a tool that will help >> manage/configure that? >> >> Multiple commands to the cluster are sent via clusterssh [1] (cssh for >> OS X). I can easily choose which nodes to control, and run those in >> sync. For any rolling procedures, we send commands one at a time, >> though we've considered sending some of these tasks to cron. >> > > Thanks again for the tip! This is quite interesting; it may help to solve > our immediate problem for now. > > Regards, > Anthony > > >> >> Hope this helps. >> >> Cheers, >> Patricia >> >> >> [0] http://planetcassandra.org/Download/DataStaxCommunityEdition >> [1] http://sourceforge.net/projects/clusterssh/ >> > > -- Jon Haddad http://www.rustyrazorblade.com skype: rustyrazorblade --047d7b34347cf63fa104e51e9c9d Content-Type: text/html; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable
An alternative to cssh is fabric. =A0It's very flexibl= e in that you can automate almost any repetitive task that you'd send t= o machines in a cluster, and it's written in python, meaning if you'= ;re in AWS you can mix it with boto to automate pretty much anything you wa= nt.


On Thu, Aug 2= 9, 2013 at 4:25 PM, Anthony Grasso <anthony.grasso@gmail.com>= ; wrote:
Hi Particia,
=

Thank you for the feedback. It has been helpful.
<= div class=3D"gmail_extra">

On Tue, Aug 27, 2013 at 12:02 AM, Patric= ia Gorla <gorla.patricia@gmail.com> wrote:
Anthony,

We use a number of tools to manage our Cassandra cluster.

* Datastax OpsCenter [0] for at a glance information, and trending
statistics. You can also run operations through here, though I prefer
to use nodetool for any mutative operation.
* nodetool for ad hoc status checks, and day-to-day node management.
* puppet for setup and initialization

> For example, if I want to make some changes to the configuration file = that resides on each node, is there a tool that will propagate the change t= o each node?

For this, we use puppet to manage any changes to the configurations (which are stored in git). We initially had Cassandra auto-restart
when the configuration changed, but you might not want the node to
automatically join a cluster, so we turned this off.
<= br>
Puppet was the first thing that came to mind for us as well. = In addition, we had the same thought about auto-restarting nodes when the c= onfiguration is changed. If a configuration on all the nodes is changed, we= would want to restart one node at a time and wait for it to rejoin before = restarting the next one. I am assuming in a case like this, you then manual= ly perform the restart operation for each node?
=A0

> Another example is if I want to have a rolling repair (nodetool repair= -pr) and clean up running on my cluster, is there a tool that will help ma= nage/configure that?

Multiple commands to the cluster are sent via clusterssh [1] (cssh fo= r
OS X). I can easily choose which nodes to control, and run those in
sync. For any rolling procedures, we send commands one at a time,
though we've considered sending some of these tasks to cron.

Thanks again for the tip! This is quite interesti= ng; it may help to solve our immediate problem for now.

Regards,
Anthony
=A0

Hope this helps.

Cheers,
Patricia


[0] http://planetcassandra.org/Download/DataStaxCommunityEd= ition
[1] http://sourceforge.net/projects/clusterssh/




--
Jon Haddadhttp://www.r= ustyrazorblade.com
skype: rustyrazorblade
--047d7b34347cf63fa104e51e9c9d--