Return-Path: Delivered-To: apmail-cassandra-user-archive@www.apache.org Received: (qmail 37561 invoked from network); 29 Sep 2010 21:45:04 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 29 Sep 2010 21:45:04 -0000 Received: (qmail 42130 invoked by uid 500); 29 Sep 2010 21:45:02 -0000 Delivered-To: apmail-cassandra-user-archive@cassandra.apache.org Received: (qmail 42089 invoked by uid 500); 29 Sep 2010 21:45:01 -0000 Mailing-List: contact user-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@cassandra.apache.org Delivered-To: mailing list user@cassandra.apache.org Received: (qmail 42077 invoked by uid 99); 29 Sep 2010 21:45:01 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 29 Sep 2010 21:45:01 +0000 X-ASF-Spam-Status: No, hits=2.2 required=10.0 tests=HTML_MESSAGE,MIME_QP_LONG_LINE,RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [208.113.200.5] (HELO homiemail-a47.g.dreamhost.com) (208.113.200.5) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 29 Sep 2010 21:44:56 +0000 Received: from homiemail-a47.g.dreamhost.com (localhost [127.0.0.1]) by homiemail-a47.g.dreamhost.com (Postfix) with ESMTP id 51ABF28406B for ; Wed, 29 Sep 2010 14:44:35 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; c=nofws; d=thelastpickle.com; h=to:from :subject:date:message-id:content-type:mime-version:in-reply-to; q=dns; s=thelastpickle.com; b=LusrB4j7T34rgxEB6Pe40TL9BFCG+NSbp GPbfn27qRtLDMjCTm4ojPvcXfG2CIkW+nSjmZEtB/FHtvAPaa3wMnOQILmJNptw+ C6N2aw8qXyRJRgG6f7Y+jB34X3PjKDmQiPDglDtJt1Jycz1W16CchCLgnTGbiWjw fw5j+LNZ8w= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=thelastpickle.com; h=to :from:subject:date:message-id:content-type:mime-version: in-reply-to; s=thelastpickle.com; bh=Kgn7IIvTkilJvV6BsF4YHj+3pP0 =; b=Xu5YbcyM6k0ZBaYqvIQtkwn/mzv7IzR/akfFcP/ShqF17MATDKWqaMxDOhq 39vIQfWxAppbaRJomSOeFVyMutuUGB9sj2sPdLDjeJgbCk4iucV5cfsznOfI1FNK KsI9kib9QkqBztDLG0ur7nE7ck7ORx1VrBzDAgYf8Pd6TqPs= Received: from localhost (webms.mac.com [17.148.16.116]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: aaron@thelastpickle.com) by homiemail-a47.g.dreamhost.com (Postfix) with ESMTPSA id 3B648284063 for ; Wed, 29 Sep 2010 14:44:35 -0700 (PDT) To: user@cassandra.apache.org From: Aaron Morton Subject: Re: Marking each node down before rolling restart Date: Wed, 29 Sep 2010 21:44:34 GMT X-Mailer: MobileMe Mail (1C3203) Message-id: <99fc6397-74b4-e833-902b-bcd9ec7b9065@me.com> Content-Type: multipart/alternative; boundary=Apple-Webmail-42--2a5fe339-ba78-c0e1-eede-c76f51173491 MIME-Version: 1.0 In-Reply-To: --Apple-Webmail-42--2a5fe339-ba78-c0e1-eede-c76f51173491 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=ISO-8859-1; format=flowed Try nodetool drain=A0=0A=0AFlushes all memtables for a node and causes the= node to stop accepting write operations. Read operations will continue to= work. This is typically used before upgrading a node to a new version of = Cassandra.=0Ahttp://www.riptano.com/docs/0.6.5/utils/nodetool=0A=0AAaron=0A= =0A=0AOn 30 Sep, 2010,at 10:15 AM, Justin Sanders w= rote:=0A=0AI looked through the documentation but couldn't find anything. = =A0I was wondering if there is a way to manually mark a node "down" in the= cluster instead of killing the cassandra process and letting the other no= des figure out the node is no longer up.=0A=0AThe reason I ask is because = we are having an issue when we perform rolling restarts on the cluster. =A0= Basically read requests that come in on other nodes will block while they = are waiting on the node that was just killed to be marked down. =A0Before = they realize the node is offline they will throw a=A0TimedOutException.=0A= =0AIf I could mark the node being down ahead of time this timeout period c= ould be avoided. =A0Any help is appreciated.=0A=0AJustin --Apple-Webmail-42--2a5fe339-ba78-c0e1-eede-c76f51173491 Content-Type: multipart/related; type="text/html"; boundary=Apple-Webmail-86--2a5fe339-ba78-c0e1-eede-c76f51173491 --Apple-Webmail-86--2a5fe339-ba78-c0e1-eede-c76f51173491 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=ISO-8859-1;
Try nodetool drain 

Flushes all memtables for a node and causes the n= ode to stop accepting write operations. Read operations will continue to w= ork. This is typically used before upgrading a node to a new version of Ca= ssandra.
http://www.riptano.com/docs/0.6.5/utils/nodetool
<= br>
Aaron


On 30 Sep, 2010,at = 10:15 AM, Justin Sanders <justin@justinjas.com> wrote:

=
I looked through the documentation but= couldn't find anything.  I was wondering if there is a way to manual= ly mark a node "down" in the cluster instead of killing the cassandra proc= ess and letting the other nodes figure out the node is no longer up.
=0A=
The reason I ask is because we are having an issue when we = perform rolling restarts on the cluster.  Basically read requests tha= t come in on other nodes will block while they are waiting on the node tha= t was just killed to be marked down.  Before they realize the node is= offline they will throw a TimedOutException.
=0A

If I could mark the node being = down ahead of time this timeout period could be avoided.  Any help is= appreciated.
=0A

Justin
=0A=0A
--Apple-Webmail-86--2a5fe339-ba78-c0e1-eede-c76f51173491-- --Apple-Webmail-42--2a5fe339-ba78-c0e1-eede-c76f51173491--