Return-Path: Delivered-To: apmail-cassandra-client-dev-archive@minotaur.apache.org Received: (qmail 15720 invoked from network); 1 Sep 2010 02:19:22 -0000 Received: from unknown (HELO mail.apache.org) (140.211.11.3) by 140.211.11.9 with SMTP; 1 Sep 2010 02:19:22 -0000 Received: (qmail 20504 invoked by uid 500); 1 Sep 2010 02:19:22 -0000 Delivered-To: apmail-cassandra-client-dev-archive@cassandra.apache.org Received: (qmail 20487 invoked by uid 500); 1 Sep 2010 02:19:22 -0000 Mailing-List: contact client-dev-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: client-dev@cassandra.apache.org Delivered-To: mailing list client-dev@cassandra.apache.org Received: (qmail 20479 invoked by uid 99); 1 Sep 2010 02:19:22 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Sep 2010 02:19:22 +0000 X-ASF-Spam-Status: No, hits=2.2 required=10.0 tests=HTML_MESSAGE,MIME_QP_LONG_LINE,RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: local policy) Received: from [208.113.200.5] (HELO homiemail-a58.g.dreamhost.com) (208.113.200.5) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Sep 2010 02:19:17 +0000 Received: from homiemail-a58.g.dreamhost.com (localhost [127.0.0.1]) by homiemail-a58.g.dreamhost.com (Postfix) with ESMTP id 65F1C7D806A for ; Tue, 31 Aug 2010 19:18:56 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; c=nofws; d=thelastpickle.com; h=to:from :subject:date:message-id:content-type:mime-version:in-reply-to; q=dns; s=thelastpickle.com; b=cvK3aP0jwTTlLbP3w3VsZQC20ALAJXCM4 KG5Ndl039/ZfHUgFLxMSXRuEtwIuaiUj0NQfMJLhrqUxuhFtPRiuQUvE0IbQ1Ath MlnNDpyQM2kX2CI1MZsCRs/UQ/alSjXbeb3TGhyMyCpc+ZuZIVwKH+WGnTdUONVM g8gGvn1LpA= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=thelastpickle.com; h=to :from:subject:date:message-id:content-type:mime-version: in-reply-to; s=thelastpickle.com; bh=Jgup5OAsezz2fx2wG4tgbfGEUsU =; b=JBjZdSVPdXgTv+zru2q5c/088pv1V6y/ruVL1vl+fE+mTTkzHldLuoT42kB g3u2+pxOXuXvsJKOlAtQn/AC/pmCaB4IDlrPK85HlN7yYdNpO09em0/Vxs2PK3yJ rk26VJN/z5aIyEfhJ41KxLflyr/xTEAV/mhvfcS9/Mi9qJVM= Received: from localhost (webms.mac.com [17.148.16.116]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: aaron@thelastpickle.com) by homiemail-a58.g.dreamhost.com (Postfix) with ESMTPSA id 433287D8063 for ; Tue, 31 Aug 2010 19:18:56 -0700 (PDT) To: client-dev@cassandra.apache.org From: Aaron Morton Subject: Re: Detecting cluster membership Date: Wed, 01 Sep 2010 02:18:55 GMT X-Mailer: MobileMe Mail (1C3202) Message-id: Content-Type: multipart/alternative; boundary=Apple-Webmail-42--f62c8d6e-4302-cce4-fd3a-945729297509 MIME-Version: 1.0 In-Reply-To: --Apple-Webmail-42--f62c8d6e-4302-cce4-fd3a-945729297509 Content-Transfer-Encoding: quoted-printable Content-Type: text/plain; charset=ISO-8859-1; format=flowed I agree.=A0=0A=0ARound robin DNS is just the right approach, when doing th= ings like draining a node or turning a box off.=A0=0A=0AAm moving to use d= escribe_ring and a either a seed list or dns. My concern with using a seed= list in the client is the chance that the seeds are down, I have 2 seeds = in a 4 node cluster so with both seeds down the cluster should still have = quorum.=A0=0A=0ASo may go with regular round robin DNS when the client nee= ds refresh it's list of nodes, e.g. when starting up.=A0=0A=0AHave the cli= ent hold a list of the up nodes returned from describe_ring(), that it shu= ffles and then round robins. The list would be refreshed periodically.=0A=0A= I also have the client periodically obtain a new connection to avoid the c= onnections getting clumped in one area of the ring.=A0=0A=0A(I'm working o= n an in house Python client that I hope to make public). =A0=A0=0A=0AAaron= =0A=0AOn 01 Sep, 2010,at 02:04 PM, Dan Washusen wrote:=0A= =0AThe Pelops provides a connection pooling impl that's using (or attempti= ng to=0Ause) the second approach, but to be honest it needs a significant = amount of=0Atesting before I'd be willing to go into production with it...= =0A=0AIMO, the connection pooling/node failure/etc logic is by far the mos= t=0Acomplex part of a client library. It would be excellent if we could av= oid=0Are-inventing the wheel when attempting to create a solution to solve= it.=0A=0ACheers,=0ADan=0A=0AOn Wed, Sep 1, 2010 at 11:35 AM, Aaron Morton= wrote:=0A=0A> When I first started writing code = against the thrift API the FAQ=0A> recommended using a round robin DNS to = select nodes=0A> http://wiki.apache.org/cassandra/FAQ#node_clients_connect= _to=0A>=0A> = The other=0A> day Ben said something like "well behaved clients use descri= be_ring to keep=0A> track of running nodes".=0A> http://www.mail-archive.c= om/user@cassandra.apache.org/msg05588.html=0A>=0A> So am wondering what ap= proach people are taking to detecting cluster=0A> membership.=0A>=0A> 1. R= ound Robin=0A>=0A> 2. List seeds in app config, connect to a seed, use des= cribe_ring.=0A>=0A> 3. Round robin and describe_ring=0A>=0A> One issue I'v= e found with round robin, is that is the machine is powered=0A> off it can= take a while for the network to work out there is no ARP for the=0A> IP. = This may just be a result of the network here, have not looked into it=0A>= too far.=0A>=0A> cheers=0A> Aaron=0A>=0A --Apple-Webmail-42--f62c8d6e-4302-cce4-fd3a-945729297509 Content-Type: multipart/related; type="text/html"; boundary=Apple-Webmail-86--f62c8d6e-4302-cce4-fd3a-945729297509 --Apple-Webmail-86--f62c8d6e-4302-cce4-fd3a-945729297509 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=ISO-8859-1;
I agree. 

Round robin DNS is just the r= ight approach, when doing things like draining a node or turning a box off=  

Am moving to use describe_ring and a eith= er a seed list or dns. My concern with using a seed list in the client is = the chance that the seeds are down, I have 2 seeds in a 4 node cluster so = with both seeds down the cluster should still have quorum. 

So may go with regular round robin DNS when the client nee= ds refresh it's list of nodes, e.g. when starting up. 

=
Have the client hold a list of the up nodes returned from descr= ibe_ring(), that it shuffles and then round robins. The list would be refr= eshed periodically.

I also have the client period= ically obtain a new connection to avoid the connections getting clumped in= one area of the ring. 

(I'm working on an i= n house Python client that I hope to make public).   
=
Aaron

On 01 Sep, 2010,at 02:04 PM, Dan Washus= en <dan@reactive.org> wrote:

The Pelops provides a connection pooling= impl that's using (or attempting to
=0Ause) the second approach, but t= o be honest it needs a significant amount of
=0Atesting before I'd be w= illing to go into production with it...
=0A
=0AIMO, the connection p= ooling/node failure/etc logic is by far the most
=0Acomplex part of a c= lient library. It would be excellent if we could avoid
=0Are-inventing= the wheel when attempting to create a solution to solve it.
=0A
=0A= Cheers,
=0ADan
=0A
=0AOn Wed, Sep 1, 2010 at 11:35 AM, Aaron Mort= on <aaron@thelastpickle.com>wrote:
=0A
=0A> When I first st= arted writing code against the thrift API the FAQ
=0A> recommended u= sing a round robin DNS to select nodes
=0A> http://wiki.apache.org/c= assandra/FAQ#node_clients_connect_to
=0A>
=0A> <http://= wiki.apache.org/cassandra/FAQ#node_clients_connect_to>The other
= =0A> day Ben said something like "well behaved clients use describe_rin= g to keep
=0A> track of running nodes".
=0A> http= ://www.mail-archive.com/user@cassandra.apache.org/msg05588.html
=0A= >
=0A> So am wondering what approach people are taking to detecti= ng cluster
=0A> membership.
=0A>
=0A> 1. Round Robin
= =0A>
=0A> 2. List seeds in app config, connect to a seed, use des= cribe_ring.
=0A>
=0A> 3. Round robin and describe_ring
=0A&= gt;
=0A> One issue I've found with round robin, is that is the machi= ne is powered
=0A> off it can take a while for the network to work o= ut there is no ARP for the
=0A> IP. This may just be a result of the= network here, have not looked into it
=0A> too far.
=0A>
=0A= > cheers
=0A> Aaron
=0A>
=0A
--Apple-Webmail-86--f62c8d6e-4302-cce4-fd3a-945729297509-- --Apple-Webmail-42--f62c8d6e-4302-cce4-fd3a-945729297509--