Return-Path: X-Original-To: apmail-zookeeper-user-archive@www.apache.org Delivered-To: apmail-zookeeper-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 747853942 for ; Wed, 4 May 2011 17:53:31 +0000 (UTC) Received: (qmail 30680 invoked by uid 500); 4 May 2011 17:53:30 -0000 Delivered-To: apmail-zookeeper-user-archive@zookeeper.apache.org Received: (qmail 30654 invoked by uid 500); 4 May 2011 17:53:30 -0000 Mailing-List: contact user-help@zookeeper.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@zookeeper.apache.org Delivered-To: mailing list user@zookeeper.apache.org Received: (qmail 30646 invoked by uid 99); 4 May 2011 17:53:30 -0000 Received: from minotaur.apache.org (HELO minotaur.apache.org) (140.211.11.9) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 04 May 2011 17:53:30 +0000 Received: from localhost (HELO mail-iw0-f170.google.com) (127.0.0.1) (smtp-auth username phunt, mechanism plain) by minotaur.apache.org (qpsmtpd/0.29) with ESMTP; Wed, 04 May 2011 17:53:30 +0000 Received: by iwn3 with SMTP id 3so1411977iwn.15 for ; Wed, 04 May 2011 10:53:29 -0700 (PDT) MIME-Version: 1.0 Received: by 10.42.230.69 with SMTP id jl5mr271353icb.108.1304531609741; Wed, 04 May 2011 10:53:29 -0700 (PDT) Received: by 10.42.98.77 with HTTP; Wed, 4 May 2011 10:53:29 -0700 (PDT) In-Reply-To: References: Date: Wed, 4 May 2011 10:53:29 -0700 Message-ID: Subject: Re: observers in occasionally disconnected data centers From: Patrick Hunt To: user@zookeeper.apache.org Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable This is odd, it's failing in the c tests but for a weird reason: in: https://builds.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/247/artifact= /trunk/build/tmp/zk.log it says: /grid/0/hudson/hudson-slave/workspace/PreCommit-ZOOKEEPER-Build/trunk/src/c= /tests/zkServer.sh: line 115: java: command not found I'll ping the hudson admins and see if this is a known issue (also hudson is very slow today for some reason). Once that's addressed we should be good to go. Patrick On Wed, May 4, 2011 at 9:57 AM, Ketan Gangatirkar wrote: > Got the patch formatted right and applying successfully, now I'll see > if I can figure out the unit test failure. > > On Wed, May 4, 2011 at 11:26 AM, Patrick Hunt wrote: >> Hi Ketan, the patch is failing to apply >> https://builds.apache.org/hudson/job/PreCommit-ZOOKEEPER-Build/246//cons= ole >> >> Looks like you used git, I usually do something like: >> git diff rev1..rev2 --no-prefix > ZOOKEEPER-784.patch >> can you give it another try? >> >> Patrick >> >> On Tue, May 3, 2011 at 6:42 PM, Ketan Gangatirkar wro= te: >>> I have updated Sergey's patch to: >>> >>> * apply to current trunk >>> * incorporate one trivial output change he made to StatCommand in >>> NettyServerCnxn.java >>> * change log4j references to slf4j >>> >>> I have successfully run ant releaseaudit on the result. =A0The updated >>> patch is now attached to the issue: >>> >>> https://issues.apache.org/jira/browse/ZOOKEEPER-784 >>> >>> I do *not* make any claim to have understood the contents of this >>> patch; all I did was synch everything and fix the obvious log4j/slf4j >>> change. =A0Now what? >>> >>> >>> On Tue, May 3, 2011 at 5:46 PM, Patrick Hunt wrote: >>>> The core tests failed on last hudson, I just kicked off a patch build, >>>> seems recent changes (logging?) have caused the patch to stop >>>> applying: >>>> https://hudson.apache.org/hudson/view/S-Z/view/ZooKeeper/job/PreCommit= -ZOOKEEPER-Build/238/console >>>> >>>> Ketan would you like to try updating the patch and resubmit? >>>> >>>> Patrick >>>> >>>> On Tue, May 3, 2011 at 3:31 PM, Ketan Gangatirkar w= rote: >>>>> Thanks, Mahadev. =A0I had seen ZOOKEEPER-892 but not ZOOKEEPER-784. = =A0The >>>>> latter may be what we need. >>>>> >>>>> I read the comments attached to that issue. =A0The most recent commen= t >>>>> was a Hudson CI message indicating that the tests against the patch >>>>> failed. =A0I was not able to find out more as it appears that the >>>>> configuration of the Apache Hudson has changed. =A0It appears that th= e >>>>> patch was approved but not merged into trunk, and it's now in limbo. >>>>> What is necessary to get that feature into the next release? =A0I may= be >>>>> able to assist, depending on what's involved. =A0Thank you. >>>>> >>>>> >>>>> On Tue, May 3, 2011 at 4:17 PM, Mahadev Konar wr= ote: >>>>>> Hi Ketan, >>>>>> =A0You are correct that observers need connection to quorum as well. >>>>>> There have been quite a few discussions on multi colo replication an= d >>>>>> read only mode of ZooKeeper. >>>>>> >>>>>> Here are the jiras for those: >>>>>> >>>>>> https://issues.apache.org/jira/browse/ZOOKEEPER-784 >>>>>> and >>>>>> https://issues.apache.org/jira/browse/ZOOKEEPER-892 >>>>>> >>>>>> These have been mostly targeted at exactly a use case like yours. >>>>>> Please take a look and them and feel free to contribute/comment on t= he >>>>>> jiras. >>>>>> >>>>>> -- >>>>>> thanks >>>>>> mahadev >>>>>> @mahadevkonar >>>>>> >>>>>> >>>>>> >>>>>> On Tue, May 3, 2011 at 2:07 PM, Ketan Gangatirkar = wrote: >>>>>>> Hi. =A0We're considering ZooKeeper for coordinating operations acro= ss >>>>>>> multiple data centers. =A0These data centers will occasionally be >>>>>>> disconnected. =A0We were planning on using observers in remote data >>>>>>> centers. =A0Our applications can survive being unable to *write* to >>>>>>> ZooKeeper, but they do need to be able to read from it, even if the >>>>>>> data were stale. >>>>>>> >>>>>>> On further examination, it looks like observers must always be >>>>>>> connected to the quorum to function at all. =A0Is this correct? =A0= Does >>>>>>> anyone have suggestions for how to work around this problem? =A0The >>>>>>> first thing that comes to mind is duplicating the required data in >>>>>>> some other local data store and falling back on that when the DC >>>>>>> becomes disconnected. =A0I imagine the disadvantages of that are ob= vious >>>>>>> to everyone. =A0I hope someone can share some great idea that allow= s me >>>>>>> to avoid that miserable fate. =A0Thanks. >>>>>>> >>>>>>> -- >>>>>>> Ketan Gangatirkar >>>>>>> ketan@indeed.com >>>>>>> >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Ketan Gangatirkar >>>>> ketan@indeed.com >>>>> Perishable Developer >>>>> >>>> >>> >>> >>> >>> -- >>> Ketan Gangatirkar >>> ketan@indeed.com >>> Perishable Developer >>> >> > > > > -- > Ketan Gangatirkar > ketan@indeed.com > Perishable Developer >