Return-Path: X-Original-To: apmail-curator-user-archive@minotaur.apache.org Delivered-To: apmail-curator-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 150E2111D9 for ; Fri, 19 Sep 2014 20:31:31 +0000 (UTC) Received: (qmail 48376 invoked by uid 500); 19 Sep 2014 20:31:31 -0000 Delivered-To: apmail-curator-user-archive@curator.apache.org Received: (qmail 48334 invoked by uid 500); 19 Sep 2014 20:31:30 -0000 Mailing-List: contact user-help@curator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@curator.apache.org Delivered-To: mailing list user@curator.apache.org Received: (qmail 48324 invoked by uid 99); 19 Sep 2014 20:31:30 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 19 Sep 2014 20:31:30 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_NEUTRAL X-Spam-Check-By: apache.org Received-SPF: neutral (athena.apache.org: local policy) Received: from [209.85.192.180] (HELO mail-pd0-f180.google.com) (209.85.192.180) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 19 Sep 2014 20:31:26 +0000 Received: by mail-pd0-f180.google.com with SMTP id r10so599040pdi.11 for ; Fri, 19 Sep 2014 13:31:05 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:date:from:to:message-id:in-reply-to:references :subject:mime-version:content-type; bh=Cquc6KKPg0xBg8RHUG9wIon2IxZYLhoheLHtZDtw00Q=; b=Jk3p9EJAM4LXtyCF0q8CmeSJCr+0VhTqjPE9S3QT+4u9k3PNWk9ycUetiW+8aDAP5K Kyc4z+nnALEgc2byYN7YH/F7apGzq4rxFW5uGno/qFBaB9rfdTouzE7Vj2BdfaZEb+DH 46DZ/uZYpkULdHQnhZae7gEF8Tk02O/mWU9OSQ7jTtHQHEpiT8yRnBDc2imkQYPtlyJE LbRfg8QE+eAYUQsZ7OSnd6lJvhy/0fnueJXHeifPv6v/SSbcI9UNgNfQe+dqq2QDC+jN MkLY2fChB9XnsSMhXswHICL5fAtdKWInvHZ+O+XZKwqnKpGDaDL7F5lrNP5M9n0PKfRx wF/Q== X-Gm-Message-State: ALoCoQkAGRxVdaOmmS29zscS/RqltYT9IO0TTgClj7k6EkoKU39sSpQpIlzRtjwtd0pc3lWGrO1/ X-Received: by 10.66.155.105 with SMTP id vv9mr4654632pab.61.1411158665062; Fri, 19 Sep 2014 13:31:05 -0700 (PDT) Received: from Jordans-MacBook-Pro.local ([190.141.38.38]) by mx.google.com with ESMTPSA id hs8sm2651966pbc.35.2014.09.19.13.31.03 for (version=SSLv3 cipher=RC4-SHA bits=128/128); Fri, 19 Sep 2014 13:31:04 -0700 (PDT) Date: Fri, 19 Sep 2014 15:31:02 -0500 From: Jordan Zimmerman To: Purshotam Shah , "=?utf-8?Q?user=40curator.apache.org?=" Message-ID: In-Reply-To: References: Subject: Re: DistributedAtomicLong fails in multithread env. X-Mailer: Airmail (247) MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="541c9286_b37e80a_116" X-Virus-Checked: Checked by ClamAV on apache.org --541c9286_b37e80a_116 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Are you using a=C2=A0PromotedToLock=3F Combined with a reasonable retry i= t should make failures almost never happen. You can also just set the num= ber of retries to a huge number. -Jordan On September 19, 2014 at 12:38:52 PM, Purshotam Shah (purushah=40yahoo-in= c.com) wrote: Hi=C2=A0Jordan, Same issue with Curator 2.5.0. I read one of your mail thread (can't find it now) where you said that Di= stributedAtomicLong is not guarantee to succeed in multithread env, we ha= ve to keep on trying until it succeed. Is that true=3F This is becoming bottleneck in our stress testing. When t= ry to call DistributedAtomicLong =C2=A0concurrently from multiple thread = (30 thread), we see few failures (with retry policy =22ExponentialBackoff= Retry(1000, 3))=22. What is the best way to guarantee that DistributedAtomicLong will always = succeed=3F Thanks, Puru =46rom: Purshotam Shah Date: Wednesday, June 25, 2014 at 4:12 PM To: Jordan Zimmerman , =22user=40curator.ap= ache.org=22 Subject: Re: DistributedAtomicLong fails in multithread env. Thanks.=C2=A0 Looks like it=E2=80=99s working fine with=C2=A0Curator 2.5.0. Will do some more testing. Will respond if it fails with 2.5.0. Thanks, Puru. =46rom: Jordan Zimmerman Date: Tuesday, June 24, 2014 at 6:38 PM To: Purshotam Shah , =22user=40curator.apache.o= rg=22 Subject: Re: DistributedAtomicLong fails in multithread env. This sounds like=C2=A0https://issues.apache.org/jira/browse/CURATOR-108=C2= =A0- Curator 2.5.0 added a new method, initialize(), to work around this = issue. Please try that and let me know. -Jordan =46rom:=C2=A0Purshotam Shah purushah=40yahoo-inc.com Reply:=C2=A0user=40curator.apache.orguser=40curator.apache.org Date:=C2=A0June 24, 2014 at 8:20:18 PM To:=C2=A0user=40curator.apache.orguser=40curator.apache.org Subject:=C2=A0 DistributedAtomicLong fails in multithread env. We are using DistributedAtomicLong to use job sequenceID in ZK. We noticed that getZKId in multithread env fails. value.preValue() and va= lue.postValue() value =3D 0 and succeeded =3D false. If we synchronized the function it works fine, but I don't think it's a r= ight approach. Other approach is to retry multiple time, but how many times. We need to = make sure that getZKId return sequence. What is the best approach=3F =C2=A0 =C2=A0 DistributedAtomicLong atomicIdGenerator; =C2=A0 =C2=A0 PromotedToLock.Builder lockBuilder =3D PromotedToLock.build= er() =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 .lo= ckPath(getPromotedLock()).retryPolicy(ZKUtils.getRetryPloicy()) =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 .ti= meout(Service.lockTimeout, TimeUnit.MILLISECONDS); =C2=A0 =C2=A0 =C2=A0atomicIdGenerator =3D new DistributedAtomicLong(zk.ge= tClient(), ZK=5FSEQUENCE=5FPATH, ZKUtils.getRetryPloicy(), =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 loc= kBuilder.build()); =C2=A0 =C2=A0 private =C2=A0long getZKId( ) =7B =C2=A0 =C2=A0 =C2=A0 =C2=A0 if (atomicIdGenerator =3D=3D null) =7B =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 throw new RuntimeException(=22S= equence generator can't be null. Path : =22 + ZK=5FSEQUENCE=5FPATH); =C2=A0 =C2=A0 =C2=A0 =C2=A0 =7D =C2=A0 =C2=A0 =C2=A0 =C2=A0 AtomicValue value =3D null; =C2=A0 =C2=A0 =C2=A0 =C2=A0 try =7B =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 value =3D atomicIdGenerator.inc= rement(); =C2=A0 =C2=A0 =C2=A0 =C2=A0 =7D =C2=A0 =C2=A0 =C2=A0 =C2=A0 catch (Exception e) =7B =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 throw new RuntimeException(=22E= xception incrementing UID for session =22, e); =C2=A0 =C2=A0 =C2=A0 =C2=A0 =7D =C2=A0 =C2=A0 =C2=A0 =C2=A0 finally =7B =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 if (value =21=3D null && value.= succeeded()) =7B =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 return value.preV= alue(); =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =7D =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 else =7B =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 throw new Runtime= Exception(=22Exception incrementing UID for session =22); =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =7D =C2=A0 =C2=A0 =C2=A0 =C2=A0 =7D =C2=A0 =C2=A0 =7D --541c9286_b37e80a_116 Content-Type: text/html; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline