From dev-return-69496-archive-asf-public=cust-asf.ponee.io@zookeeper.apache.org Tue May 8 17:13:01 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 5175B18063B for ; Tue, 8 May 2018 17:13:00 +0200 (CEST) Received: (qmail 66245 invoked by uid 500); 8 May 2018 15:12:59 -0000 Mailing-List: contact dev-help@zookeeper.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@zookeeper.apache.org Delivered-To: mailing list dev@zookeeper.apache.org Received: (qmail 66233 invoked by uid 99); 8 May 2018 15:12:58 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 08 May 2018 15:12:58 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 2AB051A2DD2 for ; Tue, 8 May 2018 15:12:58 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.229 X-Spam-Level: X-Spam-Status: No, score=0.229 tagged_above=-999 required=6.31 tests=[HEADER_FROM_DIFFERENT_DOMAINS=0.25, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id H-LXS6ztD-6U for ; Tue, 8 May 2018 15:12:56 +0000 (UTC) Received: from mail-wm0-f42.google.com (mail-wm0-f42.google.com [74.125.82.42]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTPS id 29BE55F24A for ; Tue, 8 May 2018 15:12:56 +0000 (UTC) Received: by mail-wm0-f42.google.com with SMTP id j4so19474499wme.1 for ; Tue, 08 May 2018 08:12:56 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:content-transfer-encoding:mime-version :subject:date:references:to:in-reply-to:message-id; bh=iZ1gt74Z7IpCYpJ39kvMR/9T+HnC2RBeyvHl7VBl1EY=; b=Oet3NbLLb3ZAy7E5sbS3IJ9aRiAdR29mWij2CoHMfZoT3eifZFOdbqm3akK1mkAYCo Cw+op6Cwc8oxTPDzEyMpuC89EsSFpn/mAAkAexW6l23djHgirHvH5pM6sLztBOTqOR3U IgZvRXxfTiqFqa1ICtGLHYE4JfLIFoIEKsRAsDW3750LcNUwJY33s3BTHeNW5vIECE81 6/0QdaiyW+K7M2m5hm4BhomMMEDeeuBWfnL02/16jEpPMWrEoU5ST78gdZm//aOBq958 svsUIUmhWcpauTFeWGs/jDozAKhY3+2Ei+rOqehETRuXjLfy1dR9Dw4kpzS+wDEXdBcC H5gg== X-Gm-Message-State: ALKqPweA1TwBe5uUf/4DRb3YfUWASqhTEIvdgTtoxZ2E7yd+50nq3xwD xct8Yir/qF8TmaYoYjDcyMxMLSxU X-Google-Smtp-Source: AB8JxZqquj01k8LlSUWUQTsY88nBFd5xg16EFNjNSf2r+CtDN93H3xQCvYpBfNOfSra21Jzl9YRq1g== X-Received: by 2002:a1c:6f88:: with SMTP id c8-v6mr3519177wmi.9.1525792369851; Tue, 08 May 2018 08:12:49 -0700 (PDT) Received: from [192.168.1.53] (114.red-83-60-144.dynamicip.rima-tde.net. [83.60.144.114]) by smtp.gmail.com with ESMTPSA id m9-v6sm32010420wrf.72.2018.05.08.08.12.48 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 08 May 2018 08:12:49 -0700 (PDT) From: Flavio Junqueira Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: Name resolution in StaticHostProvider Date: Tue, 8 May 2018 17:12:47 +0200 References: To: dev@zookeeper.apache.org In-Reply-To: Message-Id: <4ABFF6D3-35FA-4FF9-8CB1-E82A0C2AB1E4@apache.org> X-Mailer: Apple Mail (2.3273) Hi Andor, Thanks for your work on addressing the issue. > On 8 May 2018, at 16:06, Andor Molnar wrote: >=20 > Hi, >=20 > Updating this thread, because the PR is still being review on GitHub. >=20 > So, the reason why I refactored the original behaviour of > StaticHostProvider is that I believe that it's trying to do something = which > is not its responsibility. If refactoring is necessary to address the issue, then it should be part = of the PR. Otherwise, it is better to refactor in a separate PR. > Please tell me if there's a good historical > reason for that. I'm not sure what kind of confirmation you are after here. Could you = elaborate, please? >=20 > My approach is giving the user the following to options: > 1- Use static IP addresses, if you don't want to deal with DNS = resolution > at all - we guarantee that no DNS logic will involved in this case at = all. Sounds fine. > 2- Use DNS hostnames if you have a reliable DNS service for resolution > (with HA, secondary servers, backups, etc.) - we must use DNS in the = right > way in this case e.g. do NOT cache IP address for a longer period that = DNS > server allows to and re-resolve after TTL expries, because it's = mandatory > by protocol. Sounds fine. >=20 > My 2 cents here: > - the fix which was originally posted for re-resolution is a = workaround and > doesn't satisfy the requirement for #2, The other solution, if I remember enough of it, tried to avoid resolving = unnecessarily, but perhaps the DNS client cache is enough to avoid the = unnecessary round-trips. This observation actually brings me to the next = point: > - the solution is already built-in in JDK and DNS clients in the right = way Could you elaborate on this point, please? What exactly do you mean with = this statement? > - can't see a reason why we shouldn't use that >=20 > I checked this in some other projects as well and found very similar > approach in hadoop-common's SecurityUtil.java. It has 2 built-in = plugins > for that: > - Standard resolver uses java's built-in getByName(). > - Qualified resolver still uses getByName(), but adds some logic to = avoid > incorrect re-resolutions and reverse IP lookups. >=20 > Please let me know your thoughts. >=20 > Regards, > Andor >=20 >=20 >=20 >=20 >=20 >=20 > On Tue, Mar 6, 2018 at 8:12 AM, Andor Molnar = wrote: >=20 >> Hi Abe, >>=20 >> Unfortunately we haven't got any feedback yet. What do you think of >> implementing Option #3? >>=20 >> Regards, >> Andor >>=20 >>=20 >> On Thu, Feb 22, 2018 at 6:06 PM, Andor Molnar = wrote: >>=20 >>> Did anybody happen to take a quick look by any chance? >>>=20 >>> I don't want to push this too hard, because I know it's a time = consuming >>> topic to think about, but this is a blocker in 3.5 which has been = hanging >>> around for a while and any feedback would be extremely helpful to = close it >>> quickly. >>>=20 >>> Thanks, >>> Andor >>>=20 >>>=20 >>>=20 >>> On Mon, Feb 19, 2018 at 12:18 PM, Andor Molnar >>> wrote: >>>=20 >>>> Hi all, >>>>=20 >>>> We need more eyes and brains on the following PR: >>>>=20 >>>> https://github.com/apache/zookeeper/pull/451 >>>>=20 >>>> I added a comment few days ago about the way we currently do DNS = name >>>> resolution in this class and a suggestion on how we could simplify = things a >>>> little bit. We talked about it with Abe Fine, but we're a little = bit unsure >>>> and cannot get a conclusion. It would be extremely handy to get = more >>>> feedback from you. >>>>=20 >>>> To add some colour to it, let me elaborate on the situation here: >>>>=20 >>>> In general, the task that StaticHostProvider does is to get a list = of >>>> potentially unresolved InetSocketAddress objects, resolve them and = iterate >>>> over the resolved objects by calling next() method. >>>>=20 >>>> *Option #1 (current logic)* >>>> - Resolve addresses with getAllByName() which returns a list of IP >>>> addresses associated with the address. >>>> - Cache all these IP's, shuffle them and iterate over. >>>> - If client is unable to connect to an IP, remove all IPs from the = list >>>> which the original servername was resolved to and re-resolve it. >>>>=20 >>>> *Option #2 (getByName())* >>>> - Resolve address with getByName() instead which returns only the = first >>>> IP address of the name, >>>> - Do not cache IPs, >>>> - Shuffle the *names* and resolve with getByName() *every time* = when >>>> next() is called, >>>> - JDK's built-in caching will prevent name servers from being = flooded >>>> and will do the re-resolution automatically when cache expires, >>>> - Names with multiple IPs will be handled by DNS servers which (if >>>> configured properly) return IPs in different order - this is called = DNS >>>> Round Robin -, so getByName() will return different IP on each = call. >>>>=20 >>>> *Options #3* >>>> - There's a small problem with option#2: if DNS server is not = configured >>>> properly and handles the round-robin case in a way that it always = return >>>> the IP list in the same order, getByName() will never return the = next ip, >>>> - In order to overcome that, use getAllByName() instead, shuffle = the >>>> list and return the first IP. >>>>=20 >>>> All feedback if much appreciated. >>>>=20 >>>> Thanks, >>>> Andor >>>>=20 >>>>=20 >>>>=20 >>>>=20 >>>=20 >>=20