zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From rammohan ganapavarapu <rammohanga...@gmail.com>
Subject Re: Zookeeper resolving to old host IP addresses
Date Wed, 22 Jan 2020 18:37:09 GMT
Hi Andor,

On OS side the  hostname resolves to new IP, so it could be JVM is the one
caching. Any setting son jvm to invalidate cache? In some other posts i did
see some one recommending these but not sure it it works.

-Dnetworkaddress.cache.ttl=0

-Dnetworkaddress.cache.negative.ttl=0

Ram

On Wed, Jan 22, 2020 at 2:03 AM Andor Molnar <andor@apache.org> wrote:

> Hi Ram,
>
> As far as I can see from the code, ZooKeeper uses the standard Java calls
> getByName() and getAllByName() every time it’s trying to connect to a
> server.
>
> ————————
> // zookeeper.ipReachableTimeout is not defined
> if (ipReachableTimeout <= 0) {
>     address = InetAddress.getByName(this.hostname);
> } else {
>     address = getReachableAddress(this.hostname, ipReachableTimeout);
> }
> ————————
>
> ZK doesn’t (and definitely should not) cache IP addresses. It’s either the
> cache of JVM or your DNS server.
>
> Dynamic reconfig is available in 3.5.x versions which are already stable
> now and I think with that you don’t need to reuse existing hostnames.
> Instead use reconfig commands to properly remove old nodes and add new
> ones. Sounds like more cumbersome, but maybe more reliable.
>
> Andor
>
>
>
>
> > On 2020. Jan 21., at 23:14, rammohan ganapavarapu <
> rammohanganap@gmail.com> wrote:
> >
> > But still happening for me, is there any config on zookeeper side to make
> > this fix to work?
> >
> > Ram
> >
> > On Tue, Jan 21, 2020 at 2:12 PM Michael Han <hanm@apache.org> wrote:
> >
> >> Could be ZOOKEEPER-1506, though this should be fixed already in 3.4.14.
> >>
> >> On Tue, Jan 21, 2020 at 2:01 PM rammohan ganapavarapu <
> >> rammohanganap@gmail.com> wrote:
> >>
> >>> Hi Enrico,
> >>>
> >>> I see same with both 3.4.5 and 3.4.14
> >>>
> >>> Ram
> >>>
> >>> On Tue, Jan 21, 2020 at 1:53 PM Enrico Olivelli <eolivelli@gmail.com>
> >>> wrote:
> >>>
> >>>> Hi,
> >>>> Which version of ZK are you using?
> >>>> Enrico
> >>>>
> >>>>
> >>>> Il mar 21 gen 2020, 22:51 rammohan ganapavarapu <
> >> rammohanganap@gmail.com
> >>>>
> >>>> ha scritto:
> >>>>
> >>>>> Hi,
> >>>>>
> >>>>> Does zookeeper cache the host IP? if so how long does it cache?
I
> >> have
> >>> a
> >>>>> zk cluster in autoscaling groups and when a new node comes up, other
> >>>> nodes
> >>>>> still resolving to old IP. Is there any setting to invalidate dns
> >> cache
> >>>> for
> >>>>> zookeeper? or is it jvm dns cache? until other nodes resolves to
new
> >>> IP,
> >>>>> this node not able to join the cluster.
> >>>>>
> >>>>> Thanks,
> >>>>> Ram
> >>>>>
> >>>>
> >>>
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message