zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Hunt <ph...@apache.org>
Subject Re: How to modify Client Connection timer
Date Mon, 27 Mar 2017 23:13:14 GMT
On Wed, Mar 22, 2017 at 2:19 PM, Ray Chaudhuri, Shirsha (Nokia -
IN/Bangalore) <shirsha.ray_chaudhuri@nokia.com> wrote:

>
>
> Thanks Patrick.
>
> Yes, we did eventually figure out that this timeout value is initialised
> from the Hadoop ZKFC code with the value of "ha.zookeeper.session-timeout.
> ms" or if this is not set, then the default is 5000. Since the client
> Cnxn class sets this timer value to a value _  (session timeout) / (number
> of ZK servers), the timer value for the initial connection to complete is
> one third the value of 5000. Hence the timeout occurring at around 1.6s
> each time.
>
> Which brought us to the question that wouldn't this piece of code
> highlighted from the file zookeeper/ClientCnxn.java _
>     public ClientCnxn(String chrootPath, HostProvider hostProvider, int
> sessionTimeout, ZooKeeper zooKeeper,
>
>
>             ClientWatchManager watcher, ClientCnxnSocket clientCnxnSocket,
>
>
>             long sessionId, byte[] sessionPasswd, boolean canBeReadOnly) {
>
>
>         this.zooKeeper = zooKeeper;
>
>
>         this.watcher = watcher;
>
>
>         this.sessionId = sessionId;
>
>
>         this.sessionPasswd = sessionPasswd;
>
>
>         this.sessionTimeout = sessionTimeout;
>
>
>         this.hostProvider = hostProvider;
>
>
>         this.chrootPath = chrootPath;
>
>
>
>
>
>         connectTimeout = sessionTimeout / hostProvider.size();
>
>
>         readTimeout = sessionTimeout * 2 / 3;
>
>
>         readOnly = canBeReadOnly;
>
>
>
>
>
>         sendThread = new SendThread(clientCnxnSocket);
>
>
>         eventThread = new EventThread();
>
>
>         this.clientConfig=zooKeeper.getClientConfig();
>
>
>     }
>
>
>
>
> Be leading to trouble if the number of ZK Servers is larger?
>
> This timeout value is initially used when the Client is waiting to connect
> with the server, even before it has negotiated a sessiontimeout value.
>
> So larger the number of ZK Servers, less will be the timer value.
> Shouldn’t we have a lower guard for it and ensure that atleast a min value
> is used for this timer?
>
>
Probably a good idea. I've seen some terrible thundering herd problems that
result from this very basic logic. Some form of exponential backoff would
probably even be better. (configurable?) That said I've never seen the size
of the ensemble be that much of an issue given the other issue.

You might also check the latest trunk, can't remember if someone's been
trying to look at this recently...

Regards,

Patrick


>
>
> Regards
>
> Shirsha
>
>
>
> -----Original Message-----
> From: Patrick Hunt [mailto:phunt@apache.org]
> Sent: Wednesday, March 22, 2017 11:55 PM
> To: UserZooKeeper <user@zookeeper.apache.org>
> Subject: Re: How to modify Client Connection timer
>
>
>
> You should be able to control that by increasing the session timeout. I'm
> not familiar with the client code you are using however. Additionally the
> ZK client should retry it's connection (as long as you don't close the ZK
>
> object) continually until it is able to reconnect.
>
>
>
> Patrick
>
>
>
> On Mon, Mar 20, 2017 at 3:22 PM, Ray Chaudhuri, Shirsha (Nokia -
>
> IN/Bangalore) <shirsha.ray_chaudhuri@nokia.com<mailto:shirsha.ray_
> chaudhuri@nokia.com>> wrote:
>
>
>
> > Hi,
>
> >
>
> > We are using ZKFC  as a Zookeeper Client that tries to connect to
>
> > Zookeeper Server at the time of bringup, when we sometimes encounter
>
> > the following issue _
>
> >
>
> > 1.       Client retrieves the ZK Ensemble address (ensemble consists of 3
>
> > nodes)
>
> >
>
> > 2.       Client tries to connect to one of the ZK Server nodes
>
> >
>
> > 3.       However due to other on-going processing (required at the time
> of
>
> > bringup) at the ZK Server Node, it fails to respond in time to the
> Client.
>
> > The response goes after 3 seconds
>
> >
>
> > 4.       The Client side times out by then, typically seen around 1.6 sec
>
> >
>
> > Is there a way this timer at the Client side can be reconfigured to a
>
> > higher value?
>
> >
>
> > Regards
>
> > Shirsha
>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message