zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dan Benediktson <dbenedikt...@twitter.com.INVALID>
Subject Re: shutdown Observer
Date Thu, 09 Mar 2017 18:12:59 GMT
It's also likely you have a fair bit of packet loss between your
datacenters, unless you know you have a solid network between them. If your
observers are falling offline "randomly", packet loss is a pretty likely
culprit.

On Thu, Mar 9, 2017 at 9:54 AM, Michael Han <hanm@cloudera.com> wrote:

> The log indicates that your server socket on observer timed out after
> syncing with leader. It could simply because that the latency between your
> DCs exceeds the socket timeout configuration ZK uses. The timeout is
> calculated as tickTime * syncLimit so you might want tweak these values to
> fit the latency between your DCs.
>
> On Thu, Mar 9, 2017 at 9:00 AM, rammohan ganapavarapu <
> rammohanganap@gmail.com> wrote:
>
> > Hi,
> >
> > We have a multi data-center zk cluster with all the followers are in one
> > data-center and observers in other data-centers, for some reason
> observers
> > are going down with the following exception and i am not sure what could
> be
> > the reason and how to avoid this issue, any thoughts?
> >
> > Ram
> >
> >
> >
> > 2017-03-09 09:00:18,305 - WARN
> > [QuorumPeer[myid=41]/0:0:0:0:0:0:0:0:2181:Observer@79] - Exception when
> > observing the leader
> > java.net.SocketTimeoutException: Read timed out
> >         at java.net.SocketInputStream.socketRead0(Native Method)
> >         at java.net.SocketInputStream.read(SocketInputStream.java:152)
> >         at java.net.SocketInputStream.read(SocketInputStream.java:122)
> >         at java.io.BufferedInputStream.fill(BufferedInputStream.java:
> 235)
> >         at java.io.BufferedInputStream.read(BufferedInputStream.java:
> 254)
> >         at java.io.DataInputStream.readInt(DataInputStream.java:387)
> >         at
> > org.apache.jute.BinaryInputArchive.readInt(BinaryInputArchive.java:63)
> >         at
> > org.apache.zookeeper.server.quorum.QuorumPacket.
> > deserialize(QuorumPacket.java:83)
> >         at
> > org.apache.jute.BinaryInputArchive.readRecord(
> BinaryInputArchive.java:108)
> >         at
> > org.apache.zookeeper.server.quorum.Learner.readPacket(Learner.java:152)
> >         at
> > org.apache.zookeeper.server.quorum.Observer.observeLeader(
> > Observer.java:75)
> >         at
> > org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:727)
> > 2017-03-09 09:00:18,306 - INFO
> > [QuorumPeer[myid=41]/0:0:0:0:0:0:0:0:2181:Observer@137] - shutdown
> called
> > java.lang.Exception: shutdown Observer
> >         at
> > org.apache.zookeeper.server.quorum.Observer.shutdown(Observer.java:137)
> >
>
>
>
> --
> Cheers
> Michael.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message