zookeeper-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Norbert Kalmar <nkal...@cloudera.com.INVALID>
Subject Re: Discover LEADER from JMX
Date Thu, 21 Jun 2018 09:57:46 GMT
Good question. Wouldn't just killing the leader do the job?

Regards,
Norbert

On Thu, Jun 21, 2018 at 3:51 AM Enrico Olivelli <eolivelli@gmail.com> wrote:

> This is my patch
> https://github.com/apache/zookeeper/pull/546
>
> One question:
> I would like to force the cluster to change leader in
> HierarchicalQuorumTest, so that I can test that JMX will reflect the new
> status of the group.
> Any idea about how to bounce the leader ?
>
> Cheers
> Enrico
>
> Il giorno mer 20 giu 2018 alle ore 13:45 Enrico Olivelli <
> eolivelli@gmail.com> ha scritto:
>
> > This is my JIRA
> > I am going to work on a patch
> >
> > https://issues.apache.org/jira/browse/ZOOKEEPER-3066
> >
> > Enrico
> >
> > Il gio 10 mag 2018, 19:47 Andor Molnar <andor@cloudera.com> ha scritto:
> >
> >> "in order to guess which is the leader I have to ask to all of the three
> >> nodes in the cluster"
> >>
> >> That's correct.
> >>
> >> Regards,
> >> Andor
> >>
> >>
> >>
> >> On Thu, May 10, 2018 at 4:07 AM, Enrico Olivelli <eolivelli@gmail.com>
> >> wrote:
> >>
> >> > Il giorno mer 9 mag 2018 alle ore 20:24 Patrick Hunt <
> phunt@apache.org>
> >> ha
> >> > scritto:
> >> >
> >> > > iiuc what you are interested in the information is already
> available.
> >> The
> >> > > beans have a "state" attribute which indicates following vs leading.
> >> > >
> >> > > Try attaching a jconsole to the running servers, use the "mbeans"
> tab
> >> and
> >> > > open org.apache.ZooKeeperService -> replicatedserver -> replica
->
> >> > > attributes, you'll see the "state" attribute there.
> >> > >
> >> >
> >> >
> >> > Patrick,
> >> > I can't find this information.
> >> > If I log into a "follower" I get this info only for the 'current
> >> replica'
> >> >
> >> > Example, I have three peers,  the first one is a "Follower", on JMX I
> >> have
> >> > o.a.ZookKeeperService - ReplicatedServer_id1 - replica_1 -
> ClientAddress
> >> > o.a.ZookKeeperService - ReplicatedServer_id1 - replica_1 -
> >> ElectionAddress
> >> > o.a.ZookKeeperService - ReplicatedServer_id1 - replica_1 - LearnerType
> >> > o.a.ZookKeeperService - ReplicatedServer_id1 - replica_1 - Name
> >> > o.a.ZookKeeperService - ReplicatedServer_id1 - replica_1 -
> QuorumAddress
> >> > o.a.ZookKeeperService - ReplicatedServer_id1 - replica_1 - State =
> >> > following
> >> > o.a.ZookKeeperService - ReplicatedServer_id1 - replica_1 -
> ConfigVersion
> >> > .....
> >> >
> >> >
> >> >
> >> > for other peers I see only
> >> > o.a.ZookKeeperService - ReplicatedServer_id1 - replica_2 -
> ClientAddress
> >> > o.a.ZookKeeperService - ReplicatedServer_id1 - replica_2 -
> >> ElectionAddress
> >> > o.a.ZookKeeperService - ReplicatedServer_id1 - replica_2 - LearnerType
> >> > o.a.ZookKeeperService - ReplicatedServer_id1 - replica_2 - Name
> >> > o.a.ZookKeeperService - ReplicatedServer_id1 - replica_2 -
> QuorumAddress
> >> >
> >> > o.a.ZookKeeperService - ReplicatedServer_id1 - replica_3 -
> ClientAddress
> >> > o.a.ZookKeeperService - ReplicatedServer_id1 - replica_3 -
> >> ElectionAddress
> >> > o.a.ZookKeeperService - ReplicatedServer_id1 - replica_3 - LearnerType
> >> > o.a.ZookKeeperService - ReplicatedServer_id1 - replica_3 - Name
> >> > o.a.ZookKeeperService - ReplicatedServer_id1 - replica_3 -
> QuorumAddress
> >> >
> >> >
> >> > so quering only this server I cannot guess which is the current
> leader,
> >> the
> >> > only information I can extract is:
> >> > - I am a follower
> >> > - We are a cluster of three
> >> > - Every of the three is a PARTECIPANT (no observers)
> >> >
> >> > in order to guess which is the leader I have to ask to all of the
> three
> >> > nodes in the cluster
> >> >
> >> > Am I missing something ? I am running 3.5.3-BETA
> >> >
> >> > Enrico
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> >
> >> > >
> >> > > Patrick
> >> > >
> >> > > On Wed, May 9, 2018 at 8:02 AM, Enrico Olivelli <
> eolivelli@gmail.com>
> >> > > wrote:
> >> > >
> >> > > > Thank you Edward
> >> > > >
> >> > > > I will pack all together and send out a patch as soon as I have
> >> time.
> >> > > > I am running 3.5 in production and given than an RC for 3.5.4
is
> >> going
> >> > to
> >> > > > be cut soon I will have to wait for 3.5.5 and I assume it won't
be
> >> > > > immediate.
> >> > > >
> >> > > > Cheers
> >> > > > Enrico
> >> > > >
> >> > > > Il giorno mer 9 mag 2018 alle ore 14:37 Edward Ribeiro <
> >> > > > edward.ribeiro@gmail.com> ha scritto:
> >> > > >
> >> > > > > Sent before finishing the previous email. Only to complement,
> the
> >> > > > > findLeader() could have been as below, but this change is
only a
> >> > nitty
> >> > > > > detail and totally irrelevant to the questions you are asking.
> :)
> >> > > > >
> >> > > > > /**
> >> > > > >  * Returns the address of the node we think is the leader.
> >> > > > >  */
> >> > > > > protected QuorumServer findLeader() {
> >> > > > >
> >> > > > >     // Find the leader by id
> >> > > > >     long currentLeader = self.getCurrentVote().getId();
> >> > > > >
> >> > > > >     QuorumServer leaderServer =
> self.getView().get(currentLeader);
> >> > > > >
> >> > > > >     if (leaderServer == null) {
> >> > > > >         LOG.warn("Couldn't find the leader with id = {}",
> >> > > currentLeader);
> >> > > > >     }
> >> > > > >     return leaderServer;
> >> > > > > }
> >> > > > >
> >> > > > > Edward
> >> > > > >
> >> > > > > On Wed, May 9, 2018 at 9:29 AM, Edward Ribeiro <
> >> > > edward.ribeiro@gmail.com
> >> > > > >
> >> > > > > wrote:
> >> > > > >
> >> > > > > > Hi Enrico,
> >> > > > > >
> >> > > > > > Well, I am not an expert on QuorumPeer either (not
an expert
> on
> >> > > > anything,
> >> > > > > > really), but maybe it's the variable and method below?
> >> > > > > >
> >> > > > > > ----------------- QuorumPeer ------------------
> >> > > > > >
> >> > > > > > /**
> >> > > > > >  * This is who I think the leader currently is.
> >> > > > > >  */
> >> > > > > > volatile private Vote currentVote;
> >> > > > > >
> >> > > > > > public synchronized Vote getCurrentVote(){
> >> > > > > >     return currentVote;
> >> > > > > > }
> >> > > > > >
> >> > > > > > ---------------------------------------
> >> > > > > >
> >> > > > > >
> >> > > > > > Then it's a matter of calling
> >> quorumPeer.getCurrentVote().getId()
> >> > and
> >> > > > > > quorumPeer.getServerState()?
> >> > > > > >
> >> > > > > > Btw, the Learner class has this handy method below
(self is a
> >> > > > > QuorumPeer):
> >> > > > > >
> >> > > > > > ---------------- Learner --------------------
> >> > > > > >
> >> > > > > > /**
> >> > > > > >  * Returns the address of the node we think is the
leader.
> >> > > > > >  */
> >> > > > > > protected QuorumServer findLeader() {
> >> > > > > >     QuorumServer leaderServer = null;
> >> > > > > >     // Find the leader by id
> >> > > > > >     Vote current = self.getCurrentVote();
> >> > > > > >     for (QuorumServer s : self.getView().values())
{
> >> > > > > >         if (s.id == current.getId()) {
> >> > > > > >             leaderServer = s;
> >> > > > > >             break;
> >> > > > > >         }
> >> > > > > >     }
> >> > > > > >     if (leaderServer == null) {
> >> > > > > >         LOG.warn("Couldn't find the leader with id
= "
> >> > > > > >                 + current.getId());
> >> > > > > >     }
> >> > > > > >     return leaderServer;
> >> > > > > > }
> >> > > > > >
> >> > > > > > ---------------------------------------
> >> > > > > >
> >> > > > > > By the way, as a side note, the map traversal could
be changed
> >> by:
> >> > > > > >
> >> > > > > > ----------------------------
> >> > > > > >
> >> > > > > > if (self.getView().contains(current.getId()) {
> >> > > > > >
> >> > > > > > }
> >> > > > > >
> >> > > > > > ---------------------------
> >> > > > > >
> >> > > > > >
> >> > > > > >
> >> > > > > > You can see above this method the quorumPeer.getView()
> returns a
> >> > > > Map<sid,
> >> > > > > > QuorumServer> as below:
> >> > > > > >
> >> > > > > > -------------QuorumPeer ---------
> >> > > > > >
> >> > > > > > /**
> >> > > > > >  * A 'view' is a node's current opinion of the membership
of
> the
> >> > > entire
> >> > > > > >  * ensemble.
> >> > > > > >  */
> >> > > > > > public Map<Long,QuorumPeer.QuorumServer> getView()
{
> >> > > > > >     return Collections.unmodifiableMap(getQuorumVerifier().
> >> > > > > > getAllMembers());
> >> > > > > > }
> >> > > > > >
> >> > > > > > -----------------------------
> >> > > > > >
> >> > > > > >
> >> > > > > > And then it retrieves the QuorumServer that has many
more
> >> > information
> >> > > > > > about the node besides the sid (InetSocketAddress,
hostname,
> >> etc).
> >> > :)
> >> > > > > >
> >> > > > > >
> >> > > > > > Cheers,
> >> > > > > > Edward
> >> > > > > >
> >> > > > > > On Wed, May 9, 2018 at 8:50 AM, Enrico Olivelli <
> >> > eolivelli@gmail.com
> >> > > >
> >> > > > > > wrote:
> >> > > > > >
> >> > > > > >> So I am trying to create a patch in order to expose
on JMX
> the
> >> id
> >> > of
> >> > > > the
> >> > > > > >> current "leader" (on the JVM of a follower)
> >> > > > > >>
> >> > > > > >> I am trying to find in ZK which is the variable
which holds
> >> the ID
> >> > > of
> >> > > > > the
> >> > > > > >> current leader.
> >> > > > > >> I am new to the internal of QuorumPeer
> >> > > > > >>
> >> > > > > >> Can someone give me some hint ?
> >> > > > > >>
> >> > > > > >> Enrico
> >> > > > > >>
> >> > > > > >> Il giorno mar 8 mag 2018 alle ore 10:08 Ansel Zandegran
<
> >> > > > > >> Ansel.Zandegran@infor.com> ha scritto:
> >> > > > > >>
> >> > > > > >> > Hi,
> >> > > > > >> > That is possible with 4 letter commands. We
are using it
> >> now. In
> >> > > > 3.5.x
> >> > > > > >> it
> >> > > > > >> > is going to be removed in favour of admin
server (embedded
> >> web
> >> > > > > server).
> >> > > > > >> > We are running in an environment where it’s
not possible to
> >> run
> >> > > JMX
> >> > > > or
> >> > > > > >> > embedded web servers.
> >> > > > > >> >
> >> > > > > >> > So I am wondering if there is another way?
It would be nice
> >> to
> >> > > have
> >> > > > > this
> >> > > > > >> > info as a znode.
> >> > > > > >> >
> >> > > > > >> > Best regards,
> >> > > > > >> > Ansel
> >> > > > > >> >
> >> > > > > >> > > On 8 May 2018, at 09:55, Flavio Junqueira
<
> fpj@apache.org>
> >> > > wrote:
> >> > > > > >> > >
> >> > > > > >> > > Hi Enrico,
> >> > > > > >> > >
> >> > > > > >> > > You can determine the state of a server
it via 4-letter
> >> > > commands.
> >> > > > > >> Would
> >> > > > > >> > that work for you?
> >> > > > > >> > >
> >> > > > > >> > > -Flavio
> >> > > > > >> > >
> >> > > > > >> > >> On 8 May 2018, at 09:09, Enrico Olivelli
<
> >> > eolivelli@gmail.com>
> >> > > > > >> wrote:
> >> > > > > >> > >>
> >> > > > > >> > >> Hi,
> >> > > > > >> > >> is there any way to see in JMX which
is the leader of a
> >> > > ZooKeeper
> >> > > > > >> > cluster?
> >> > > > > >> > >>
> >> > > > > >> > >> My problem is: given access to any
of the nodes of the
> >> > cluster
> >> > > I
> >> > > > > >> want to
> >> > > > > >> > >> know from JMX which is the current
leader.
> >> > > > > >> > >> It seems to me that this information
is not available,
> you
> >> > can
> >> > > > know
> >> > > > > >> > only if
> >> > > > > >> > >> the local node is Leader or Follower.
> >> > > > > >> > >>
> >> > > > > >> > >> Cheers
> >> > > > > >> > >> Enrico
> >> > > > > >> > >
> >> > > > > >> >
> >> > > > > >> >
> >> > > > > >>
> >> > > > > >
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> > --
> >
> >
> > -- Enrico Olivelli
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message