zookeeper-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From enixon <...@git.apache.org>
Subject [GitHub] zookeeper pull request #628: ZOOKEEPER-3140: Allow Followers to host Observe...
Date Wed, 07 Nov 2018 02:38:23 GMT
Github user enixon commented on a diff in the pull request:

    https://github.com/apache/zookeeper/pull/628#discussion_r231364616
  
    --- Diff: zookeeper-server/src/main/java/org/apache/zookeeper/server/quorum/ObserverMaster.java
---
    @@ -0,0 +1,514 @@
    +/**
    + * Licensed to the Apache Software Foundation (ASF) under one
    + * or more contributor license agreements.  See the NOTICE file
    + * distributed with this work for additional information
    + * regarding copyright ownership.  The ASF licenses this file
    + * to you under the Apache License, Version 2.0 (the
    + * "License"); you may not use this file except in compliance
    + * with the License.  You may obtain a copy of the License at
    + *
    + *     http://www.apache.org/licenses/LICENSE-2.0
    + *
    + * Unless required by applicable law or agreed to in writing, software
    + * distributed under the License is distributed on an "AS IS" BASIS,
    + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
    + * See the License for the specific language governing permissions and
    + * limitations under the License.
    + */
    +
    +package org.apache.zookeeper.server.quorum;
    +
    +import org.apache.zookeeper.jmx.MBeanRegistry;
    +import org.apache.zookeeper.server.Request;
    +import org.apache.zookeeper.server.ZKDatabase;
    +
    +import java.io.BufferedInputStream;
    +import java.io.ByteArrayInputStream;
    +import java.io.DataInputStream;
    +import java.io.IOException;
    +import java.net.ServerSocket;
    +import java.net.Socket;
    +import java.net.SocketAddress;
    +import java.util.ArrayList;
    +import java.util.Arrays;
    +import java.util.Collections;
    +import java.util.Iterator;
    +import java.util.Set;
    +import java.util.concurrent.ConcurrentHashMap;
    +import java.util.concurrent.ConcurrentLinkedQueue;
    +import java.util.concurrent.Executors;
    +import java.util.concurrent.ScheduledExecutorService;
    +import java.util.concurrent.TimeUnit;
    +import java.util.concurrent.atomic.AtomicLong;
    +
    +import org.apache.zookeeper.server.quorum.auth.QuorumAuthServer;
    +import org.slf4j.Logger;
    +import org.slf4j.LoggerFactory;
    +
    +/**
    + * Used by Followers to host Observers. This reduces the network load on the Leader process
by pushing
    + * the responsibility for keeping Observers in sync off the leading peer.
    + *
    + * It is expected that Observers will continue to perform the initial vetting of clients
and requests.
    + * Observers send the request to the follower where it is received by an ObserverMaster.
    + *
    + * The ObserverMaster forwards a copy of the request to the ensemble Leader and inserts
it into its own
    + * request processor pipeline where it can be matched with the response comes back. All
commits received
    + * from the Leader will be forwarded along to every Learner connected to the ObserverMaster.
    + *
    + * New Learners connecting to a Follower will receive a LearnerHandler object and be
party to its syncing logic
    + * to be brought up to date.
    + *
    + * The logic is quite a bit simpler than the corresponding logic in Leader because it
only hosts observers.
    + */
    +public class ObserverMaster implements LearnerMaster, Runnable {
    +    private static final Logger LOG = LoggerFactory.getLogger(ObserverMaster.class);
    +
    +    //Follower counter
    +    private final AtomicLong followerCounter = new AtomicLong(-1);
    --- End diff --
    
    This copies the followerCounter of the Leader as used in the beginning of LearnerHandler::run.
My understanding is that this provides backwards compatibility for when the first FOLLOWERINFO
or OBSERVERINFO packet from the Learner does not supply a LearnerInfo and the Leader (or ObserverMaster
here) needs to come up with a unique server id for the new Learner with which to refer to
it in all internal collections.
    
    It will also end up being used in #337 if/when that patch is accepted.
    
    I'm happy to add explanatory comments for these counters in the Leader and ObserverMaster
but am unsure of how much of the above should be included. What do you think of a simple "assigns
unique server id for internal tracking of Learners that do not provide a unique id"?


---

Mime
View raw message