hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Appy (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-19290) Reduce zk request when doing split log
Date Wed, 22 Nov 2017 18:17:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-19290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16263051#comment-16263051
] 

Appy commented on HBASE-19290:
------------------------------

{quote}
bq. Since expectedTasksPerRS only relay on availableRSs and numTasks, I think it is better
to move them to getAvailableRSs() and change a name?
I do not think so, and i think getAvailableRSs and calculateAvailableSplitters is clear.
{quote}

Yeah, they are definitely clear, but [~tianjingyun]'s suggestion to improve the code further
was appropriate and made sense.
Here's the diff on what he was suggesting (and what i was thinking earlier).
{noformat}
diff --git a/hbase-server/src/main/java/org/apache/hadoop/hbase/coordination/ZkSplitLogWorkerCoordination.java
b/hbase-server/src/main/java/org/apache/hadoop/hbase/coordination/ZkSplitLogWorkerCoordination.java
index e7d15bbc16..e7fd93bf46 100644
--- a/hbase-server/src/main/java/org/apache/hadoop/hbase/coordination/ZkSplitLogWorkerCoordination.java
+++ b/hbase-server/src/main/java/org/apache/hadoop/hbase/coordination/ZkSplitLogWorkerCoordination.java
@@ -318,7 +318,14 @@ public class ZkSplitLogWorkerCoordination extends ZKListener implements
     server.getExecutorService().submit(hsh);
   }
 
-  private int getAvailableRSs() {
+  /**
+   * This function calculates how many splitters this RS should create based on expected
average
+   * tasks per RS and the hard limit upper bound(maxConcurrentTasks) set by configuration.
<br>
+   * At any given time, a RS allows spawn MIN(Expected Tasks/RS, Hard Upper Bound)
+   * @param numTasks total number of split tasks available
+   * @return number of tasks this RS can grab
+   */
+  private int getNumExpectedTasksPerRS(int numTasks) {
     // at lease one RS(itself) available
     int availableRSs = 1;
     try {
@@ -329,22 +336,17 @@ public class ZkSplitLogWorkerCoordination extends ZKListener implements
       // do nothing
       LOG.debug("getAvailableRegionServers got ZooKeeper exception", e);
     }
-    return availableRSs;
+    int expectedTasksPerRS = (numTasks / availableRSs) + ((numTasks % availableRSs == 0)
? 0 : 1);
+    return Math.max(1, expectedTasksPerRS); // at least be one
   }
 
   /**
-   * This function calculates how many splitters it could create based on expected average
tasks per
-   * RS and the hard limit upper bound(maxConcurrentTasks) set by configuration. <br>
-   * At any given time, a RS allows spawn MIN(Expected Tasks/RS, Hard Upper Bound)
-   * @param availableRSs current total number of available regionservers
-   * @param numTasks current total number of available tasks
+   * @param expectedTasksPerRS Average number of tasks to be handled by each RS
+   * @return true if more splitters are available, otherwise false.
    */
-  private int calculateAvailableSplitters(int availableRSs, int numTasks) {
-    int expectedTasksPerRS = (numTasks / availableRSs) + ((numTasks % availableRSs == 0)
? 0 : 1);
-    expectedTasksPerRS = Math.max(1, expectedTasksPerRS); // at least be one
-    // calculate how many more splitters we could spawn
-    return Math.min(expectedTasksPerRS, maxConcurrentTasks)
-        - this.tasksInProgress.get();
+  private boolean areSplittersAvailable(int expectedTasksPerRS) {
+    return (Math.min(expectedTasksPerRS, maxConcurrentTasks)
+        - this.tasksInProgress.get()) > 0;
   }
 
   /**
@@ -425,11 +427,11 @@ public class ZkSplitLogWorkerCoordination extends ZKListener implements
         }
       }
       int numTasks = paths.size();
-      int availableRSs = getAvailableRSs();
+      int expectedTasksPerRS = getNumExpectedTasksPerRS(numTasks);
       int taskGrabbed = 0;
       for (int i = 0; i < numTasks; i++) {
         while (!shouldStop) {
-          if (this.calculateAvailableSplitters(availableRSs, numTasks) > 0) {
+          if (this.areSplittersAvailable(expectedTasksPerRS)) {
             LOG.debug("Current region server " + server.getServerName()
                 + " is ready to take more tasks, will get task list and try grab tasks again.");
             int idx = (i + offset) % paths.size();
{noformat}

> Reduce zk request when doing split log
> --------------------------------------
>
>                 Key: HBASE-19290
>                 URL: https://issues.apache.org/jira/browse/HBASE-19290
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: binlijin
>            Assignee: binlijin
>         Attachments: HBASE-19290.master.001.patch, HBASE-19290.master.002.patch, HBASE-19290.master.003.patch
>
>
> We observe once the cluster has 1000+ nodes and when hundreds of nodes abort and doing
split log, the split is very very slow, and we find the regionserver and master wait on the
zookeeper response, so we need to reduce zookeeper request and pressure for big cluster.
> (1) Reduce request to rsZNode, every time calculateAvailableSplitters will get rsZNode's
children from zookeeper, when cluster is huge, this is heavy. This patch reduce the request.

> (2) When the regionserver has max split tasks running, it may still trying to grab task
and issue zookeeper request, we should sleep and wait until we can grab tasks again.  



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message