hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Heng Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15765) RPC handlers should steal from other queues in the same or higher priority
Date Wed, 18 May 2016 02:35:12 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15288112#comment-15288112

Heng Chen commented on HBASE-15765:

In the same priority level, we have (numHandlers / 10) queues by default. So if numHandlers=60,
then we have 6 different queues in the same priority level. The handler is assigned a queue
based on hash, and the handler can only take RPC requests from its own queue, not the other
5 queues. 
If so,  why you said that "with this patch, the handlers still do not steal from the queues
in the same tier" above.
In the patch v1,  if the assigned queue is empty,  we will get task from queue with max entries
in the same priority level.  
+  private CallRunner stealTaskFromOtherQueue() throws InterruptedException {
+    // Firstly, we try to get task from same Level queues
+    CallRunner task = getTaskFromMaxQueue(getQueues());
+    if (task == null) {
+      // we get task from High Level queues
+      task = getTaskFromMaxQueue(getHighLevelQueues());
+    }
+    return task;
+  }

So we block on our queue for 3 seconds before going to the higher priority queue. Let's say
that the high priority queue is all full and our queue is always empty. We can still process
1 RPC from the higher priority queue every 3 seconds with the patch.
You are right. It is about the question "when should we do steal task from other queue". 

In the patch, handler do steal when current assigned queue is empty.  I add 3s timeout in
{{poll}} to avoid the handler always in busy mode and waste a lot of cpu resources.
What's your suggestion about ""when should we do steal task from other queue"?  [~enis]

> RPC handlers should steal from other queues in the same or higher priority
> --------------------------------------------------------------------------
>                 Key: HBASE-15765
>                 URL: https://issues.apache.org/jira/browse/HBASE-15765
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Enis Soztutar
>         Attachments: HBASE-15765.patch, HBASE-15765_v1.patch
> With the simple rpc scheduler model by default, a request first falls in one type of
queue (priority, general, replication) and it will get "sharded" in a queue within that priority
level (for example there are 2 queues for the high priority level). The handlers can only
take from their specific queues, and nothing else. 
> Even if a request is waiting in a high priority queue, the handlers within the same priority
level or a lower priority level will not see these requests (which is the root cause for HBASE-15613).
We should be able to steal from other queues, without losing the prioritization and sharding

This message was sent by Atlassian JIRA

View raw message