hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hiroshi Ikeda (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14331) a single callQueue related improvements
Date Tue, 01 Sep 2015 02:40:46 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14724631#comment-14724631

Hiroshi Ikeda commented on HBASE-14331:

Can you put the three new Java classes in a single patch with proper Apache License header
For the two BlockingQueue implementations, please add unit tests.

The added classes are yet trial and they even have no package declaration. I'll make them
into a single patch with tests when I find how I should inject these classes into the call-queue
logic. It is the easiest to just replace existing classes with the new classes, but it may
be too rough.

Looks like SemaphoreBasedLinkedBlockingQueue is better than SemaphoreBasedPriorityBlockingQueue
when waked read threads - tasks are high.
What's your interpretation ?

The priority queue is not FIFO. Added elements in the queue are sorted with a comparator,
and the least element are given by polling. This process is apparently heavier than just adding
a element to the tail of the queue, and comparing such queues is not fair.

> a single callQueue related improvements
> ---------------------------------------
>                 Key: HBASE-14331
>                 URL: https://issues.apache.org/jira/browse/HBASE-14331
>             Project: HBase
>          Issue Type: Improvement
>          Components: IPC/RPC, Performance
>            Reporter: Hiroshi Ikeda
>            Priority: Minor
>         Attachments: BlockingQueuesPerformanceTestApp-output.pdf, BlockingQueuesPerformanceTestApp-output.txt,
BlockingQueuesPerformanceTestApp.java, CallQueuePerformanceTestApp.java, SemaphoreBasedBlockingQueue.java,
SemaphoreBasedLinkedBlockingQueue.java, SemaphoreBasedPriorityBlockingQueue.java
> {{LinkedBlockingQueue}} well separates locks between the {{take}} method and the {{put}}
method, but not between takers, and not between putters. These methods are implemented to
take locks at the almost beginning of their logic. HBASE-11355 introduces multiple call-queues
to reduce such possible congestion, but I doubt that it is required to stick to {{BlockingQueue}}.
> There are the other shortcomings of using {{BlockingQueue}}. When using multiple queues,
since {{BlockingQueue}} blocks threads it is required to prepare enough threads for each queue.
It is possible that there is a queue starving for threads while there is another queue where
threads are idle. Even if you can tune parameters to avoid such situations, the tuning is
not so trivial.
> I suggest using a single {{ConcurrentLinkedQueue}} with {{Semaphore}}.

This message was sent by Atlassian JIRA

View raw message