hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ramkrishna.s.vasudevan (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-12790) Support fairness across parallelized scans
Date Thu, 16 Apr 2015 11:06:59 GMT

     [ https://issues.apache.org/jira/browse/HBASE-12790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

ramkrishna.s.vasudevan updated HBASE-12790:
    Attachment: HBASE-12790_callwrapper.patch

Attaching a patch that has been tested in real cluster.  Am working on taking the performance
numbers for scans.  Currently this patch implements the round robin thing only for scans.
 Am not very clear on how to write a generic test case. Any suggestions in this area?
But pls take note of the following changes,
The RpcExecutor and the related Queue data structures will no longer work on CallRunner instead
they will work on CallRunnerWrapper.  (better name?). This is because CallRunner every time
depends on the 'call' object to determine its deadline when we use Deadline based comparator.
 But since the 'call' can be nullified every time, the new data structure like AbstractPriorityRoundRobinQueue
(Using TreeMap) suffers from NPE because of this nullification when it tries to apply the
comparison. Also with this patch we calculate the deadline once when the 'call' starts and
not every time. We ensure that we no longer depend on the 'call' object to know the deadline
for comparisons.  
You could see that CallPriorityComparator has been removed.

> Support fairness across parallelized scans
> ------------------------------------------
>                 Key: HBASE-12790
>                 URL: https://issues.apache.org/jira/browse/HBASE-12790
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: James Taylor
>            Assignee: ramkrishna.s.vasudevan
>              Labels: Phoenix
>         Attachments: AbstractRoundRobinQueue.java, HBASE-12790.patch, HBASE-12790_1.patch,
> Some HBase clients parallelize the execution of a scan to reduce latency in getting back
results. This can lead to starvation with a loaded cluster and interleaved scans, since the
RPC queue will be ordered and processed on a FIFO basis. For example, if there are two clients,
A & B that submit largish scans at the same time. Say each scan is broken down into 100
scans by the client (broken down into equal depth chunks along the row key), and the 100 scans
of client A are queued first, followed immediately by the 100 scans of client B. In this case,
client B will be starved out of getting any results back until the scans for client A complete.
> One solution to this is to use the attached AbstractRoundRobinQueue instead of the standard
FIFO queue. The queue to be used could be (maybe it already is) configurable based on a new
config parameter. Using this queue would require the client to have the same identifier for
all of the 100 parallel scans that represent a single logical scan from the clients point
of view. With this information, the round robin queue would pick off a task from the queue
in a round robin fashion (instead of a strictly FIFO manner) to prevent starvation over interleaved
parallelized scans.

This message was sent by Atlassian JIRA

View raw message