hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-12790) Support fairness across parallelized scans
Date Wed, 22 Apr 2015 17:02:01 GMT

    [ https://issues.apache.org/jira/browse/HBASE-12790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14507418#comment-14507418

Andrew Purtell commented on HBASE-12790:

Why both FairShareBlockingQueue and FairSharePriorityBasedBlockingQueue? Which one would Phoenix
use? Pick that one.

In the new *BlockingQueue classes, each time we want to remove an item from the queue we have
to do a string comparison? An instanceof check for ScanRequest on the result of {{o.getCall().param}}
would be faster, probably a lot faster. Need to be performance conscious here, at least in
this case I'm not seeing it.

Have these changes been benchmarked? What is the performance difference? Going to be important
to put up numbers gathered in a statistically meaningful way I think.

SimpleRpcScheduler is getting more complex. Beginning to not live up to its naming.

Are the changes made to SimpleRpcScheduler available to the other schedulers? In other words,
are you teaching all of the schedulers about call grouping? If not, why not? 

Have we tried using one of the concurrent queue types instead of a blocking (locking) queue
type? Does that scale better as the concurrency is turned up?

Consider JMH for collecting microbenchmarks.

> Support fairness across parallelized scans
> ------------------------------------------
>                 Key: HBASE-12790
>                 URL: https://issues.apache.org/jira/browse/HBASE-12790
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: James Taylor
>            Assignee: ramkrishna.s.vasudevan
>              Labels: Phoenix
>         Attachments: AbstractRoundRobinQueue.java, HBASE-12790.patch, HBASE-12790_1.patch,
> Some HBase clients parallelize the execution of a scan to reduce latency in getting back
results. This can lead to starvation with a loaded cluster and interleaved scans, since the
RPC queue will be ordered and processed on a FIFO basis. For example, if there are two clients,
A & B that submit largish scans at the same time. Say each scan is broken down into 100
scans by the client (broken down into equal depth chunks along the row key), and the 100 scans
of client A are queued first, followed immediately by the 100 scans of client B. In this case,
client B will be starved out of getting any results back until the scans for client A complete.
> One solution to this is to use the attached AbstractRoundRobinQueue instead of the standard
FIFO queue. The queue to be used could be (maybe it already is) configurable based on a new
config parameter. Using this queue would require the client to have the same identifier for
all of the 100 parallel scans that represent a single logical scan from the clients point
of view. With this information, the round robin queue would pick off a task from the queue
in a round robin fashion (instead of a strictly FIFO manner) to prevent starvation over interleaved
parallelized scans.

This message was sent by Atlassian JIRA

View raw message