hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ramkrishna.s.vasudevan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-12790) Support fairness across parallelized scans
Date Wed, 25 Mar 2015 14:19:53 GMT

    [ https://issues.apache.org/jira/browse/HBASE-12790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14379912#comment-14379912
] 

ramkrishna.s.vasudevan commented on HBASE-12790:
------------------------------------------------

bq.seems burdensome expecting that clients identify their own scan 'group'; can't we pick
an identifier internal to the scan and not bother users with such detail?
This is based on the Phoenix use case. Pls take a look at 
https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=blob;f=phoenix-core/src/main/java/org/apache/phoenix/job/JobManager.java;h=31ef7424d4ad04ddbd286c32ef27f187266a7728;hb=a94a6f4195af2867379803f19c90045eb3943c2d.
Phoenix currently tries to use its own thread pool executor to execute the task in parallel
which are grouped.
Currently in HBase we have only FIFO model for which we use the LinkedBlockingQueue that java
provides and for the deadline  based prioirty we use the BoundedPrioirtyQueue impl.  

In order to implement the round robin impl based on the Grouping the current RPC executor
does not have any data structure.  
Regarding the change to Scan - we could actually do that setting the Attributes. That was
initially thought as the best way.  But the thing is that inside the Executor inorder to identify
the Attribute corresponding to the groupId we have to iterate the list of attributes because
from the Protobuf we don't get a map. If that is fine I can remove the changes to Scan.java
and the patch will not have protobuf changes. 

But regarding the round robin nature I think we need a generic queue impl that would do the
round robin work like how a new BoundedPriorityQueue was implemented just to have the FIFO
nature.
 

> Support fairness across parallelized scans
> ------------------------------------------
>
>                 Key: HBASE-12790
>                 URL: https://issues.apache.org/jira/browse/HBASE-12790
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: James Taylor
>            Assignee: ramkrishna.s.vasudevan
>              Labels: Phoenix
>         Attachments: AbstractRoundRobinQueue.java, HBASE-12790.patch, HBASE-12790_1.patch
>
>
> Some HBase clients parallelize the execution of a scan to reduce latency in getting back
results. This can lead to starvation with a loaded cluster and interleaved scans, since the
RPC queue will be ordered and processed on a FIFO basis. For example, if there are two clients,
A & B that submit largish scans at the same time. Say each scan is broken down into 100
scans by the client (broken down into equal depth chunks along the row key), and the 100 scans
of client A are queued first, followed immediately by the 100 scans of client B. In this case,
client B will be starved out of getting any results back until the scans for client A complete.
> One solution to this is to use the attached AbstractRoundRobinQueue instead of the standard
FIFO queue. The queue to be used could be (maybe it already is) configurable based on a new
config parameter. Using this queue would require the client to have the same identifier for
all of the 100 parallel scans that represent a single logical scan from the clients point
of view. With this information, the round robin queue would pick off a task from the queue
in a round robin fashion (instead of a strictly FIFO manner) to prevent starvation over interleaved
parallelized scans.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message