hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ramkrishna.s.vasudevan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-12790) Support fairness across parallelized scans
Date Tue, 31 Mar 2015 16:18:53 GMT

    [ https://issues.apache.org/jira/browse/HBASE-12790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14388771#comment-14388771

ramkrishna.s.vasudevan commented on HBASE-12790:

Thanks Stack for the review. 
bq.a new concept – 'grouping' – is added to Scan as public apis with no doc on what it
is about.
I added the comment (not as doc) on the groupingId String variable. Will add a doc on the
setters and getters. bq.Why is a bq.grouping id a 'string'?
What do you think we could add, an Integer? I thought a groupingId could a be a simple random
string formed from a Unique random Id generated and use it with a clientId - Atleast currently
in Phoenix it is a String.
bq.Does FairShareBalancedRPCExecutor add anything? Why in class comment does it talk about
scan when it plainly does nothing with scan. Ditto FairShareRWQueueRPCExecutor.
Yes.  It actually instantiates the PriorityQueue that will be used.  In the BalancedRPCExecutor
it will be LinkedBlockingQueue or BoundedPriorityQueue.  Here we instantiate the FairShareBlockingQueue
or the FairSharePriorityBasedBlockingQueue.
What I mean in the comment is that currently as I had tried out only for scan wanted to highlight
that this feature works with SCan only.
bq.and other grouping pollution. Is this because you wanted to avoid putting in place a fair
scheduler implementation?
May be yes.  That would be cleaner. I did this way because wanted to use this new queue only
in the read part.  
As you suggested if you go with a new scheduler then should we use this for Replication, write
and read also?
bq.Any micro-benchmark compare of this implementation to current q implementation even with
no grouping enabled?
Yes sure.
bq.Is a 'producer' a 'group'? (Odd having method named extractProducer return group).
Yes here producer is a group. The AbstractPriorityRoundRobinQueue is a generic impl. So here
in this case we have the producer as a simple grouping String.
bq.Do you have any proof this code delivers what is suggested at the top of this issue, that
if all cilent A's scans are queued before all of client B's, that client B will get some action.
Yes I will add a real world test case.  Mainly this patch is to highlight the idea.  
My main question would be should we use this scheduler in all cases like writes and replication?
 OR may be as a first cut as in this patch we go only 'scans' or 'gets'?
Even in the current way we have different queues for writes and reads.
bq.So, users would have to 'enable' this on the cluster? It would not be on by default? If
no degradation in scheduler, why would we not want this always on 
Sure we could try this, based on the above question.
Once again thanks for the review.

> Support fairness across parallelized scans
> ------------------------------------------
>                 Key: HBASE-12790
>                 URL: https://issues.apache.org/jira/browse/HBASE-12790
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: James Taylor
>            Assignee: ramkrishna.s.vasudevan
>              Labels: Phoenix
>         Attachments: AbstractRoundRobinQueue.java, HBASE-12790.patch, HBASE-12790_1.patch
> Some HBase clients parallelize the execution of a scan to reduce latency in getting back
results. This can lead to starvation with a loaded cluster and interleaved scans, since the
RPC queue will be ordered and processed on a FIFO basis. For example, if there are two clients,
A & B that submit largish scans at the same time. Say each scan is broken down into 100
scans by the client (broken down into equal depth chunks along the row key), and the 100 scans
of client A are queued first, followed immediately by the 100 scans of client B. In this case,
client B will be starved out of getting any results back until the scans for client A complete.
> One solution to this is to use the attached AbstractRoundRobinQueue instead of the standard
FIFO queue. The queue to be used could be (maybe it already is) configurable based on a new
config parameter. Using this queue would require the client to have the same identifier for
all of the 100 parallel scans that represent a single logical scan from the clients point
of view. With this information, the round robin queue would pick off a task from the queue
in a round robin fashion (instead of a strictly FIFO manner) to prevent starvation over interleaved
parallelized scans.

This message was sent by Atlassian JIRA

View raw message