hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ramkrishna.s.vasudevan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-12790) Support fairness across parallelized scans
Date Wed, 09 Sep 2015 10:07:46 GMT

    [ https://issues.apache.org/jira/browse/HBASE-12790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14736595#comment-14736595
] 

ramkrishna.s.vasudevan commented on HBASE-12790:
------------------------------------------------

In between my HBASE-11425 tests, I spent some time in this JIRA and completed the testing
that is needed. The set up was to load a Phoenix table with 70000000 rows. 
The RS is configured with 5 handler threads.  The table is major compacted and ensure that
the STATS table is filled up with stats that would be enough to fill up the handler threads
with parallel scan queries with same GROUP ID  when a count(*) query is issued.
We basically set up a client program which infinitely runs two queries in two threads - count(*)
and another a point query (a query that exactly retrieves a single row).
The count(*) is split into around 15 parallel queries each with the same group id.
Now before we implement such a patch where we could round robin on a group id we get this

In case of FIFO based implementation (0.98)
Without patch
==========
{code}
The completion time for point query is 5
The completion time for point query is 5
The completion time for countquery is 19793
The completion time for point query is 12
The completion time for point query is 20656
......
The completion time for countquery is 21127
The completion time for point query is 5
The completion time for point query is 22454
The completion time for point query is 8
....
The completion time for point query is 3
The completion time for countquery is 24144
The completion time for point query is 26
The completion time for point query is 18876
The completion time for point query is 6
{code}

With patch
========
{code}
The completion time for point query is 8
The completion time for countquery is 22226
The completion time for point query is 8
The completion time for point query is 2131
The completion time for point query is 12628
The completion time for point query is 6359

....

The completion time for countquery is 22656
The completion time for point query is 7
The completion time for point query is 2476
The completion time for point query is 14469
The completion time for point query is 318
{code}

Deadline based priority queue
Without patch
===========
{code}
The completion time for point query is 4
The completion time for countquery is 29877
The completion time for point query is 5
The completion time for point query is 26895
The completion time for point query is 9

The completion time for point query is 4
The completion time for countquery is 29027
The completion time for point query is 6
The completion time for point query is 27288
The completion time for point query is 5
{code}

With patch
========
{code}
The completion time for point query is 6
The completion time for countquery is 26445
The completion time for point query is 8
The completion time for point query is 3098
The completion time for point query is 215
The completion time for point query is 1682
The completion time for point query is 963
The completion time for point query is 760
The completion time for point query is 1386
The completion time for point query is 1185
The completion time for point query is 2013
The completion time for point query is 809
The completion time for point query is 2410
The completion time for point query is 869
The completion time for point query is 1661
The completion time for point query is 1430
The completion time for point query is 1748
The completion time for point query is 1046
The completion time for point query is 1299
The completion time for point query is 698
The completion time for point query is 1320
The completion time for point query is 23
The completion time for point query is 6
The completion time for point query is 11


The completion time for countquery is 27141
The completion time for point query is 5
The completion time for point query is 3158
The completion time for point query is 177
The completion time for point query is 1828
The completion time for point query is 1222
The completion time for point query is 3745
The completion time for point query is 30
The completion time for point query is 207
The completion time for point query is 2639
The completion time for point query is 1396
The completion time for point query is 2425
The completion time for point query is 903
The completion time for point query is 668
The completion time for point query is 2718
The completion time for point query is 1863
{code}

so we could see that instead of waiting for a single point query upto 25 to 29 secs we try
to round robin with the point queries that are interspersed with the count query's scans(with
same groupid).  Thus avoiding the delay due to that single point query. 

> Support fairness across parallelized scans
> ------------------------------------------
>
>                 Key: HBASE-12790
>                 URL: https://issues.apache.org/jira/browse/HBASE-12790
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: James Taylor
>            Assignee: ramkrishna.s.vasudevan
>              Labels: Phoenix
>         Attachments: AbstractRoundRobinQueue.java, HBASE-12790.patch, HBASE-12790_1.patch,
HBASE-12790_callwrapper.patch
>
>
> Some HBase clients parallelize the execution of a scan to reduce latency in getting back
results. This can lead to starvation with a loaded cluster and interleaved scans, since the
RPC queue will be ordered and processed on a FIFO basis. For example, if there are two clients,
A & B that submit largish scans at the same time. Say each scan is broken down into 100
scans by the client (broken down into equal depth chunks along the row key), and the 100 scans
of client A are queued first, followed immediately by the 100 scans of client B. In this case,
client B will be starved out of getting any results back until the scans for client A complete.
> One solution to this is to use the attached AbstractRoundRobinQueue instead of the standard
FIFO queue. The queue to be used could be (maybe it already is) configurable based on a new
config parameter. Using this queue would require the client to have the same identifier for
all of the 100 parallel scans that represent a single logical scan from the clients point
of view. With this information, the round robin queue would pick off a task from the queue
in a round robin fashion (instead of a strictly FIFO manner) to prevent starvation over interleaved
parallelized scans.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message