hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matteo Bertozzi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-10993) Deprioritize long-running scanners
Date Thu, 17 Apr 2014 20:47:18 GMT

    [ https://issues.apache.org/jira/browse/HBASE-10993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13973382#comment-13973382

Matteo Bertozzi commented on HBASE-10993:

{quote}That is good. What 'effect' should I see now this is on? Any? (Since SCAN_VTIME_WEIGHT_CONF_KEY
has a default of 1.0f?"{quote}
The weight is just a weight, more scan.next() you do more delayed you will be in case there
are requests with no delay. If you increase the weight, each single next may delay you more.
The testRpcScheduler() is showing that the "long scan" will be executed after all the other

Yeah, some explanation here would help.. why we are sqrt'ing and rounding and multiplying
weight ...
+ long vtime = rpcServices.getScannerVirtualTime(request.getScannerId());
+ return Math.round(Math.sqrt(vtime * scanVirtualTimeWeight));
The sqrt gives you a nice curve that represent in a quite good way what we want to do.. after
some time start delay and keep increase the delay more you are running (but not too much)

{quote}This class needs doc: FixedPriorityBlockingQueue Is it 'fixed' priority? Doesn't it
change w/ how long scan has been going on?{quote}
I should use "bounded" instead of "fixed" since it is referring to the number of elements.
this is a generic priority queue that keeps the fifo order if t

{quote}Is the below a timestamp? + return scannerHolder.nextCallSeq;{quote}
No at the moment the vtime, is the number of scanner.next() calls that you do

> Deprioritize long-running scanners
> ----------------------------------
>                 Key: HBASE-10993
>                 URL: https://issues.apache.org/jira/browse/HBASE-10993
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Matteo Bertozzi
>            Assignee: Matteo Bertozzi
>            Priority: Minor
>             Fix For: 1.0.0
>         Attachments: HBASE-10993-v0.patch
> Currently we have a single call queue that serves all the "normal user"  requests, and
the requests are executed in FIFO.
> When running map-reduce jobs and user-queries on the same machine, we want to prioritize
the user-queries.
> Without changing too much code, and not having the user giving hints, we can add a “vtime”
field to the scanner, to keep track from how long is running. And we can replace the callQueue
with a priorityQueue. In this way we can deprioritize long-running scans, the longer a scan
request lives the less priority it gets.

This message was sent by Atlassian JIRA

View raw message