hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-10993) Deprioritize long-running scanners
Date Thu, 17 Apr 2014 19:56:15 GMT

    [ https://issues.apache.org/jira/browse/HBASE-10993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13973321#comment-13973321

stack commented on HBASE-10993:

Should getDeadline be in the PriorityFunction Interface?  It seems like an implementation
detail of a particular PF implementation?

On this:

+  // Comparator used by the "normal callQueue".

We are changing the 'normal', or default SimpleRpcScheduler to 'deprioritize long-running
scanners', right?   It is on by default:

+    if (conf.getBoolean(DEADLINE_CALL_QUEUE_CONF_KEY, true)) {

That is good.  What 'effect' should I see now this is on?  Any?  (Since SCAN_VTIME_WEIGHT_CONF_KEY
has a default of 1.0f?"

This has no doc:

+  public static final String SCAN_VTIME_WEIGHT_CONF_KEY = "ipc.server.scan.vtime.weight";

Somewhere we should have description of what vtime and 'weight' is about.

This class needs doc: FixedPriorityBlockingQueue   Is it 'fixed' priority?  Doesn't it change
w/ how long scan has been going on?

Yeah, some explanation here would help.. why we are sqrt'ing and rounding and multiplying
weight ...

+      long vtime = rpcServices.getScannerVirtualTime(request.getScannerId());
+      return Math.round(Math.sqrt(vtime * scanVirtualTimeWeight));

Is the below a timestamp?

+      return scannerHolder.nextCallSeq;

Nice test in TestSimpleRpcScheduler

Patch looks great Matteo.  I did not review the queue implementation closely.  The test for
it seems fine.

> Deprioritize long-running scanners
> ----------------------------------
>                 Key: HBASE-10993
>                 URL: https://issues.apache.org/jira/browse/HBASE-10993
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Matteo Bertozzi
>            Assignee: Matteo Bertozzi
>            Priority: Minor
>             Fix For: 1.0.0
>         Attachments: HBASE-10993-v0.patch
> Currently we have a single call queue that serves all the "normal user"  requests, and
the requests are executed in FIFO.
> When running map-reduce jobs and user-queries on the same machine, we want to prioritize
the user-queries.
> Without changing too much code, and not having the user giving hints, we can add a “vtime”
field to the scanner, to keep track from how long is running. And we can replace the callQueue
with a priorityQueue. In this way we can deprioritize long-running scans, the longer a scan
request lives the less priority it gets.

This message was sent by Atlassian JIRA

View raw message