hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matteo Bertozzi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-10993) Deprioritize long-running scanners
Date Fri, 18 Apr 2014 04:42:16 GMT

    [ https://issues.apache.org/jira/browse/HBASE-10993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13973786#comment-13973786

Matteo Bertozzi commented on HBASE-10993:

{quote}Why call getPriority rather than add this new getDeadline call?{quote}
At the moment we have 3 queues, HIGH_QOS (Meta Requests), REPLICATION_QOS, NORMAL_QOS (all
the user requests). Currently getPriority() means more "get request type" and is used to select
the queue.
getDeadline() instead defines how urgent is the request among the requests in the same queue,
and given the fact that we can recognize long scans we can automatically deprioritize them.

{quote}How we know if it is working – or how you think we will be able to observe that it
is operating well on a cluster?{quote}
The unittest with the cumulative sum shows how the priority queue/deadline works. A live test
is having a large table and a couple of full table scan together with get/puts. NOTE that
if you have less clients (small requests) than handlers you'll not see any difference.

> Deprioritize long-running scanners
> ----------------------------------
>                 Key: HBASE-10993
>                 URL: https://issues.apache.org/jira/browse/HBASE-10993
>             Project: HBase
>          Issue Type: Sub-task
>            Reporter: Matteo Bertozzi
>            Assignee: Matteo Bertozzi
>            Priority: Minor
>             Fix For: 1.0.0
>         Attachments: HBASE-10993-v0.patch, HBASE-10993-v1.patch
> Currently we have a single call queue that serves all the "normal user"  requests, and
the requests are executed in FIFO.
> When running map-reduce jobs and user-queries on the same machine, we want to prioritize
the user-queries.
> Without changing too much code, and not having the user giving hints, we can add a “vtime”
field to the scanner, to keep track from how long is running. And we can replace the callQueue
with a priorityQueue. In this way we can deprioritize long-running scans, the longer a scan
request lives the less priority it gets.

This message was sent by Atlassian JIRA

View raw message