hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Li (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-10281) Create a scheduler, which assigns schedulables a priority level
Date Fri, 13 Jun 2014 04:41:01 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-10281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Chris Li updated HADOOP-10281:
------------------------------

    Description: 
The Scheduler decides which sub-queue to assign a given Call. It implements a single method
getPriorityLevel(Schedulable call) which returns an integer corresponding to the subqueue
the FairCallQueue should place the call in.

The HistoryRpcScheduler is one such implementation which uses the username of each call and
determines what % of calls in recent history were made by this user.

It is configured with a historyLength (how many calls to track) and a list of integer thresholds
which determine the boundaries between priority levels.

For instance, if the scheduler has a historyLength of 8; and priority thresholds of 4,2,1;
and saw calls made by these users in order:
Alice, Bob, Alice, Alice, Bob, Jerry, Alice, Alice

* Another call by Alice would be placed in queue 3, since she has already made >= 4 calls
* Another call by Bob would be placed in queue 2, since he has >= 2 but less than 4 calls
* A call by Carlos would be placed in queue 0, since he has no calls in the history

Also, some versions of this patch include the concept of a 'service user', which is a user
that is always scheduled high-priority. Currently this seems redundant and will probably be
removed in later patches, since its not too useful.

  was:
The Scheduler decides which sub-queue to assign a given Call. It implements a single method
getPriorityLevel(Schedulable call) which returns an integer corresponding to the subqueue
the FairCallQueue should place the call in.

The HistoryRpcScheduler is one such implementation which uses the username of each call and
determines what % of calls in recent history were made by this user.

It is configured with a historyLength (how many calls to track) and a list of integer thresholds
which determine the boundaries between priority levels.

For instance, if the scheduler has a historyLength of 8; and priority thresholds of 4,2,1;
and saw calls made by these users in order:
Alice, Bob, Alice, Alice, Bob, Jerry, Alice, Alice

* Another call by Alice would be placed in queue 3, since she has already made >= 4 calls
* Another call by Bob would be placed in queue 2, since he has >= 2 but less than 4 calls
* A call by Carlos would be placed in queue 0, since he has no calls in the history



> Create a scheduler, which assigns schedulables a priority level
> ---------------------------------------------------------------
>
>                 Key: HADOOP-10281
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10281
>             Project: Hadoop Common
>          Issue Type: Sub-task
>            Reporter: Chris Li
>         Attachments: HADOOP-10281.patch, HADOOP-10281.patch
>
>
> The Scheduler decides which sub-queue to assign a given Call. It implements a single
method getPriorityLevel(Schedulable call) which returns an integer corresponding to the subqueue
the FairCallQueue should place the call in.
> The HistoryRpcScheduler is one such implementation which uses the username of each call
and determines what % of calls in recent history were made by this user.
> It is configured with a historyLength (how many calls to track) and a list of integer
thresholds which determine the boundaries between priority levels.
> For instance, if the scheduler has a historyLength of 8; and priority thresholds of 4,2,1;
and saw calls made by these users in order:
> Alice, Bob, Alice, Alice, Bob, Jerry, Alice, Alice
> * Another call by Alice would be placed in queue 3, since she has already made >=
4 calls
> * Another call by Bob would be placed in queue 2, since he has >= 2 but less than
4 calls
> * A call by Carlos would be placed in queue 0, since he has no calls in the history
> Also, some versions of this patch include the concept of a 'service user', which is a
user that is always scheduled high-priority. Currently this seems redundant and will probably
be removed in later patches, since its not too useful.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message