hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hemanth Yamijala (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5187) Provide an option to turn off priorities in jobs
Date Mon, 09 Feb 2009 03:48:59 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12671718#action_12671718
] 

Hemanth Yamijala commented on HADOOP-5187:
------------------------------------------

Matei, the "fifo" setting in the fairshare scheduler currently is priority based fifo, meaning
the jobs are sorted by priorities, and then time. I was hoping that there would be a way to
turn off priority based comparison completely, and make it pure fifo.

I hope the use case made sense. Since users from possibly competing organizations could submit
jobs to a default pool, their job priorities cannot be compared apples to apples. Hence, the
site administrator might prefer if the scheduler can be asked to completely turn off using
priorities in comparison, and let only first come first served win.

So, I was thinking if there could be a 'support-priorities' tag per pool. If this is off,
then priorities of jobs are not used by the scheduler. Note that the scheduling mode can be
'fairshare' or 'fifo' mode still. In both cases, priorities would be ignored.

If you had also meant fifo in your comment above to mean the same, then we are in sync.



> Provide an option to turn off priorities in jobs
> ------------------------------------------------
>
>                 Key: HADOOP-5187
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5187
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: contrib/fair-share
>            Reporter: Hemanth Yamijala
>            Priority: Minor
>
> The fairshare scheduler can define pools mapping to queues (as defined in the capacity
scheduler - HADOOP-3445). When used in this manner, one can imagine queues set up to be used
by users who come from disparate teams or organizations (say a default queue). For such a
queue, it makes sense to ignore job priorities and consider the queue as strict FIFO, as it
is difficult to compare priorities of jobs from different users. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message