hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matei Zaharia (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5187) Provide an option to turn off priorities in jobs
Date Fri, 06 Feb 2009 21:34:59 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12671335#action_12671335
] 

Matei Zaharia commented on HADOOP-5187:
---------------------------------------

Right now, we do fair sharing between jobs in the same pool, so the jobs in the default queue
would share the excess capacity in the cluster equally. This is what we wanted at Facebook
because jobs were submitted to a default queue by multiple users and we wanted short jobs
to be able to finish while long jobs are also running. However, it would make a lot of sense
to support a "fifo" setting on each pool to make some pools be  FIFO. Would that be acceptable?

> Provide an option to turn off priorities in jobs
> ------------------------------------------------
>
>                 Key: HADOOP-5187
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5187
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: contrib/fair-share
>            Reporter: Hemanth Yamijala
>            Priority: Minor
>
> The fairshare scheduler can define pools mapping to queues (as defined in the capacity
scheduler - HADOOP-3445). When used in this manner, one can imagine queues set up to be used
by users who come from disparate teams or organizations (say a default queue). For such a
queue, it makes sense to ignore job priorities and consider the queue as strict FIFO, as it
is difficult to compare priorities of jobs from different users. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message