hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod K V (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5187) Provide an option to turn off priorities in jobs
Date Wed, 11 Feb 2009 04:27:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12672500#action_12672500
] 

Vinod K V commented on HADOOP-5187:
-----------------------------------

Regarding distribution of excess capacity, it looks like a fairly common thing to do and can
be made pluggable - to distribute it evenly across pools, or according to pool weights, or
by how much pools are running behind etc.

It is looking to me that the design is moving more and more towards the objectives of HADOOP-5199.
As far as fair-scheduler is concerned, this seems like a big design change(though may not
be that big a code change). So, shouldn't we do all this as part of HADOOP-5199 and do a quick
fix to disable priorities in the current fair scheduler here?

My 2 cents. Thoughts?

> Provide an option to turn off priorities in jobs
> ------------------------------------------------
>
>                 Key: HADOOP-5187
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5187
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: contrib/fair-share
>            Reporter: Hemanth Yamijala
>            Priority: Minor
>
> The fairshare scheduler can define pools mapping to queues (as defined in the capacity
scheduler - HADOOP-3445). When used in this manner, one can imagine queues set up to be used
by users who come from disparate teams or organizations (say a default queue). For such a
queue, it makes sense to ignore job priorities and consider the queue as strict FIFO, as it
is difficult to compare priorities of jobs from different users. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message