hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matei Zaharia (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5187) Provide an option to turn off priorities in jobs
Date Mon, 09 Feb 2009 04:06:59 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12671722#action_12671722

Matei Zaharia commented on HADOOP-5187:

Right now the FIFO mode is global and ignores pools altogether - it was meant to be a way
to switch back to FIFO at runtime if for some reason the fair sharing is buggy, not as a final
feature of the scheduler. This is why it used priorities, to act exactly like the old scheduler.
So what I meant is having a setting for each pool to determine whether it will run its jobs
in FIFO order or perform fair sharing between them, but always doing sharing between pools.
If we do this, then I think having an "ignore priorities" option per pool also works for the
default pool use case you mentioned above where the pool is a common resource and it doesn't
make sense to let users set priorities. 

> Provide an option to turn off priorities in jobs
> ------------------------------------------------
>                 Key: HADOOP-5187
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5187
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: contrib/fair-share
>            Reporter: Hemanth Yamijala
>            Priority: Minor
> The fairshare scheduler can define pools mapping to queues (as defined in the capacity
scheduler - HADOOP-3445). When used in this manner, one can imagine queues set up to be used
by users who come from disparate teams or organizations (say a default queue). For such a
queue, it makes sense to ignore job priorities and consider the queue as strict FIFO, as it
is difficult to compare priorities of jobs from different users. 

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message