hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sunil G (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4362) Too many preemption activity when nodelabels are non exclusive
Date Tue, 17 Nov 2015 15:18:11 GMT

    [ https://issues.apache.org/jira/browse/YARN-4362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15008812#comment-15008812

Sunil G commented on YARN-4362:

Ideally at first thought we should fix at preemption logic as more preemption is happening
there. But in this specific case, some non-used partition has resources in it, and preemption
logic will always  mark this as over-used resources.
I am not very much inclined in having a new non-exclusive partition which makes allocation
more complex (we have no_label already for general cases). However I feel we can come with
a default allocation limit/unit per partition even though it is not used/configured.
Hence in cases like this, guaranteed resources wont be Zero. But when real allocation starts
happening to that label, this default allocation must be removed/shrinked. With this we can
get some flexibility in using these free nodes w/o violating node label strict labeling concept
(only no_label can use this limit).

If we do not allocate to these labels, it will be more stricter and these resource will be
non-usable. This is also acceptable but comes with the cost of some non-usable nodes.

> Too many preemption activity when nodelabels are non exclusive
> --------------------------------------------------------------
>                 Key: YARN-4362
>                 URL: https://issues.apache.org/jira/browse/YARN-4362
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Bibin A Chundatt
>            Assignee: Varun Saxena
>            Priority: Critical
>         Attachments: Preemptedpartition.log, ProportionalDefaultQueue.log, ProportionalPolicy.log,
> Steps to reproduce
> ===============
> 1.Configure HA cluster with 6 nodes and 3 partition(1,2,3) all non exclusive
> *Partition configuration is as follows*
> 1,2 NM's mapped with Label 1
> NM 3  to  label 2
> 4,5 NM's mapped to Label 3
> NM 6 in DEFAULT partition
> In capacity scheduler the queue are linked only to 1,3 partition.
> The NM 3 with label 2 is a backup node for any partition whenever required will change
the label.
> Submit and application/job with 200 containers to default queue.
> All containers that gets assigned to partition 2 gets preempted 
> The application/map task execution is taking more time since 30-40 task gets assigned
to partition 2 then gets preempted and all of them needs to be relaunched.

This message was sent by Atlassian JIRA

View raw message