hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2744) Under some scenario, it is possible to end up with capacity scheduler configuration that uses labels that no longer exist
Date Mon, 27 Oct 2014 17:26:35 GMT

    [ https://issues.apache.org/jira/browse/YARN-2744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14185458#comment-14185458
] 

Wangda Tan commented on YARN-2744:
----------------------------------

As offline discussed with [~vinodkv], what this patch has done is not only fix for memory-based-config-store,
it is still possible that when we use filesystem-based-config-store, some labels will not
be validated. We should fix that.

> Under some scenario, it is possible to end up with capacity scheduler configuration that
uses labels that no longer exist
> -------------------------------------------------------------------------------------------------------------------------
>
>                 Key: YARN-2744
>                 URL: https://issues.apache.org/jira/browse/YARN-2744
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: capacityscheduler
>    Affects Versions: 2.5.1
>            Reporter: Sumit Mohanty
>            Assignee: Wangda Tan
>         Attachments: YARN-2744-20141025-1.patch, YARN-2744-20141025-2.patch
>
>
> Use the following steps:
> * Ensure default in-memory storage is configured for labels
> * Define some labels and assign nodes to labels (e.g. define two labels and assign both
labels to the host on a one host cluster)
> * Invoke refreshQueues
> * Modify capacity scheduler to create two top level queues and allow access to the labels
from both the queues
> * Assign appropriate "label + queue" specific capacities
> * Restart resource manager
> Noticed that RM starts without any issues. The labels are not preserved across restart
and thus the capacity-scheduler ends up using labels that are no longer present.
> At this point submitting an application to YARN will not succeed as there are no resources
available with the labels.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message