hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ming Ma (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-5072) Support comma separated list of includes and excludes files
Date Thu, 12 May 2016 17:10:13 GMT

     [ https://issues.apache.org/jira/browse/YARN-5072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Ming Ma updated YARN-5072:
    Attachment: YARN-5072.patch

Here is the draft path. YARN and HDFS use HostsFileReader differently. The patch includes
changes changes to enable end-to-end support for YARN as well as necessary changes in HostsFileReader
to support HDFS. While it is possible to make {{HostsFileReader.readFileToSet}} to support
multiple files so that it automatically works for HDFS, it seems better to create a separate
HDFS jira to include things like config description updates.

> Support comma separated list of includes and excludes files
> -----------------------------------------------------------
>                 Key: YARN-5072
>                 URL: https://issues.apache.org/jira/browse/YARN-5072
>             Project: Hadoop YARN
>          Issue Type: Improvement
>            Reporter: Ming Ma
>         Attachments: YARN-5072.patch
> When a yarn cluster shares the same hosts as the underlying HDFS cluster, we have {{yarn.resourcemanager.nodes.include-path}}
point to the same file or symlink of the {{dfs.hosts}} file used by HDFS to make admin easier.
> If we want to set up a yarn cluster to run on the same hosts of several HDFS clusters
combined, it means {{yarn.resourcemanager.nodes.include-path}} should be able to point to
a list of files each of which belongs to one HDFS cluster.
> Backward compatibility, it seems ok to continue to reuse {{yarn.resourcemanager.nodes.include-path}}
as long as it can still take a single file. 

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message