Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Wed, 14 Jun 2017 10:06:00 +0000 (UTC)
From: "Bibin A Chundatt (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.13079693.1497429192000.15902.1497434760855@Atlassian.JIRA>
In-Reply-To: <JIRA.13079693.1497429192000@Atlassian.JIRA>
References: <JIRA.13079693.1497429192000@Atlassian.JIRA> <JIRA.13079693.1497429192838@jira-lw-us.apache.org>
Subject: [jira] [Commented] (YARN-6711) log aggregation can't use other
 filesystem instead of fs.defaultFS
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Wed, 14 Jun 2017 10:06:07 -0000


    [ https://issues.apache.org/jira/browse/YARN-6711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16049002#comment-16049002 ] 

Bibin A Chundatt commented on YARN-6711:
----------------------------------------

Does YARN-3269 handle the issue.??

> log aggregation can't use other filesystem instead of fs.defaultFS
> ------------------------------------------------------------------
>
>                 Key: YARN-6711
>                 URL: https://issues.apache.org/jira/browse/YARN-6711
>             Project: Hadoop YARN
>          Issue Type: Bug
>    Affects Versions: 2.7.1, 3.0.0-alpha3
>            Reporter: KWON BYUNGCHANG
>            Priority: Minor
>
> hdfs support federated namespace.
> fs.defaultFS should choice one of federated namespace.
> however if {{yarn.nodemanager.remote-app-log-dir}} does not belong to default fs, 
> nodemanager will stop.
> absolutely log-aggregation should can use other namespace instead of default fs.
> below is error log
> configuration is
> {quote}
>   fs.defaultFS=hdfs://x3
>   yarn.nodemanager.remote-app-log-dir=hdfs://x3v/logs
> {quote}
> {code}
> 2017-06-14 15:59:47,792 FATAL org.apache.hadoop.yarn.event.AsyncDispatcher: Error in dispatcher thread
> java.lang.IllegalArgumentException: Wrong FS: hdfs://x3v/logs, expected: hdfs://x3
>     at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:646)
>     at org.apache.hadoop.hdfs.DistributedFileSystem.getPathName(DistributedFileSystem.java:194)
>     at org.apache.hadoop.hdfs.DistributedFileSystem.access$000(DistributedFileSystem.java:106)
>     at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1305)
>     at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:1301)
>     at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
>     at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1301)
>     at org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.verifyAndCreateRemoteLogDir(LogAggregationService.java:192)
>     at org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.initApp(LogAggregationService.java:319)
>     at org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.handle(LogAggregationService.java:443)
>     at org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.handle(LogAggregationService.java:67)
>     at org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:175)
>     at org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:108)
>     at java.lang.Thread.run(Thread.java:745)
> 2017-06-14 15:59:47,794 INFO org.apache.hadoop.yarn.event.AsyncDispatcher: Exiting, bbye..
> {code}
>  


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org