hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Craig Welch (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2803) MR distributed cache not working correctly on Windows after NodeManager privileged account changes.
Date Tue, 04 Nov 2014 22:40:35 GMT

    [ https://issues.apache.org/jira/browse/YARN-2803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14197005#comment-14197005
] 

Craig Welch commented on YARN-2803:
-----------------------------------

So, having looked at this a bit (with some help from [~cnauroth]) I've added some logging
and it looks to be some kind of pathing issue, the files are there, just not quite how the
classloader expects them to be.  

2014-11-04 14:22:53,956 INFO [main] org.apache.hadoop.mapreduce.v2.TestMRJobs: THE PATH TO
THE PLACE IS file:/C:/boo/1415139741659/org.apache.hadoop.mapreduce.v2.TestMRJobs-localDir-nm-0_0/usercache/cwelch/appcache/application_1415139755128_0001/container_1415139755128_0001_01_000001/job.jar/job.jar/^M

(ignore the c:/boo, that was a different troubleshooting step (which made no difference...))
 notice the duplicated job.jar/job.jar - the files are present in the higher level "job.jar"
directory, but not in the duplicated path, so in some form or fashion an additional level
is being added to the path now, it's not clear to me why as of yet.  Continuing to investigate.

> MR distributed cache not working correctly on Windows after NodeManager privileged account
changes.
> ---------------------------------------------------------------------------------------------------
>
>                 Key: YARN-2803
>                 URL: https://issues.apache.org/jira/browse/YARN-2803
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: nodemanager
>            Reporter: Chris Nauroth
>            Priority: Critical
>
> This problem is visible by running {{TestMRJobs#testDistributedCache}} or {{TestUberAM#testDistributedCache}}
on Windows.  Both tests fail.  Running git bisect, I traced it to the YARN-2198 patch to remove
the need to run NodeManager as a privileged account.  The tests started failing when that
patch was committed.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message