hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hemanth Yamijala (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5828) Use absolute path for JobTracker's mapred.local.dir in MiniMRCluster
Date Thu, 14 May 2009 05:40:45 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12709272#action_12709272

Hemanth Yamijala commented on HADOOP-5828:

The failure was in DiskChecker.checkDir(), which called File.isDirectory() on the relative
mapred.local.dir path. Debugging this, we found the issue to be one documented in http://bugs.sun.com/bugdatabase/view_bug.do;:YfiG?bug_id=4483097.
When relative paths are passed to java.io.* APIs, some of them resolve them to absolute paths
by prepending the path with the value of the system property user.dir. Some of them resolve
them relative to the current working directory.

For contrib tests, we set the user.dir to a value that will typically be different from the
current working directory. This triggers inconsistencies in the resolution to absolute files
and causes APIs like File.isDirectory() to fail.

The recommendation from Sun is to use absolute rather than relative paths for such cases.
Hence, the issue.

> Use absolute path for JobTracker's mapred.local.dir in MiniMRCluster
> --------------------------------------------------------------------
>                 Key: HADOOP-5828
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5828
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: test
>            Reporter: Hemanth Yamijala
>            Assignee: Hemanth Yamijala
> MiniMRCluster's JobTracker sets the mapred.local.dir to a relative path: "build/test/mapred/local".
While currently this does not cause any problem, if at some time in future, the JobTracker
starts using LocalDirAllocator for localizing files, this could cause failures in contrib
tests. This issue was faced when working with an internal patch that tried to introduce LocalDirAllocator
into the JobTracker, and capacity scheduler test cases started failing.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message