hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amareshwari Sri Ramadasu (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-2872) Default value for hadoop.job.history.location is broken
Date Fri, 22 Feb 2008 08:24:19 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-2872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12571304#action_12571304
] 

Amareshwari Sri Ramadasu commented on HADOOP-2872:
--------------------------------------------------

bq.  use the Path code:{code}
new Path(get("hadoop.log.dir", conf), get("hadoop.job.history.location"));
{code}
 This wouldnt work, because job history location need not be in hadoop.log.dir when it is
on dfs. By default it will be on local fs and in 'hadoop.log.dir'. So, its location has to
be specified as file:/// uri. 
I uploaded patch which fixes the case relative path in hadoop.log.dir at HADOOP-2871.  There
the location is built as
{code}
        LOG_DIR = conf.get("hadoop.job.history.location" ,   "file:///" + new File ( System.getProperty("hadoop.log.dir")).getAbsolutePath()
+ File.separator + "history");
{code}



> Default value for hadoop.job.history.location is broken
> -------------------------------------------------------
>
>                 Key: HADOOP-2872
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2872
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.17.0
>            Reporter: Owen O'Malley
>            Assignee: Amareshwari Sri Ramadasu
>             Fix For: 0.17.0
>
>
> The default value for hadoop.job.history.location is currently:
> file://${hadoop.log.dir}/history
> which assumes that hadoop.log.dir is an absolute path. If the path is relative, the job
tracker dies with:
> 08/02/21 23:43:04 FATAL mapred.JobTracker: java.lang.IllegalArgumentException: Wrong
FS: file://./logs/history, expected: file:///
>         at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:268)
>         at org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:54)
>         at org.apache.hadoop.fs.RawLocalFileSystem.exists(RawLocalFileSystem.java:223)
>         at org.apache.hadoop.fs.FilterFileSystem.exists(FilterFileSystem.java:147)
>         at org.apache.hadoop.mapred.JobHistory.init(JobHistory.java:124)
>         at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:706)
>         at org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:124)
>         at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:2146)
> I think that the right fix is to make the default value:
> "history"
> and when building, use the Path code:
> {code}
> new Path(get("hadoop.log.dir", conf), get("hadoop.job.history.location"));
> {code}

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message