hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsz Wo (Nicholas), SZE (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (MAPREDUCE-2463) Job History files are not moving to done folder when job history location is hdfs location
Date Thu, 28 Jul 2011 04:15:10 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-2463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Tsz Wo (Nicholas), SZE updated MAPREDUCE-2463:
----------------------------------------------

    Resolution: Fixed
        Status: Resolved  (was: Patch Available)

I have committed this.  Thanks, Devaraj!

> Job History files are not moving to done folder when job history location is hdfs location
> ------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-2463
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2463
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: jobtracker
>    Affects Versions: 0.23.0
>            Reporter: Devaraj K
>            Assignee: Devaraj K
>             Fix For: 0.23.0
>
>         Attachments: MAPREDUCE-2463-1.patch, MAPREDUCE-2463-2.patch, MAPREDUCE-2463.patch
>
>
> If "mapreduce.jobtracker.jobhistory.location" is configured as HDFS location then either
during initialization of Job Tracker (while moving old job history files) or after completion
of the job, history files are not moving to done and giving following exception.
> {code:xml} 
> 2011-04-29 15:27:27,813 ERROR org.apache.hadoop.mapreduce.jobhistory.JobHistory: Unable
to move history file to DONE folder.
> java.lang.IllegalArgumentException: Wrong FS: hdfs://10.18.52.146:9000/history/job_201104291518_0001_root,
expected: file:///
> 	at org.apache.hadoop.fs.FileSystem.checkPath(FileSystem.java:402)
> 	at org.apache.hadoop.fs.RawLocalFileSystem.pathToFile(RawLocalFileSystem.java:58)
> 	at org.apache.hadoop.fs.RawLocalFileSystem.getFileStatus(RawLocalFileSystem.java:419)
> 	at org.apache.hadoop.fs.FilterFileSystem.getFileStatus(FilterFileSystem.java:294)
> 	at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:215)
> 	at org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1516)
> 	at org.apache.hadoop.fs.FileSystem.copyFromLocalFile(FileSystem.java:1492)
> 	at org.apache.hadoop.fs.FileSystem.moveFromLocalFile(FileSystem.java:1482)
> 	at org.apache.hadoop.mapreduce.jobhistory.JobHistory.moveToDoneNow(JobHistory.java:348)
> 	at org.apache.hadoop.mapreduce.jobhistory.JobHistory.access$200(JobHistory.java:61)
> 	at org.apache.hadoop.mapreduce.jobhistory.JobHistory$1.run(JobHistory.java:439)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
> 	at java.lang.Thread.run(Thread.java:619)
> {code} 

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message