hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Peter Romianowski (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1285) DistCp cannot handle -delete if destination is local filesystem
Date Wed, 09 Dec 2009 19:30:18 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12788254#action_12788254

Peter Romianowski commented on MAPREDUCE-1285:

I don't think that the patch breaks anything at this place. I only changed a single line in
deleteNonexisting() where FilePair is not even used. The problem has been that the SequenceFile.Writer
was tied to FileStatus.class but it didn't have to since we know the concrete implementation.

So the patch should be save even if someone introduces new fields in RawLocalFileStatus.

Did I miss something?

> DistCp cannot handle -delete if destination is local filesystem
> ---------------------------------------------------------------
>                 Key: MAPREDUCE-1285
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1285
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: distcp
>    Affects Versions: 0.20.1
>            Reporter: Peter Romianowski
>         Attachments: MAPREDUCE-1285.patch
> The following exception is thrown:
> {code}
> Copy failed: java.io.IOException: wrong value class: org.apache.hadoop.fs.RawLocalFileSystem$RawLocalFileStatus
is not class org.apache.hadoop.fs.FileStatus
> 	at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:988)
> 	at org.apache.hadoop.io.SequenceFile$Writer.append(SequenceFile.java:977)
> 	at org.apache.hadoop.tools.DistCp.deleteNonexisting(DistCp.java:1226)
> 	at org.apache.hadoop.tools.DistCp.setup(DistCp.java:1134)
> 	at org.apache.hadoop.tools.DistCp.copy(DistCp.java:650)
> 	at org.apache.hadoop.tools.DistCp.run(DistCp.java:857)
> 	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> 	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> {code}

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message