hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Harsh J (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-2406) Failed validate copy in distcp
Date Sat, 09 Jul 2011 20:25:59 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-2406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13062613#comment-13062613
] 

Harsh J commented on MAPREDUCE-2406:
------------------------------------

Looking at validateCopy in trunk tells me that it does not seem to check anything beyond the
crc (if available, and if asked for) and the length of whole files. That doesn't seem to be
a cause for a failure due to differing block sizes here?

> Failed validate copy in distcp
> ------------------------------
>
>                 Key: MAPREDUCE-2406
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2406
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>    Affects Versions: 0.21.0
>            Reporter: Rosie Li
>            Priority: Minor
>
> Each time the distcp is done, {{validateCopy(srcstat, absdst)}} will be called. 
> When doing distcp, if the -pb(preserve block size) is not set, the dst will use the default
block size. However, if the src file use block size other than the default block size, and
-pb is not set, after copying, the src and dst will have different block size. It will not
pass the validateCopy check in this case.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message