hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-738) dfs get or copyToLocal should not copy crc file
Date Tue, 28 Nov 2006 18:48:24 GMT
    [ http://issues.apache.org/jira/browse/HADOOP-738?page=comments#action_12454070 ] 
Doug Cutting commented on HADOOP-738:

> I see two different issues here.

Me too, but I'm not sure they are the same two.

One is a bug.  From the original description, "When we -put, since the crc files already exist
on DFS, this put fails."  When local CRC files are enabled, a -get followed by a -put of the
same file should not fail, but apparently it does.  This should work, no?

The other is a feature request to be able to disable CRC creation in -get.  I'm okay with
this feature.

I'd rather fix the bug before we introduce the feature, especially if CRC-free -get becomes
the default, since it masks the bug.

> dfs get or copyToLocal should not copy crc file
> -----------------------------------------------
>                 Key: HADOOP-738
>                 URL: http://issues.apache.org/jira/browse/HADOOP-738
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.8.0
>         Environment: all
>            Reporter: Milind Bhandarkar
>         Assigned To: Milind Bhandarkar
>             Fix For: 0.9.0
>         Attachments: hadoop-crc.patch
> Currently, when we -get or -copyToLocal a directory from DFS, all the files including
crc files are also copied. When we -put or -copyFromLocal again, since the crc files already
exist on DFS, this put fails. The solution is not to copy checksum files when copying to local.
Patch is forthcoming.

This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message