hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Doug Cutting (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-738) dfs get or copyToLocal should not copy crc file
Date Tue, 28 Nov 2006 18:48:24 GMT
    [ http://issues.apache.org/jira/browse/HADOOP-738?page=comments#action_12454070 ] 
            
Doug Cutting commented on HADOOP-738:
-------------------------------------

> I see two different issues here.

Me too, but I'm not sure they are the same two.

One is a bug.  From the original description, "When we -put, since the crc files already exist
on DFS, this put fails."  When local CRC files are enabled, a -get followed by a -put of the
same file should not fail, but apparently it does.  This should work, no?

The other is a feature request to be able to disable CRC creation in -get.  I'm okay with
this feature.

I'd rather fix the bug before we introduce the feature, especially if CRC-free -get becomes
the default, since it masks the bug.


> dfs get or copyToLocal should not copy crc file
> -----------------------------------------------
>
>                 Key: HADOOP-738
>                 URL: http://issues.apache.org/jira/browse/HADOOP-738
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.8.0
>         Environment: all
>            Reporter: Milind Bhandarkar
>         Assigned To: Milind Bhandarkar
>             Fix For: 0.9.0
>
>         Attachments: hadoop-crc.patch
>
>
> Currently, when we -get or -copyToLocal a directory from DFS, all the files including
crc files are also copied. When we -put or -copyFromLocal again, since the crc files already
exist on DFS, this put fails. The solution is not to copy checksum files when copying to local.
Patch is forthcoming.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message