hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Konstantin Shvachko (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-738) dfs get or copyToLocal should not copy crc file
Date Tue, 28 Nov 2006 03:23:23 GMT
    [ http://issues.apache.org/jira/browse/HADOOP-738?page=comments#action_12453783 ] 
            
Konstantin Shvachko commented on HADOOP-738:
--------------------------------------------

I see two different issues here.
The first is that MR needs to copy locally dfs files and it turns out to be using local fs
for that.
And the second is that -put and -get commands of DFSShell are not symmetrical, meaning
when a user "puts" he doesn't need crc, while when he "gets" he sees a crc file.

> dfs get or copyToLocal should not copy crc file
> -----------------------------------------------
>
>                 Key: HADOOP-738
>                 URL: http://issues.apache.org/jira/browse/HADOOP-738
>             Project: Hadoop
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.8.0
>         Environment: all
>            Reporter: Milind Bhandarkar
>         Assigned To: Milind Bhandarkar
>             Fix For: 0.9.0
>
>         Attachments: hadoop-crc.patch
>
>
> Currently, when we -get or -copyToLocal a directory from DFS, all the files including
crc files are also copied. When we -put or -copyFromLocal again, since the crc files already
exist on DFS, this put fails. The solution is not to copy checksum files when copying to local.
Patch is forthcoming.

-- 
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message