hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ravi Gummadi (JIRA)" <j...@apache.org>
Subject [jira] Updated: (MAPREDUCE-644) distcp does not skip copying file if we are updating single file
Date Wed, 09 Sep 2009 08:31:57 GMT

     [ https://issues.apache.org/jira/browse/MAPREDUCE-644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Ravi Gummadi updated MAPREDUCE-644:
-----------------------------------

    Attachment: d_singlefile_update644.patch

Attaching new patch with testcase added.
Testcase checks if setup() returns false as expected for the case of 'single file update 
with destination containing the file already'(Also had to make class Arguments and setup()
method as package private --- to be accessed from testcase). This testcase works only with
the fix of MAPREDUCE-648. So marking that as a blocker for this JIRA.

This patch applies only after MAPREDUCE-649, MAPREDUCE-654, MAPREDUCE-645, MAPREDUCE-664,
MAPREDUCE-661, MAPREDUCE-650, MAPREDUCE-648, MAPREDUCE-564 and MAPREDUCE-653(as these patches
are all created in that order, I want this patch to go in after them --- otherwise many of
those patches will have to be recreated for applying).

> distcp does not skip copying file if we are updating single file
> ----------------------------------------------------------------
>
>                 Key: MAPREDUCE-644
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-644
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: distcp
>            Reporter: Ravi Gummadi
>            Assignee: Ravi Gummadi
>         Attachments: d_singlefile_update.patch, d_singlefile_update644.patch
>
>
> distcp doesn't skip copying file when we do -update on single file if the destfile already
exists.
> When we do 
> hadoop distcp -update srcfilename destfilename
> it seems to be comparing checksums of srcfilename and destfilename/srcfilename and so
skip is not done. It should compare checksums of srcfilename and destfilename.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message