hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Johan Oskarson (JIRA)" <j...@apache.org>
Subject [jira] Created: (HADOOP-515) Rsync like way of retrieving data from the dfs
Date Fri, 08 Sep 2006 13:17:22 GMT
Rsync like way of retrieving data from the dfs

                 Key: HADOOP-515
                 URL: http://issues.apache.org/jira/browse/HADOOP-515
             Project: Hadoop
          Issue Type: New Feature
          Components: dfs
    Affects Versions: 0.5.0
            Reporter: Johan Oskarson

As the dfs in our cluster contains a lot of important data, being able to retrieve them to
a non dfs backup node is essential.
However, a lot of the files don't change inbetween backups, so a way to get only the files
that have changed would be preferable.

Since the blocks themselves already have a crc calculated half the job is already done, if
it's possible to split the destination files in similar blocks and calculate the crc for them.

This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message