Mailing-List: contact hdfs-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hdfs-issues@hadoop.apache.org
Message-ID: <738074360.1259948900713.JavaMail.jira@brutus>
Date: Fri, 4 Dec 2009 17:48:20 +0000 (UTC)
From: "Allen Wittenauer (JIRA)" <jira@apache.org>
To: hdfs-issues@hadoop.apache.org
Subject: [jira] Commented: (HDFS-808) Implement something like PAR2 support?
In-Reply-To: <1991207286.1259945060653.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HDFS-808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12786026#action_12786026 ] 

Allen Wittenauer commented on HDFS-808:
---------------------------------------

I'm thinking about the situation where you have the complete file except one or two blocks are completely missing (i.e., no replicas).  Using something like PAR2 you'd be able to reconstruct the missing block completely.

> Implement something like PAR2 support?
> --------------------------------------
>
>                 Key: HDFS-808
>                 URL: https://issues.apache.org/jira/browse/HDFS-808
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>            Reporter: Allen Wittenauer
>            Priority: Minor
>
> We really need an Idea issue type, because I'm not sure if this is really viable. :)  Just sort of thinking "out loud".
> I was thinking about how file recovery works on services like Usenet to fix data corruption when chunks of files are missing.  I wonder how hard it would be to implement something like PAR2 [ http://en.wikipedia.org/wiki/Parchive ] automatically for large files.  We'd have the advantage of being able to do it in binary of course and could likely hide the details within HDFS itself.  

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.