hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eli Collins (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HDFS-808) Implement something like PAR2 support?
Date Fri, 04 Dec 2009 17:02:20 GMT

    [ https://issues.apache.org/jira/browse/HDFS-808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12785999#action_12785999

Eli Collins commented on HDFS-808:

You could use the parity blocks in the erasure encoding layer (HDFS-503) to implement the
parity blocks in PAR2, though given that HDFS has multiple replicas why are parity blocks
better than the current recovery mechanism (which just uses a non-corrupt block rather than
reconstructing the corrupt block from parity)?

> Implement something like PAR2 support?
> --------------------------------------
>                 Key: HDFS-808
>                 URL: https://issues.apache.org/jira/browse/HDFS-808
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>            Reporter: Allen Wittenauer
>            Priority: Minor
> We really need an Idea issue type, because I'm not sure if this is really viable. :)
 Just sort of thinking "out loud".
> I was thinking about how file recovery works on services like Usenet to fix data corruption
when chunks of files are missing.  I wonder how hard it would be to implement something like
PAR2 [ http://en.wikipedia.org/wiki/Parchive ] automatically for large files.  We'd have the
advantage of being able to do it in binary of course and could likely hide the details within
HDFS itself.  

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message