hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Scott Chen (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-3361) Ability to use SimpleRegeratingCode to fix missing blocks
Date Mon, 07 Nov 2011 18:36:52 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-3361?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13145682#comment-13145682
] 

Scott Chen commented on MAPREDUCE-3361:
---------------------------------------

I think one thing we need to do is to refactor the raid.Encoder and raid.Decoder. So they
are generic to any ErasureCode. That way we can easily add different codes to Raid.
                
> Ability to use SimpleRegeratingCode to fix missing blocks
> ---------------------------------------------------------
>
>                 Key: MAPREDUCE-3361
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3361
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: contrib/raid
>            Reporter: dhruba borthakur
>            Assignee: dhruba borthakur
>
> ReedSolomon encoding (n, k) has n storage nodes and can tolerate n-k failures. Regenerating
a block needs to access k blocks. This is a problem when n and k are large. Instead, we can
use simple regenerating codes (n, k, f) that does first does ReedSolomon (n,k) and then does
XOR with f stripe size. Then, a single disk failure needs to access only f nodes and f can
be very small.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message