hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kai Zheng (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-11908) Erasure coding: Should be able to encode part of parity blocks.
Date Mon, 04 May 2015 01:59:06 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-11908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14526158#comment-14526158

Kai Zheng commented on HADOOP-11908:

Hi [~hitliuyi],

Thanks for the good input. Yes it's a good option to recover part of parity blocks via calling
raw encoder API, as I thought and planed before. In HADOOP-11847 it was allowed to decode
parity blocks as data blocks in raw decoder, and test cases were added as well. Would you
check with that and see if it works? Thanks.

> Erasure coding: Should be able to encode part of parity blocks.
> ---------------------------------------------------------------
>                 Key: HADOOP-11908
>                 URL: https://issues.apache.org/jira/browse/HADOOP-11908
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: io
>            Reporter: Yi Liu
>            Assignee: Kai Zheng
> {code}
> public void encode(ByteBuffer[] inputs, ByteBuffer[] outputs);
> {code}
> Currently when we do encode, the outputs are all parity blocks, we should be able to
encode part of parity blocks. 
> This is required when we do datanode striped block recovery, if one or more parity blocks
are missed, we need to do encode to recovery them. Only encode part of parity blocks should
be more efficient than all.

This message was sent by Atlassian JIRA

View raw message