hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kai Zheng (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8319) Erasure Coding: support decoding for stateful read
Date Tue, 02 Jun 2015 22:51:50 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14569927#comment-14569927

Kai Zheng commented on HDFS-8319:

Thanks Jing and Zhe for the good work and discussion here!

I really like {{StripeReader}} and the family. Could it be good to move on a little bit in
the direction? I thought eventually we will need a separate standalone high level construct
like {{StripeDecoder}} which mainly takes care of collecting inputs, perform decoding, and
processing outputs and etc. For now, maybe StrippeDecoder could be StripeDecoder, move the
relevant members to it, or have something like below
public class StripeDecoder {
  private final int cellSize;
  private final short dataBlkNum;
  private final short parityBlkNum;
  private final short groupSize;
  private final ECSchema schema;
  private final RawErasureDecoder decoder;

  StripeDecoder (ECSchema schema, int cellSize)  {
    assert schema != null;
    this.schema = schema;
    this.cellSize = cellSize;
    dataBlkNum = (short) schema.getNumDataUnits();
    parityBlkNum = (short) schema.getNumParityUnits();
    groupSize = dataBlkNum;
    decoder = new RSRawDecoder(dataBlkNum, parityBlkNum);
In future higher level {{ErasureCoder}} can be used to replace the current {{RawErasureCoder}}
based on such work, we won't need to involve much change then in {{DFSStripedInputStream}}.

In addition, PositionStripeReader could be PositionalStripeReader, similar to StatefulStripeReader.

> Erasure Coding: support decoding for stateful read
> --------------------------------------------------
>                 Key: HDFS-8319
>                 URL: https://issues.apache.org/jira/browse/HDFS-8319
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Jing Zhao
>            Assignee: Jing Zhao
>         Attachments: HDFS-8319.001.patch, HDFS-8319.002.patch
> HDFS-7678 adds the decoding functionality for pread. This jira plans to add decoding
to stateful read.

This message was sent by Atlassian JIRA

View raw message