hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhe Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8319) Erasure Coding: support decoding for stateful read
Date Thu, 04 Jun 2015 23:22:38 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8319?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14573776#comment-14573776
] 

Zhe Zhang commented on HDFS-8319:
---------------------------------

# Follow-on: {{divideOneStripe}} can be further simplified. The argument {{rangeEndInBlockGroup}}
is not necessary since it can be calculated from {{rangeStartInBlockGroup}} and cell size.
# I think the current {{blockReaderOffsets}} logic works correctly. IIUC each offset in the
array represents the actual {{startOffset}} of the block reader (looks like {{BlockReader}}
should ideally have a getter for this internal state?). For better readability, maybe we can
just emulate how the block reader offset evolves?
#* Is it possible that a failed or canceled request has also changed the internal state of
the block reader? If so we might {{skip}} too much.
#* Beside the above question, the main change I'd suggest is to add an assert and increment
the offset in {{StatefulStripeReader#updateState4SuccessRead}}:
{code}
Preconditions.checkState(blockReaderOffsets[result.index] == alignedStripe.getOffsetInBlock());
blockReaderOffsets[result.index] += alignedStripe.getSpanInBlock();
{code}
#* Nit: in {{prepareBlockReader}}, {{blockReaderOffsets\[i\] = alignedStripe.getOffsetInBlock()}}
can be changed to {{offsetInBlock}} to be consistent with the {{getBlockReaderWithRetry}}
call.
# {{prepareBlockReader}} can be {{prepareParityBlockReader}}?

> Erasure Coding: support decoding for stateful read
> --------------------------------------------------
>
>                 Key: HDFS-8319
>                 URL: https://issues.apache.org/jira/browse/HDFS-8319
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Jing Zhao
>            Assignee: Jing Zhao
>         Attachments: HDFS-8319.001.patch, HDFS-8319.002.patch, HDFS-8319.003.patch, HDFS-8319.004.patch
>
>
> HDFS-7678 adds the decoding functionality for pread. This jira plans to add decoding
to stateful read.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message