hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Walter Su (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-8453) Erasure coding: properly assign start offset for internal blocks in a block group
Date Wed, 27 May 2015 01:19:17 GMT

    [ https://issues.apache.org/jira/browse/HDFS-8453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14560258#comment-14560258

Walter Su commented on HDFS-8453:

I didn't see any difference between {{refreshLocatedBlock}} and old version's {{getBlockAt}}
This is old version
protected LocatedBlock getBlockAt(long blkStartOffset) 
This is new version
protected LocatedBlock refreshLocatedBlock(LocatedBlock block) {
  LocatedBlock lb = getBlockGroupAt(block.getStartOffset());
You didn't pass blkStartOffset, you get blkStartOffset from inside. So there is no difference.
This line really matters. It resolves the issue.
+          bg.getStartOffset(), bg.isCorrupt(), null);
The solution is to make it meaningless.

> Erasure coding: properly assign start offset for internal blocks in a block group
> ---------------------------------------------------------------------------------
>                 Key: HDFS-8453
>                 URL: https://issues.apache.org/jira/browse/HDFS-8453
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>            Reporter: Zhe Zhang
>            Assignee: Zhe Zhang
>         Attachments: HDFS-8453-HDFS-7285.00.patch
> {code}
>   void actualGetFromOneDataNode(final DNAddrPair datanode,
>     ...
>       LocatedBlock block = getBlockAt(blockStartOffset);
>     ...
>       fetchBlockAt(block.getStartOffset());
> {code}
> The {{blockStartOffset}} here is from inner block. For parity blocks, the offset will
overlap with the next block group, and we may end up with fetching wrong block. So we have
to assign a meaningful start offset for internal blocks in a block group, especially for parity

This message was sent by Atlassian JIRA

View raw message