hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Virajith Jalaparti (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11639) [READ] Encode the BlockAlias in the client protocol
Date Wed, 17 May 2017 20:14:05 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11639?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16014695#comment-16014695

Virajith Jalaparti commented on HDFS-11639:

Thanks for the updated patch [~ehiggs]! A couple of questions:

# Any particular reason for changing {{BlockProvider}} to implement  {{Iterable<BlockAlias>}}
from {{Iterable<Block>}}? 
# Was {{blockId}} intentionally left out of {{FileRegionProto}} even though {{FileRegion}}
contains it?

I fixed the following issues (along with some checkstyle fixes) and am posting a modified

- In {{ProvidedBlocksBuilder#newLocatedBlock}}, the following code: 

      FileRegion fileRegion;
      try {
        fileRegion = (FileRegion) blockProvider.resolve(eb.getLocalBlock());
      } catch (IOException e) {
        LOG.error("Could not resolve PROVIDED block: {}", e);
        fileRegion = null;

is moved inside the {{if(hasProvidedLocations)}} block.

- Modified {{Sender#transferBlock}} to add an null check for {{blockAlias}}

- Removed a redundant {{proto.build()}} in {{Sender#readBlock}}

> [READ] Encode the BlockAlias in the client protocol
> ---------------------------------------------------
>                 Key: HDFS-11639
>                 URL: https://issues.apache.org/jira/browse/HDFS-11639
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: hdfs
>            Reporter: Ewan Higgs
>            Assignee: Ewan Higgs
>         Attachments: HDFS-11639-HDFS-9806.001.patch, HDFS-11639-HDFS-9806.002.patch,
HDFS-11639-HDFS-9806.003.patch, HDFS-11639-HDFS-9806.004.patch
> As part of the {{PROVIDED}} storage type, we have a {{BlockAlias}} type which encodes
information about where the data comes from. i.e. URI, offset, length, and nonce value. This
data should be encoded in the protocol ({{LocatedBlockProto}} and the {{BlockTokenIdentifier}})
when a block is available using the PROVIDED storage type.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message