hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ewan Higgs (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11956) Fix BlockToken compatibility with Hadoop 2.x clients
Date Thu, 22 Jun 2017 22:08:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060089#comment-16060089

Ewan Higgs commented on HDFS-11956:

Hi Andrew

IIUC, we know the storage type even for an old client since it passes it in the writeBlock
request. Can an old client correctly pass along an unknown StorageType (e.g. PROVIDED)? 
I think you understood correctly. I don't think an old client will be able to deserialise
a PROVIDED StorageType from the protobuf, so it will fail to pass along that StorageType (though
I have not yet done the cross-version testing with Hadoop 2.6). I think this is the same as
would be the case any time a new StorageType is introduced (e.g. if we hypothetically added
{{StorageType.NVME}}, {{StorageType.SMR}}, etc.). Maybe the forward compatibility of StorageTypes
is another JIRA I should raise?

If so, then I see how this works; essentially, only require storageIDs when writing to provided

For 3.0.0-alpha4 I can also revert HDFS-9807 while we figure out this JIRA. We did this internally
to unblock testing.
I'm traveling today so I won't be able to furnish a patch just yet. What's your time frame
for tagging alpha4?

> Fix BlockToken compatibility with Hadoop 2.x clients
> ----------------------------------------------------
>                 Key: HDFS-11956
>                 URL: https://issues.apache.org/jira/browse/HDFS-11956
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 3.0.0-alpha4
>            Reporter: Andrew Wang
>            Assignee: Ewan Higgs
>            Priority: Blocker
>         Attachments: HDFS-11956.001.patch, HDFS-11956.002.patch, HDFS-11956.003.patch
> Seems like HDFS-9807 broke backwards compatibility with Hadoop 2.x clients. When talking
to a 3.0.0-alpha4 DN with security on:
> {noformat}
> 2017-06-06 23:27:22,568 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Block token
verification failed: op=WRITE_BLOCK, remoteAddress=/, message=Block token
with StorageIDs [DS-c0f24154-a39b-4941-93cd-5b8323067ba2] not valid for access with StorageIDs
> {noformat}

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message