hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ewan Higgs (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11956) Fix BlockToken compatibility with Hadoop 2.x clients
Date Thu, 22 Jun 2017 22:08:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16060089#comment-16060089
] 

Ewan Higgs commented on HDFS-11956:
-----------------------------------

Hi Andrew

{quote}
IIUC, we know the storage type even for an old client since it passes it in the writeBlock
request. Can an old client correctly pass along an unknown StorageType (e.g. PROVIDED)? 
{quote}
I think you understood correctly. I don't think an old client will be able to deserialise
a PROVIDED StorageType from the protobuf, so it will fail to pass along that StorageType (though
I have not yet done the cross-version testing with Hadoop 2.6). I think this is the same as
would be the case any time a new StorageType is introduced (e.g. if we hypothetically added
{{StorageType.NVME}}, {{StorageType.SMR}}, etc.). Maybe the forward compatibility of StorageTypes
is another JIRA I should raise?

{quote}
If so, then I see how this works; essentially, only require storageIDs when writing to provided
storage.
{quote}
Yes.

{quote}
For 3.0.0-alpha4 I can also revert HDFS-9807 while we figure out this JIRA. We did this internally
to unblock testing.
{quote}
I'm traveling today so I won't be able to furnish a patch just yet. What's your time frame
for tagging alpha4?

> Fix BlockToken compatibility with Hadoop 2.x clients
> ----------------------------------------------------
>
>                 Key: HDFS-11956
>                 URL: https://issues.apache.org/jira/browse/HDFS-11956
>             Project: Hadoop HDFS
>          Issue Type: Bug
>    Affects Versions: 3.0.0-alpha4
>            Reporter: Andrew Wang
>            Assignee: Ewan Higgs
>            Priority: Blocker
>         Attachments: HDFS-11956.001.patch, HDFS-11956.002.patch, HDFS-11956.003.patch
>
>
> Seems like HDFS-9807 broke backwards compatibility with Hadoop 2.x clients. When talking
to a 3.0.0-alpha4 DN with security on:
> {noformat}
> 2017-06-06 23:27:22,568 WARN org.apache.hadoop.hdfs.server.datanode.DataNode: Block token
verification failed: op=WRITE_BLOCK, remoteAddress=/172.28.208.200:53900, message=Block token
with StorageIDs [DS-c0f24154-a39b-4941-93cd-5b8323067ba2] not valid for access with StorageIDs
[]
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message