hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Douglas (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11026) Convert BlockTokenIdentifier to use Protobuf
Date Tue, 13 Dec 2016 18:08:59 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15745796#comment-15745796
] 

Chris Douglas commented on HDFS-11026:
--------------------------------------

v003 of the patch looks like a reasonable approach, particularly given the constraints we're
working with. It addresses backwards compatibility, and I'd like to commit this instead of
waiting for HDFS-11096 to shake out. 2.x/3.x rolling upgrades will be a journey, and [~ehiggs]'s
latest patch at least gives us a mechanism to evolve {{BlockTokenIdentifier}}.

[~andrew.wang], [~daryn]? If there's a way to validate this, we might want to consider it
for branch-2. For users upgrading their clusters to 3.x, we'll almost certainly require an
intermediate upgrade to a recent 2.x release. To [~daryn]'s earlier point, applications that
started in a 2.x environment will need to cope with 3.x tokens during a rolling upgrade.

> Convert BlockTokenIdentifier to use Protobuf
> --------------------------------------------
>
>                 Key: HDFS-11026
>                 URL: https://issues.apache.org/jira/browse/HDFS-11026
>             Project: Hadoop HDFS
>          Issue Type: Task
>          Components: hdfs, hdfs-client
>    Affects Versions: 2.9.0, 3.0.0-alpha1
>            Reporter: Ewan Higgs
>             Fix For: 3.0.0-alpha2
>
>         Attachments: HDFS-11026.002.patch, HDFS-11026.003.patch, blocktokenidentifier-protobuf.patch
>
>
> {{BlockTokenIdentifier}} currently uses a {{DataInput}}/{{DataOutput}} (basically a {{byte[]}})
and manual serialization to get data into and out of the encrypted buffer (in {{BlockKeyProto}}).
Other TokenIdentifiers (e.g. {{ContainerTokenIdentifier}}, {{AMRMTokenIdentifier}}) use Protobuf.
The {{BlockTokenIdenfitier}} should use Protobuf as well so it can be expanded more easily
and will be consistent with the rest of the system.
> NB: Release of this will require a version update since 2.8.x won't be able to decipher
{{BlockKeyProto.keyBytes}} from 2.8.y.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message