hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Wang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11026) Convert BlockTokenIdentifier to use Protobuf
Date Tue, 13 Dec 2016 22:09:58 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15746429#comment-15746429

Andrew Wang commented on HDFS-11026:

Got it, thanks for the explanation. In this case, I think we should consider backporting dual-decoding
support to the 2.6 and 2.7 release lines as well. Historically, users have been more willing
to upgrade to a maintenance release than a new minor release.

Looking at the patch, it overall looks good. I'd like to see some tests though that verify
both legacy and PB modes are actually generating the expected bytes on the wire when the boolean
is passed. I see the read/write legacy/protobuf methods are annotated VisibleForTesting but
not tested directly.

* typo "deseriablize" in the readFields comment.
* BlockTokenSecretProto could also use a comment saying "only add optional/repeated fields",
since legacy tokens won't have these new fields.

Would be good for Daryn to review too, since I'm not really an expert on this part of the

> Convert BlockTokenIdentifier to use Protobuf
> --------------------------------------------
>                 Key: HDFS-11026
>                 URL: https://issues.apache.org/jira/browse/HDFS-11026
>             Project: Hadoop HDFS
>          Issue Type: Task
>          Components: hdfs, hdfs-client
>    Affects Versions: 2.9.0, 3.0.0-alpha1
>            Reporter: Ewan Higgs
>             Fix For: 3.0.0-alpha2
>         Attachments: HDFS-11026.002.patch, HDFS-11026.003.patch, blocktokenidentifier-protobuf.patch
> {{BlockTokenIdentifier}} currently uses a {{DataInput}}/{{DataOutput}} (basically a {{byte[]}})
and manual serialization to get data into and out of the encrypted buffer (in {{BlockKeyProto}}).
Other TokenIdentifiers (e.g. {{ContainerTokenIdentifier}}, {{AMRMTokenIdentifier}}) use Protobuf.
The {{BlockTokenIdenfitier}} should use Protobuf as well so it can be expanded more easily
and will be consistent with the rest of the system.
> NB: Release of this will require a version update since 2.8.x won't be able to decipher
{{BlockKeyProto.keyBytes}} from 2.8.y.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message