hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Allen Wittenauer (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-12563) Updated utility to create/modify token files
Date Fri, 11 Dec 2015 19:29:46 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-12563?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15053412#comment-15053412
] 

Allen Wittenauer commented on HADOOP-12563:
-------------------------------------------

Ping [~owen.omalley] to help review this. ;)

I haven't had a chance to apply and execute, but some feedback based upon visual inspection:

1) In hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/Credentials.java

{code}
@InterfaceAudience.LimitedPrivate({"HDFS", "MapReduce"})
{code}

Not part of this patch, but clearly wrong nonetheless especially with YARN-4435 in the pipeline.
 We should update it to include YARN while we're here.

2)
writeLegacyTokenStorageFile, etc.

I think I'd rather see these called something with version 0 or java serialization or something
else.  This way if there is ever a version 2 (we drop protobuf?), we're covered.  Bonus points
if we could somehow tie the dtutil -format option to the methods and version.

3) TestDtUtilShell.java:
System.getProperty("test.build.data", "/tmp"), "TestDtUtilShell");

Let's set this to target/ instead of /tmp to be less racy with multiple unit tests running
on the same machine.

Thanks for fixing the service name in the usage. :)


> Updated utility to create/modify token files
> --------------------------------------------
>
>                 Key: HADOOP-12563
>                 URL: https://issues.apache.org/jira/browse/HADOOP-12563
>             Project: Hadoop Common
>          Issue Type: New Feature
>    Affects Versions: 3.0.0
>            Reporter: Allen Wittenauer
>            Assignee: Matthew Paduano
>         Attachments: HADOOP-12563.01.patch, HADOOP-12563.02.patch, HADOOP-12563.03.patch,
HADOOP-12563.04.patch, HADOOP-12563.05.patch, example_dtutil_commands_and_output.txt, generalized_token_case.pdf
>
>
> hdfs fetchdt is missing some critical features and is geared almost exclusively towards
HDFS operations.  Additionally, the token files that are created use Java serializations which
are hard/impossible to deal with in other languages. It should be replaced with a better utility
in common that can read/write protobuf-based token files, has enough flexibility to be used
with other services, and offers key functionality such as append and rename. The old version
file format should still be supported for backward compatibility, but will be effectively
deprecated.
> A follow-on JIRA will deprecrate fetchdt.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message