hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-9525) hadoop utilities need to support provided delegation tokens
Date Fri, 29 Jan 2016 21:42:39 GMT

    [ https://issues.apache.org/jira/browse/HDFS-9525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15124263#comment-15124263

Steve Loughran commented on HDFS-9525:

Catching up on this by way of looking at UGI and seeing some new code there that I wasn't

h2. sysprops vs config options

{{"hadoop.token.files"}} is not a core-default file, it is a system property. 

Adding a core-default entry here is misleading, as it will make people believe that they can
set token files this way. Remove and fix the javadocs to match.

h2. documentation

We now have yet another undocumented configuratin point for Hadoop security, while I am still
trying to understand what there was already. Please document in hadoop security docs

h2. logging and error reporting

Add some more logging too. Print out the files before they are loaded? Please.

Finally, why skip files that aren't there or aren't files? Isn't that a sign of an error?
At the very least, WARN. Otherwise, someone —and I fear it shall be me— will end up trying
to debug why a launched YARN app hasn't picked up credentials from oozie, with the cause being
a typo in the path *which was logged at all*

h3. integration with {{HADOOP_TOKEN_FILE_LOCATION}},
w.r.t {{HADOOP_TOKEN_FILE_LOCATION}}, that has the advantage of working with non-java apps.
What may be nice would be for both  {{HADOOP_TOKEN_FILE_LOCATION}} and {{"hadoop.token.files"}}
to have the same processing logic.

you'd go 
String files = System.getProperty("hadoop.token.files", System.getEnv("HADOOP_TOKEN_FILE_LOCATION"))
the env would get picked up, the sysprop override. Then have one followon codepath with the
logging I mentioned earlier.

As it is, there's now the situation that both options can be set. Is that really what is wanted?

> hadoop utilities need to support provided delegation tokens
> -----------------------------------------------------------
>                 Key: HDFS-9525
>                 URL: https://issues.apache.org/jira/browse/HDFS-9525
>             Project: Hadoop HDFS
>          Issue Type: New Feature
>          Components: security
>    Affects Versions: 3.0.0
>            Reporter: Allen Wittenauer
>            Assignee: HeeSoo Kim
>            Priority: Blocker
>             Fix For: 3.0.0
>         Attachments: HDFS-7984.001.patch, HDFS-7984.002.patch, HDFS-7984.003.patch, HDFS-7984.004.patch,
HDFS-7984.005.patch, HDFS-7984.006.patch, HDFS-7984.007.patch, HDFS-7984.patch, HDFS-9525.008.patch,
HDFS-9525.009.patch, HDFS-9525.009.patch, HDFS-9525.branch-2.008.patch, HDFS-9525.branch-2.009.patch
> When using the webhdfs:// filesystem (especially from distcp), we need the ability to
inject a delegation token rather than webhdfs initialize its own.  This would allow for cross-authentication-zone
file system accesses.

This message was sent by Atlassian JIRA

View raw message