hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rushabh S Shah (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-11702) Remove indefinite caching of key provider uri in DFSClient
Date Fri, 28 Apr 2017 00:49:04 GMT

    [ https://issues.apache.org/jira/browse/HDFS-11702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15987959#comment-15987959

Rushabh S Shah commented on HDFS-11702:

Thanks [~hanishakoneru] for looking into this jira.
bq.  the patch doesn't apply cleanly for me.
Which branch are you applying on ?
This patch is for trunk. I haven't tested on 2.8 or 2.7.
Given that jenkins build didn't complain about applying/compiling, that makes me think maybe
something wrong in your environment ?

bq. Every time DFSClient#getKeyProviderUri is called, keyProviderUri is set to null and recalulated.
There is no caching being done.
If you notice, then there are mainly 2 places to get keyprovider uri other than local conf.
1.     byte[] keyProviderUriBytes = credentials.getSecretKey(getKeyProviderMapKey());
First if dfs client is in a task and if EZ was enabled during job submission then the keyprovider
from which kms token was fetched is added to credentials object.

2.     FsServerDefaults serverDefaults = getServerDefaults();
If you notice the {{getServerDefaults}} carefully, then the caching is done there. The cache
is good for an hour after which it will again call Namenode#getServerDefaults.
Relevant piece of code
  public FsServerDefaults getServerDefaults() throws IOException {
    long now = Time.monotonicNow();
    if ((serverDefaults == null) ||
        (now - serverDefaultsLastUpdate > SERVER_DEFAULTS_VALIDITY_PERIOD)) {
      serverDefaults = namenode.getServerDefaults();
      serverDefaultsLastUpdate = now;
    assert serverDefaults != null;
    return serverDefaults;
Let me know if you I missed something.

> Remove indefinite caching of key provider uri in DFSClient
> ----------------------------------------------------------
>                 Key: HDFS-11702
>                 URL: https://issues.apache.org/jira/browse/HDFS-11702
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: hdfs-client
>            Reporter: Rushabh S Shah
>            Assignee: Rushabh S Shah
>         Attachments: HDFS-11702.patch
> There is an indefinite caching of key provider uri in dfsclient.
> Relevant piece of code.
> {code:title=DFSClient.java|borderStyle=solid}
>   /**
>    * The key provider uri is searched in the following order.
>    * 1. If there is a mapping in Credential's secrets map for namenode uri.
>    * 2. From namenode getServerDefaults rpc.
>    * 3. Finally fallback to local conf.
>    * @return keyProviderUri if found from either of above 3 cases,
>    * null otherwise
>    * @throws IOException
>    */
>   URI getKeyProviderUri() throws IOException {
>     if (keyProviderUri != null) {
>       return keyProviderUri;
>     }
>     // Lookup the secret in credentials object for namenodeuri.
>     Credentials credentials = ugi.getCredentials();
>    ...
>    ...
> {code}
> Once the key provider uri is set, it won't refresh the value even if the key provider
uri on namenode is changed.
> For long running clients like on oozie servers, this means we have to bounce all the
oozie servers to get the change reflected.
> After this change, the client will cache the value for an hour after which it will issue
getServerDefaults call and will refresh the key provider uri.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message