hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rushabh S Shah (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-11702) Remove indefinite caching of key provider uri in DFSClient
Date Tue, 25 Apr 2017 22:10:04 GMT
Rushabh S Shah created HDFS-11702:

             Summary: Remove indefinite caching of key provider uri in DFSClient
                 Key: HDFS-11702
                 URL: https://issues.apache.org/jira/browse/HDFS-11702
             Project: Hadoop HDFS
          Issue Type: Bug
          Components: hdfs-client
            Reporter: Rushabh S Shah
            Assignee: Rushabh S Shah

There is an indefinite caching of key provider uri in dfsclient.
Relevant piece of code.
   * The key provider uri is searched in the following order.
   * 1. If there is a mapping in Credential's secrets map for namenode uri.
   * 2. From namenode getServerDefaults rpc.
   * 3. Finally fallback to local conf.
   * @return keyProviderUri if found from either of above 3 cases,
   * null otherwise
   * @throws IOException
  URI getKeyProviderUri() throws IOException {
    if (keyProviderUri != null) {
      return keyProviderUri;

Once the key provider uri is set, it won't refresh the value even if the key provider uri
on namenode is changed.
For long running clients like on oozie servers, this means we have to bounce all the oozie
servers to get the change reflected.
After this change, the client will cache the value for an hour after which it will issue getServerDefaults
call and will refresh the key provider uri.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

View raw message