hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Larry McCay (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-10829) Iteration on CredentialProviderFactory.serviceLoader is thread-unsafe
Date Thu, 27 Oct 2016 11:26:58 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-10829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15611587#comment-15611587
] 

Larry McCay commented on HADOOP-10829:
--------------------------------------

[~rakeshr] - I am inclined to prefer the approach used in KeyProviderFactory instead which
avoids the lazy loading by iterating over the providers in a static initialization block.
This avoids the need for synchronization during each call to getProviders for every client
in a JVM.

{code}
  public abstract KeyProvider createProvider(URI providerName,
                                             Configuration conf
                                             ) throws IOException;

  private static final ServiceLoader<KeyProviderFactory> serviceLoader =
      ServiceLoader.load(KeyProviderFactory.class,
          KeyProviderFactory.class.getClassLoader());

  // Iterate through the serviceLoader to avoid lazy loading.
  // Lazy loading would require synchronization in concurrent use cases.
  static {
    Iterator<KeyProviderFactory> iterServices = serviceLoader.iterator();
    while (iterServices.hasNext()) {
      iterServices.next();
    }
  }
  
  public static List<KeyProvider> getProviders(Configuration conf
                                               ) throws IOException {
    List<KeyProvider> result = new ArrayList<KeyProvider>();
    for(String path: conf.getStringCollection(KEY_PROVIDER_PATH)) {
      try {
        URI uri = new URI(path);
        KeyProvider kp = get(uri, conf);
        if (kp != null) {
          result.add(kp);
        } else {
          throw new IOException("No KeyProviderFactory for " + uri + " in " +
              KEY_PROVIDER_PATH);
        }
      } catch (URISyntaxException error) {
        throw new IOException("Bad configuration of " + KEY_PROVIDER_PATH +
            " at " + path, error);
      }
    }
    return result;
  }
{code}

What do you think?

> Iteration on CredentialProviderFactory.serviceLoader  is thread-unsafe
> ----------------------------------------------------------------------
>
>                 Key: HADOOP-10829
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10829
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: security
>    Affects Versions: 2.6.0
>            Reporter: Benoy Antony
>            Assignee: Benoy Antony
>              Labels: BB2015-05-TBR
>         Attachments: HADOOP-10829.003.patch, HADOOP-10829.patch, HADOOP-10829.patch
>
>
> CredentialProviderFactory uses _ServiceLoader_ framework to load _CredentialProviderFactory_
> {code}
>   private static final ServiceLoader<CredentialProviderFactory> serviceLoader =
>       ServiceLoader.load(CredentialProviderFactory.class);
> {code}
> The _ServiceLoader_ framework does lazy initialization of services which makes it thread
unsafe. If accessed from multiple threads, it is better to synchronize the access.
> Similar synchronization has been done while loading compression codec providers via HADOOP-8406.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message