accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean Busbey (JIRA)" <>
Subject [jira] [Commented] (ACCUMULO-3890) Use of CredentialProvider results in a lot of NN ops
Date Fri, 12 Jun 2015 05:08:01 GMT


Sean Busbey commented on ACCUMULO-3890:

Well, I was trying to avoid synchronizing every read, since we were discussing that this could
result in worse performance. It seems like if the initial read doesn't find the object, it
will load new providers, and then do a synchronized read or put, throwing away the new providers
if it can read them. Do you think that's not a good approach?

For future reference, the approach would be fine but HashMap isn't threadsafe for concurrent
reads and writes. So even if you are synchronizing all write access, that doesn't allow safe
concurrent reading. The switch to ConcurrentHashMap obviates this concern, natch.

+      providersList = (List<Object>) providersObj;
+      List<Object> previousValue = cachedProviders.putIfAbsent(path, providersList);

nit:ConcurrentHashMap doesn't allow null values (HashMap did). The current implementation
of {{CredentialProviderFactory.getProviders}} (in 2.6.0 and 2.7.0) can't return null, but
there's nothing in its javadoc to say that it won't in the future. In the event that it does,
we'll propagate a NullPointerException. not sure if it's worth an addendum to properly null
check (or a Hadoop jira to javadoc that it can't return null?).

> Use of CredentialProvider results in a lot of NN ops
> ----------------------------------------------------
>                 Key: ACCUMULO-3890
>                 URL:
>             Project: Accumulo
>          Issue Type: Bug
>    Affects Versions: 1.6.1, 1.6.2, 1.7.0
>            Reporter: Billie Rinaldi
>            Assignee: Billie Rinaldi
>             Fix For: 1.6.3, 1.7.1, 1.8.0
>         Attachments: ACCUMULO-3890.1.patch, ACCUMULO-3890.2.patch, ACCUMULO-3890.3.patch,
ACCUMULO-3890.4.patch, ACCUMULO-3890.5.part2.patch, ACCUMULO-3890.5.patch, ACCUMULO-3890.6.patch,
>          Time Spent: 1h
>  Remaining Estimate: 0h
> Every time we access a sensitive property or iterate over a configuration when there
is a CredentialProvider configured, it results in NN operations (as evidenced by FSNamesystem.audit
logs).  I think that we could assume the CredentialProvider is static, read its properties
once and cache them in memory to avoid these unnecessary reads.

This message was sent by Atlassian JIRA

View raw message