accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Elser (JIRA)" <>
Subject [jira] [Commented] (ACCUMULO-3033) SiteConfiguration gets invoked frequently and scans all configuration items
Date Thu, 31 Jul 2014 15:42:38 GMT


Josh Elser commented on ACCUMULO-3033:

* We could push down the logic of efficient filtering to the PropertyFilter implementation,
but that would be difficult to implement in a single location as it would have to know how
to handle the variety of sources in which we get our Property's from (ZooKeeper, accumulo-site.xml,
Property defaults).
* Add some sort of LRU cache to the immutable AccumuloConfigurations (SiteConfiguration and
DefaultConfiguration) that would be able to cache results for a given PropertyFilter implementation

I think the latter is the better, but would be open to other ideas people have.

> SiteConfiguration gets invoked frequently and scans all configuration items
> ---------------------------------------------------------------------------
>                 Key: ACCUMULO-3033
>                 URL:
>             Project: Accumulo
>          Issue Type: Improvement
>    Affects Versions: 1.5.0, 1.5.1, 1.6.0
>            Reporter: Josh Elser
>            Assignee: Josh Elser
>             Fix For: 1.5.2, 1.6.1, 1.7.0
> I noticed yesterday that {{SiteConfiguration#getProperties(Map, PropertyFilter)}} is
invoked very frequently (at least 30 times a second on my single machine) to extract the Iterator
configurations before a table scan happens. Printing a stack trace shows that this is the
dominating caller of {{getProperties}} is {{IteratorUtil#parseIterConf()}}.
> What this translates to is repeatedly reading, filtering and adding the same elements
to a map to satisfy the {{AccumuloConfiguration#getPropertiesWithPrefix}} from an immutable
source (accumulo-site.xml). While this is probably not a huge time waste in terms of actual
time spent, it's repeatedly happening and will slow things down (especially when the underlying
Hadoop Configuration is creating a new HashMap with the contents of the accumulo-site.xml
for every call).
> Need to think about some options for making the "extract keys with this prefix" case
faster from SiteConfiguration.

This message was sent by Atlassian JIRA

View raw message