accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Elser (JIRA)" <>
Subject [jira] [Commented] (ACCUMULO-3079) improve system iterator performance by collapsing call stack
Date Tue, 02 Sep 2014 19:59:21 GMT


Josh Elser commented on ACCUMULO-3079:

bq. Do you see any reason why we would need synchronization on the compaction scope?

It's possible that a user configured their own crazy com.multi.threaded.MyIterator at compaction
time which *could* need synchronization, but that's also a reach given that the use-case for
multi-threading is a stretch anyways.

bq. another approach to solving this problem would be to use the java SecurityManager to prevent
iterators from using multiple threads.

That's certainly stronger than my first thought of documenting somewhere "don't start threads
in an iterator" :). Hopefully if a user tried to do this on their own, they'd realize that
they might be doing something weird when they don't have the hooks to ensure 100% that any
threads started also terminate.

> improve system iterator performance by collapsing call stack
> ------------------------------------------------------------
>                 Key: ACCUMULO-3079
>                 URL:
>             Project: Accumulo
>          Issue Type: Improvement
>            Reporter: Adam Fuchs
>            Assignee: Adam Fuchs
>             Fix For: 1.6.1, 1.7.0
>         Attachments: iterator_performance_20140822_1.patch, iterator_performance_test_harness.tar.gz
> System iterators are at the core of the tightest loops in Accumulo, handling every key/value
pair that traverses through a scan or a compaction. In many cases, iterators are the current
performance bottleneck for Accumulo. Every bit that we can improve performance in the iterators
translates into better performance for Accumulo.
> There are several strategies that can be applied to the current code base to improve
performance, including:
>  # Inlining calls that are hard for the JVM to inline at runtime
>  # Moving checks for null outside of tight loops when they are invariants within the
>  # Eliminating "no-op" iterators at iterator tree construction time
>  # Making frequently used and assigned-once objects final (like iterator sources)

This message was sent by Atlassian JIRA

View raw message