accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Adam Fuchs (JIRA)" <j...@apache.org>
Subject [jira] [Created] (ACCUMULO-1753) evaluate options for optimizing a SortedKeyValueIterator-like API on the client side
Date Mon, 07 Oct 2013 14:45:42 GMT
Adam Fuchs created ACCUMULO-1753:
------------------------------------

             Summary: evaluate options for optimizing a SortedKeyValueIterator-like API on
the client side
                 Key: ACCUMULO-1753
                 URL: https://issues.apache.org/jira/browse/ACCUMULO-1753
             Project: Accumulo
          Issue Type: New Feature
            Reporter: Adam Fuchs


Currently, there is a significant performance difference between running something like the
IntersectingIterator in the TabletServer vs. in the ClientSideIteratorScanner, even on a single
node with the client running alongside the TabletServer. There are many reasons for this difference,
including:
1. Batching between the TabletServer and Scanner makes frequent seeks inefficient.
2. The wire protocol used by the Scanner makes seeks inefficient.
3. Interprocess communication in general adds latency.
4. Encoding and decoding adds latency.

Server-side iterators are still going to be better even if we completely optimize client-side
iterators. However, server-side iterators come with risks to stability and security, especially
if the set of iterators grows quickly. If we could optimize some of these problems, we could
theoretically enable more programmability of complex operations with less risk to security
and reliability of a multi-tenant instance of Accumulo.

This ticket is related to the concept of running iterators in a separate process, but includes
the RPC aspect as well.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message