accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Keith Turner (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ACCUMULO-1409) Make MiniAccumuloCluster support native maps
Date Thu, 16 May 2013 20:07:16 GMT

    [ https://issues.apache.org/jira/browse/ACCUMULO-1409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13659906#comment-13659906
] 

Keith Turner commented on ACCUMULO-1409:
----------------------------------------

bq. At what size does the having the native map truly affect performance?

Its used to sort incoming data.  Sorting data w/ C++ is faster than Java.   A difference would
be observed if you started writing to an empty Accumulo instance w/ native maps.   

bq. At what size does the having the native map truly affect performance?

The performance of native maps should be invariant with data size.  However sorting incoming
data is only one part of what Accumulo does.  Accumulo eventually has to do compactions, which
have nothing to do with the native map.  So I suspect you would see the largest performance
increase from native maps when Accumulo is empty.  I am thinking Amdahl's law kicks in as
more data is written, compactions start and now native maps are only one part of the system
thats processing data.  It would be interesting to run experiments w/ and w/o native maps
for 10M, 100M, 1B, and 10B entries.
                
> Make MiniAccumuloCluster support native maps
> --------------------------------------------
>
>                 Key: ACCUMULO-1409
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-1409
>             Project: Accumulo
>          Issue Type: Sub-task
>            Reporter: Keith Turner
>            Assignee: Corey J. Nolet
>              Labels: proposal
>             Fix For: 1.6.0
>
>
> If a user sets up a single node accumulo instance and uses it for testing, then w/o native
maps the performance will not be as good as it could.  I did some testing of the new GC in
java 7, along the way I noticed a significant difference between the native map and java map.
> http://kaccumulo.blogspot.com/2012/11/testing-accumulo-with-g1.html

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message