accumulo-notifications mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Josh Elser (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (ACCUMULO-2749) 1.6.0RC4 slower than 1.5.1
Date Tue, 29 Apr 2014 01:51:18 GMT

    [ https://issues.apache.org/jira/browse/ACCUMULO-2749?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13983869#comment-13983869
] 

Josh Elser commented on ACCUMULO-2749:
--------------------------------------

Shrunk the IMM to 512MB (from 4GB) and saw 1.6.0 ingest at a rate of 98.3% of 1.5.1 (163,180.2
records/sec compared to 165,973.4 records/sec)

Ran the test with DevNullIterator on MinC and saw 1.6.0 ingest at a rate of 90.0% of 1.5.1
(342,894 records/sec compared to 380,676.2 records/sec)

Other system details: Apache Hadoop 2.4.0, ZK 3.4.5. 8core with 32G of RAM. Using a single
spinning disk. I did modify the ingest_test.sh script to be a little easier to run and consume
output from. I'll attach the diff, and here is the little for loop to run it a few times (how
I did devnull):

{code}
for i in 1 2 3 4 5; do
../../../bin/accumulo shell -u root -p secret -e 'deletetable -f -t test_ingest'
../../../bin/accumulo org.apache.accumulo.test.TestIngest --createTable -u root -p secret
--splits 100 --rows 100 | fgrep 'records/sec'
../../../bin/accumulo shell -u root -p secret -e 'config -t test_ingest -s table.walog.enabled=false'
../../../bin/accumulo shell -u root -p secret -e 'config -t test_ingest -s table.iterator.minc.devnull=10,org.apache.accumulo.core.iterators.DevNull'
sleep 15
./ingest_test.sh
sleep 15
done
{code}

> 1.6.0RC4 slower than 1.5.1
> --------------------------
>
>                 Key: ACCUMULO-2749
>                 URL: https://issues.apache.org/jira/browse/ACCUMULO-2749
>             Project: Accumulo
>          Issue Type: Bug
>    Affects Versions: 1.6.0
>            Reporter: Keith Turner
>             Fix For: 1.6.1
>
>
> I am seeing consistently slower write rates when comparing 1.5.1 and 1.6.0 RC4.   Following
env :
>    * hadoop 2.2.0
>    * Zookeeper 3.4.5
>    * Centos 6 (native, not a VM)
>    * using examples/3GB/native-standalone config
>    * setting tserver.mutation.queue.max 4M
>    * I am running test/system/test1/ingest_test.sh
>    * single node
> Saw the following rates
> {noformat}
>   1.5.1 aggregate rate : 200,806 records/sec
>   1.6.0 aggregate rate : 181,264 records/sec (90%)
> {noformat}
> Reran test setting  table.walog.enabled=false
> {noformat}
>   1.5.1 aggregate rate : 302,307 records/sec
>   1.6.0 aggregate rate : 232,685 records/sec (77%)
> {noformat}
> Since the relative performance gap seemed to widen when disabling walog I tried another
test where I disabled walogs and set the devnull iterator 
> {noformat}
>   1.5.1 aggregate rate : 554,853 records/sec
>   1.6.0 aggregate rate : 373,556 records/sec (67%)
> {noformat}
> The relative performance gap grew even further.
> Used the following command to enable dev null iterator in 1.5.1 and RC4
> {noformat}
> config -t test_ingest -s table.iterator.minc.devnull=10,org.apache.accumulo.core.iterators.DevNull
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message