accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From keith-turner <...@git.apache.org>
Subject [GitHub] accumulo issue #257: ACCUMULO-4636 system iterator improvements for 1.7
Date Thu, 11 May 2017 23:34:38 GMT
Github user keith-turner commented on the issue:

    https://github.com/apache/accumulo/pull/257
  
    @joshelser I used the following script to run some performance test locally using [Uno](https://github.com/astralway/uno).
 I didn't bother computing stats, but there is definitely an observable performance improvement
(and no degradation).   That's pretty cool to see, because the performance test run earlier
were targeted to just the iterator stack.  This is showing improvements in an end to end test.
    
    ```bash
    #! /usr/bin/env bash
    
    eval "$(./bin/uno env)"
    
    accumulo shell -u root -p secret -e 'createtable test_ingest'
    
    accumulo org.apache.accumulo.test.TestIngest -i uno -u root -p secret --timestamp 1 --size
50 --random 56 --rows 1000000 --start 0 --cols 1 &
    accumulo org.apache.accumulo.test.TestIngest -i uno -u root -p secret --timestamp 1 --size
50 --random 56 --rows 1000000 --start 1000000 --cols 1 &
    accumulo org.apache.accumulo.test.TestIngest -i uno -u root -p secret --timestamp 1 --size
50 --random 56 --rows 1000000 --start 2000000 --cols 1 &
    accumulo org.apache.accumulo.test.TestIngest -i uno -u root -p secret --timestamp 1 --size
50 --random 56 --rows 1000000 --start 3000000 --cols 1 &
    accumulo org.apache.accumulo.test.TestIngest -i uno -u root -p secret --timestamp 1 --size
50 --random 56 --rows 1000000 --start 4000000 --cols 1 &
    
    wait
    
    accumulo shell -u root -p secret -e 'compact -t test_ingest -w'
    
    for run in {1..10}
    do
      accumulo  org.apache.accumulo.test.VerifyIngest -i uno -u root -p secret --size 50 --timestamp
1 --random 56 --rows 5000000 --start 0 --cols 1
    done
    
    ```
    
    Below are the results for 1.7.3
    ```
       5,000,000 records read |  390,502 records/sec |  395,000,000 bytes read | 30,849,734
bytes/sec | 12.804 secs   
       5,000,000 records read |  411,861 records/sec |  395,000,000 bytes read | 32,537,067
bytes/sec | 12.140 secs   
       5,000,000 records read |  424,268 records/sec |  395,000,000 bytes read | 33,517,182
bytes/sec | 11.785 secs   
       5,000,000 records read |  428,338 records/sec |  395,000,000 bytes read | 33,838,773
bytes/sec | 11.673 secs   
       5,000,000 records read |  427,350 records/sec |  395,000,000 bytes read | 33,760,683
bytes/sec | 11.700 secs   
       5,000,000 records read |  428,228 records/sec |  395,000,000 bytes read | 33,830,078
bytes/sec | 11.676 secs   
       5,000,000 records read |  427,606 records/sec |  395,000,000 bytes read | 33,780,894
bytes/sec | 11.693 secs   
       5,000,000 records read |  427,606 records/sec |  395,000,000 bytes read | 33,780,894
bytes/sec | 11.693 secs   
       5,000,000 records read |  431,816 records/sec |  395,000,000 bytes read | 34,113,481
bytes/sec | 11.579 secs   
       5,000,000 records read |  414,250 records/sec |  395,000,000 bytes read | 32,725,766
bytes/sec | 12.070 secs   
    
    ```
    
    Below are the results for this branch
    ```
       5,000,000 records read |  408,563 records/sec |  395,000,000 bytes read | 32,276,515
bytes/sec | 12.238 secs   
       5,000,000 records read |  426,657 records/sec |  395,000,000 bytes read | 33,705,947
bytes/sec | 11.719 secs   
       5,000,000 records read |  447,027 records/sec |  395,000,000 bytes read | 35,315,154
bytes/sec | 11.185 secs   
       5,000,000 records read |  442,830 records/sec |  395,000,000 bytes read | 34,983,615
bytes/sec | 11.291 secs   
       5,000,000 records read |  446,388 records/sec |  395,000,000 bytes read | 35,264,708
bytes/sec | 11.201 secs   
       5,000,000 records read |  445,990 records/sec |  395,000,000 bytes read | 35,233,253
bytes/sec | 11.211 secs   
       5,000,000 records read |  435,957 records/sec |  395,000,000 bytes read | 34,440,666
bytes/sec | 11.469 secs   
       5,000,000 records read |  442,869 records/sec |  395,000,000 bytes read | 34,986,713
bytes/sec | 11.290 secs   
       5,000,000 records read |  439,521 records/sec |  395,000,000 bytes read | 34,722,222
bytes/sec | 11.376 secs   
       5,000,000 records read |  424,592 records/sec |  395,000,000 bytes read | 33,542,798
bytes/sec | 11.776 secs  
    ```


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message