accumulo-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From keith-turner <...@git.apache.org>
Subject [GitHub] accumulo pull request: ACCUMULO-4187: Added rate limiting for majo...
Date Wed, 13 Apr 2016 21:08:14 GMT
Github user keith-turner commented on the pull request:

    https://github.com/apache/accumulo/pull/90#issuecomment-209647745
  
    I played around with this branch locally.  I created a table with 10,000,000 entries using
test_ingest using the following commands. 
    
    ```
    ./bin/accumulo shell -u root -p secret -e "createtable test_ingest"
    ./bin/accumulo org.apache.accumulo.test.TestIngest -u root -p secret --timestamp 1 --size
50 --random 56 --rows 10000000 --start 0 --cols 1 --instance instance16
    ```
    I set the rate limit to 5M and forced a compaction.  I saw the following in the tserver
logs.
    
    ```
    Compaction 2<< 10,000,000 read | 10,000,000 written | 122,925 entries/sec | 81.350
secs |  431,758,096 bytes | 5307413.596 byte/sec
    ```
    
    Then I split the table into 8 tablets and forced a compaction to test the rate limit for
multiple threads.  I had the default of 3 compaction threads.  I saw the following in the
logs for this test.
    
    ```
    Compaction 2;row_0003749;row_00025 1,249,000 read | 1,249,000 written | 41,866 entries/sec
| 29.833 secs |   53,926,291 bytes | 1807605.370 byte/sec
    Compaction 2;row_00025;row_000125 1,250,000 read | 1,250,000 written | 41,899 entries/sec
| 29.833 secs |   53,970,229 bytes | 1809078.168 byte/sec
    Compaction 2;row_000125< 1,250,000 read | 1,250,000 written | 41,783 entries/sec |
29.916 secs |   53,969,343 bytes | 1804029.382 byte/sec
    Compaction 2;row_000625;row_0005 1,250,000 read | 1,250,000 written | 42,134 entries/sec
| 29.667 secs |   53,970,847 bytes | 1819221.593 byte/sec
    Compaction 2;row_0005;row_0003749 1,251,000 read | 1,251,000 written | 42,109 entries/sec
| 29.708 secs |   54,012,874 bytes | 1818125.555 byte/sec
    Compaction 2;row_00075;row_000625 1,250,000 read | 1,250,000 written | 41,881 entries/sec
| 29.846 secs |   53,969,549 bytes | 1808267.406 byte/sec
    Compaction 2;row_000875;row_00075 1,250,000 read | 1,250,000 written | 63,909 entries/sec
| 19.559 secs |   53,969,511 bytes | 2759318.523 byte/sec
    Compaction 2<;row_000875 1,250,000 read | 1,250,000 written | 63,798 entries/sec |
19.593 secs |   53,969,987 bytes | 2754554.535 byte/sec
    ```
    
    



---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message