hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nick Dimiduk (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-9806) Add PerfEval tool for BlockCache
Date Sun, 03 Nov 2013 04:03:19 GMT

     [ https://issues.apache.org/jira/browse/HBASE-9806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Nick Dimiduk updated HBASE-9806:
--------------------------------

    Attachment: test1_run2_20g.pdf
                test1_run2_3g.pdf
                test1_run1_20g.pdf
                test1_run1_3g.pdf
                conf_20g.patch
                conf_3g.patch

Sorry for the delays, [~jmspaggi] -- Strata and all that. I wanted to add a kind of self-tuning
to the patch, give it the ability to write records until it fills up the cache and no more.
No luck as of yet, which means determining how many rows to write requires a little guess-work
and checking the logs.

I've run the test for 3g (-r 1700000) and 20g (-r 12000000) heaps using the attached configs
and 100 iterations. Attached also are the charts I generated from the logs. I tried 1000 iterations
to see if anything happens over a longer interval but nothing exciting. I'm also updating
my log-parsing script to overly the GC events. More to follow.

On your fancy rig, can you run for maybe 8g, 16g, 20g, and 28g? You'll have to apply one of
these conf patches and then adjust the heap size yourself. You'll also need to play with the
-r param to work out how many rows you can fit into the cache w.o eviction (please let me
know what you end up using!) -i 100 should be a good starting point, but if you have time
I'd take logs from 500 or 1000.

Thank you much!

> Add PerfEval tool for BlockCache
> --------------------------------
>
>                 Key: HBASE-9806
>                 URL: https://issues.apache.org/jira/browse/HBASE-9806
>             Project: HBase
>          Issue Type: Test
>          Components: Performance, test
>            Reporter: Nick Dimiduk
>            Assignee: Nick Dimiduk
>         Attachments: HBASE-9806.00.patch, HBASE-9806.01.patch, conf_20g.patch, conf_3g.patch,
test1_run1_20g.pdf, test1_run1_3g.pdf, test1_run2_20g.pdf, test1_run2_3g.pdf
>
>
> We have at least three different block caching layers with myriad configuration settings.
Let's add a tool for evaluating memory allocations and configuration combinations with different
access patterns.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message