hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-10201) Port 'Make flush decisions per column family' to trunk
Date Sat, 06 Dec 2014 03:55:13 GMT

     [ https://issues.apache.org/jira/browse/HBASE-10201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

stack updated HBASE-10201:
--------------------------
    Attachment: compactions.png
                io.png
                count.png
                memstore.png

Ran some loadings.  Small cluster with one regionserver hosting one region.  Used the test
packaged in this patch modifying it so could run ten clients in parallel rather than a single
client.  The included test has a table schema of three column families and it fills them unevenly
so it is 'ideal' for demonstrating benefit.  I ran with patch turned off twice and then turned
on twice.  Set flushes at 64M.

I see less compactions and less hfiles (so less i/o), memstores carrying more (its hard to
see but you should be able to make out memstore sizes do not go to zero or near zero when
the patch is enabled)

Looks good.  Let me review again to recheck sequenceid accounting and run some MTTR tests.

> Port 'Make flush decisions per column family' to trunk
> ------------------------------------------------------
>
>                 Key: HBASE-10201
>                 URL: https://issues.apache.org/jira/browse/HBASE-10201
>             Project: HBase
>          Issue Type: Improvement
>          Components: wal
>            Reporter: Ted Yu
>            Assignee: zhangduo
>            Priority: Critical
>             Fix For: 1.0.0, 2.0.0, 0.98.9
>
>         Attachments: 3149-trunk-v1.txt, HBASE-10201-0.98.patch, HBASE-10201-0.98_1.patch,
HBASE-10201-0.98_2.patch, HBASE-10201-0.99.patch, HBASE-10201.patch, HBASE-10201_1.patch,
HBASE-10201_10.patch, HBASE-10201_11.patch, HBASE-10201_12.patch, HBASE-10201_13.patch, HBASE-10201_13.patch,
HBASE-10201_2.patch, HBASE-10201_3.patch, HBASE-10201_4.patch, HBASE-10201_5.patch, HBASE-10201_6.patch,
HBASE-10201_7.patch, HBASE-10201_8.patch, HBASE-10201_9.patch, compactions.png, count.png,
io.png, memstore.png
>
>
> Currently the flush decision is made using the aggregate size of all column families.
When large and small column families co-exist, this causes many small flushes of the smaller
CF. We need to make per-CF flush decisions.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message