hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-14540) Write Ahead Log Batching Optimization
Date Fri, 30 Oct 2015 21:00:28 GMT

     [ https://issues.apache.org/jira/browse/HBASE-14540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

stack updated HBASE-14540:
    Attachment: writes.png

I ran a test. Empty table. YCSB all writes going to a single RegionServer sitting on a HDFS
of 5 nodes.  25 clients.  As is, the new strategy slows writes pretty radically (the strategy
is on place for the 10% of the graph that is on the far right side). Let me try but upping
the clients a bunch... but would be good if the strategy was a bit smarter and didn't wait
when loading was relatively low.

> Write Ahead Log Batching Optimization
> -------------------------------------
>                 Key: HBASE-14540
>                 URL: https://issues.apache.org/jira/browse/HBASE-14540
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: John Leach
>            Assignee: John Leach
>         Attachments: 14540.txt, HBaseWALBlockingWaitStrategy.java, writes.png
> The new write ahead log mechanism seems to batch too few mutations when running inside
the disruptor.  As we scaled our load up (many threads with small writes), we saw the number
of hdfs sync operations grow in concert with the number of writes.  Generally, one would expect
the size of the batches to grow but the number of actual sync operations to settle. 

This message was sent by Atlassian JIRA

View raw message