hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jean-Daniel Cryans (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-8450) Update hbase-default.xml and general recommendations to better suit current hw, h2, experience, etc.
Date Mon, 13 May 2013 18:29:16 GMT

    [ https://issues.apache.org/jira/browse/HBASE-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13656219#comment-13656219

Jean-Daniel Cryans commented on HBASE-8450:

The bigger block size, the lesser namenode rpc call. 
We have a cluster with hbase.regionserver.hlog.blocksize=128m and the write performance is
improved a lot compare with hbase.regionserver.hlog.blocksize=64m, the cell is big and most
is 15K+.

I don't think less RPCs to the NN really matters a lot here, other parts of HBase abuse the
NN a lot more, but most likely your write performance improved because you are spending less
time locking when rolling the logs. I pushed this to the extreme and tried 1GB and 2GB blocksizes
and that's when I got the bad performance mostly.

Now that I think about it more, it might make sense to keep bigger logs but keep the block
size itself low. For example, have hbase.regionserver.hlog.blocksize=64MB but hbase.regionserver.logroll.multiplier=3.95
instead of 0.95. Is this something you have tried or considered [~aoxiang]?
> Update hbase-default.xml and general recommendations to better suit current hw, h2, experience,
> ----------------------------------------------------------------------------------------------------
>                 Key: HBASE-8450
>                 URL: https://issues.apache.org/jira/browse/HBASE-8450
>             Project: HBase
>          Issue Type: Task
>          Components: Usability
>            Reporter: stack
>            Priority: Critical
>             Fix For: 0.95.1
>         Attachments: 8450.txt
> This is a critical task we need to do before we release; review our defaults.
> On cursory review, there are configs in hbase-default.xml that no longer have matching
code; there are some that should be changed because we know better now and others that should
be amended because hardware and deploys are bigger than they used to be.
> We could also move stuff around so that the must-edit stuff is near the top (zk quorum
config. is mid-way down the page) and beef up the descriptions -- especially since these descriptions
shine through in the refguide.
> Lastly, I notice that our tgz does not "include" an hbase-default.xml other than the
one bundled up in the jar.  Maybe we should make it more accessible.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message