hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jean-Daniel Cryans (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-8450) Update hbase-default.xml and general recommendations to better suit current hw, h2, experience, etc.
Date Mon, 13 May 2013 18:29:16 GMT

    [ https://issues.apache.org/jira/browse/HBASE-8450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13656219#comment-13656219
] 

Jean-Daniel Cryans commented on HBASE-8450:
-------------------------------------------

{quote}
The bigger block size, the lesser namenode rpc call. 
We have a cluster with hbase.regionserver.hlog.blocksize=128m and the write performance is
improved a lot compare with hbase.regionserver.hlog.blocksize=64m, the cell is big and most
is 15K+.
{quote}

I don't think less RPCs to the NN really matters a lot here, other parts of HBase abuse the
NN a lot more, but most likely your write performance improved because you are spending less
time locking when rolling the logs. I pushed this to the extreme and tried 1GB and 2GB blocksizes
and that's when I got the bad performance mostly.

Now that I think about it more, it might make sense to keep bigger logs but keep the block
size itself low. For example, have hbase.regionserver.hlog.blocksize=64MB but hbase.regionserver.logroll.multiplier=3.95
instead of 0.95. Is this something you have tried or considered [~aoxiang]?
                
> Update hbase-default.xml and general recommendations to better suit current hw, h2, experience,
etc.
> ----------------------------------------------------------------------------------------------------
>
>                 Key: HBASE-8450
>                 URL: https://issues.apache.org/jira/browse/HBASE-8450
>             Project: HBase
>          Issue Type: Task
>          Components: Usability
>            Reporter: stack
>            Priority: Critical
>             Fix For: 0.95.1
>
>         Attachments: 8450.txt
>
>
> This is a critical task we need to do before we release; review our defaults.
> On cursory review, there are configs in hbase-default.xml that no longer have matching
code; there are some that should be changed because we know better now and others that should
be amended because hardware and deploys are bigger than they used to be.
> We could also move stuff around so that the must-edit stuff is near the top (zk quorum
config. is mid-way down the page) and beef up the descriptions -- especially since these descriptions
shine through in the refguide.
> Lastly, I notice that our tgz does not "include" an hbase-default.xml other than the
one bundled up in the jar.  Maybe we should make it more accessible.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message