hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Harsh J (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-6497) Revisit HLog sizing and roll parameters
Date Thu, 02 Aug 2012 12:10:03 GMT

    [ https://issues.apache.org/jira/browse/HBASE-6497?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13427260#comment-13427260
] 

Harsh J commented on HBASE-6497:
--------------------------------

bq. Should we increase the maxlogs number (default is 32)?

We should decrease the maxlogs number for recovery to be faster, right? Increasing help prevents
any premature form of flushes on regions caused by the log roller during heavy writes, but
impacts recovery time as all HLogs will then get unnecessarily processed.

In order to keep the recovery times same as today, we can switch the numbers this way:

If current is 128 MB x 32 = 4096 MB (4 GB) of logs approx. before full flush, then lets change
that to have fewer than 32 files (reduces NN RPCs during recovery and increases the sequential
read length) on to 8 maxlogs at 512 MB default size (8x512 = 4096 again). Or we could set
a target of 8 GB and work out from that?
                
> Revisit HLog sizing and roll parameters
> ---------------------------------------
>
>                 Key: HBASE-6497
>                 URL: https://issues.apache.org/jira/browse/HBASE-6497
>             Project: HBase
>          Issue Type: Improvement
>          Components: regionserver
>            Reporter: Lars George
>
> The last major update to the HLog sizing and roll features were done in HBASE-1394. I
am proposing to revisit these settings to overcome recent issues where the HLog becomes a
major bottleneck.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message