hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vladimir Rodionov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-14951) Make hbase.regionserver.maxlogs obsolete
Date Tue, 22 Dec 2015 19:33:46 GMT

    [ https://issues.apache.org/jira/browse/HBASE-14951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15068616#comment-15068616
] 

Vladimir Rodionov commented on HBASE-14951:
-------------------------------------------

[~saint.ack@gmail.com]
{quote}
Can you explain the *2 in the release note Vladimir Rodionov *2 seems a bit arbitrary no?
And if it is to stop blocking because too many WALs, is that going to happen now with these
new limits?
{quote}

Yes, 2 is arbitrary. We need to make sure that we avoid fully or minimize events when RS has
to flush memstores prematurely only because it reached artificial limit of hbase.regionserver.maxlogs,
this is why I put this  multiplier into equation, this gives us maximum WAL capacity of 2
x RS memstore-size. As for the second question,  we won't have runaway memstore files in any
case, LogRoller will be archiving old WALs periodically (hbase.regionserver.logroll.period
= 1 h, by default). User has three options, either override default  *hbase.regionserver.maxlogs*
or override default  *hbase.regionserver.logroll.period*, or both to control runaway WALs.
I will update Release notes.

> Make hbase.regionserver.maxlogs obsolete
> ----------------------------------------
>
>                 Key: HBASE-14951
>                 URL: https://issues.apache.org/jira/browse/HBASE-14951
>             Project: HBase
>          Issue Type: Improvement
>          Components: Performance, wal
>            Reporter: Vladimir Rodionov
>            Assignee: Vladimir Rodionov
>            Priority: Minor
>             Fix For: 2.0.0, 1.2.0, 1.3.0
>
>         Attachments: HBASE-14951-v1.patch, HBASE-14951-v2.patch
>
>
> There was a discussion in HBASE-14388 related to maximum number of log files. It was
an agreement that we should calculate this number in a code but still need to honor user's
setting. 
> Maximum number of log files now is calculated as following:
>  maxLogs = HEAP_SIZE * memstoreRatio * 2/ LogRollSize



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message