Mailing-List: contact hbase-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hbase-dev@hadoop.apache.org
Message-ID: <429873978.14591262580664094.JavaMail.jira@brutus.apache.org>
Date: Mon, 4 Jan 2010 04:51:04 +0000 (UTC)
From: "Jean-Daniel Cryans (JIRA)" <jira@apache.org>
To: hbase-dev@hadoop.apache.org
Subject: [jira] Commented: (HBASE-2087) The wait on compaction because "Too
 many store files" holds up all flushing
In-Reply-To: <2069566320.1262305709398.JavaMail.jira@brutus.apache.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HBASE-2087?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12796078#action_12796078 ] 

Jean-Daniel Cryans commented on HBASE-2087:
-------------------------------------------

bq. And this issue is about something else anyway, never holding up flushes

As I said in my first comment, it's either too much WALs or too much store files. If we let all flushes go then we are overrun by store files. If we force flush memstores to be able to roll WALs then we easily create too much store files. We have seen stores that needed to compact 100 files and this is why we have a limit.

So, I question the feasibility of this jira.

In the particular case of WALs waiting on flushes waiting on too many store files, what I said is that it's by setting a very low number of WALs that we easily hit the limit. Setting it to a higher number means less chance of hitting this jira's problem, hence making it invalid?

> The wait on compaction because "Too many store files" holds up all flushing
> ---------------------------------------------------------------------------
>
>                 Key: HBASE-2087
>                 URL: https://issues.apache.org/jira/browse/HBASE-2087
>             Project: Hadoop HBase
>          Issue Type: Bug
>            Reporter: stack
>
> The method MemStoreFlusher#checkStoreFileCount is called from flushRegion.  flushRegion is called by MemStoreFlusher#run thread.  If the checkStoreFileCount finds too many store files, it'll stick around waiting on a compaction to happen.  While its hanging, the MemStoreFlusher#run is held up.  No other region can flush.  Meantime WALs will be rolling and memory will be accumulating writes.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.