hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jean-Daniel Cryans (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HBASE-2087) The wait on compaction because "Too many store files" holds up all flushing
Date Wed, 31 Mar 2010 23:56:27 GMT

     [ https://issues.apache.org/jira/browse/HBASE-2087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Jean-Daniel Cryans updated HBASE-2087:

    Attachment: HBASE-2087.patch

Here's a patch that does what Stack describes. 

 - If the flush comes from flushSomeRegions, we will wait since that doesn't hold up the other
 - If the flush comes from the main flushing thread, we check if there's too many store files.
If so, we wait a bit and add it back to the queue.

I tried it on the randomWrite PE, works as advertised but it may be a bit chatty when the
compactions are taking a long time. Could be improved by doing the "triggered" thing ensureStoreFileCount
is doing.

> The wait on compaction because "Too many store files" holds up all flushing
> ---------------------------------------------------------------------------
>                 Key: HBASE-2087
>                 URL: https://issues.apache.org/jira/browse/HBASE-2087
>             Project: Hadoop HBase
>          Issue Type: Bug
>            Reporter: stack
>             Fix For: 0.20.4
>         Attachments: HBASE-2087.patch
> The method MemStoreFlusher#checkStoreFileCount is called from flushRegion.  flushRegion
is called by MemStoreFlusher#run thread.  If the checkStoreFileCount finds too many store
files, it'll stick around waiting on a compaction to happen.  While its hanging, the MemStoreFlusher#run
is held up.  No other region can flush.  Meantime WALs will be rolling and memory will be
accumulating writes.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message