hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Lars Hofhansl (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-5930) Periodically flush the Memstore?
Date Thu, 03 May 2012 20:10:48 GMT

    [ https://issues.apache.org/jira/browse/HBASE-5930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13267765#comment-13267765
] 

Lars Hofhansl commented on HBASE-5930:
--------------------------------------

What should trigger the flush is an interesting discussion in itself. Should we flush:
* after N timeunits of write inactivity, or
* when the last flush happened more than N TUs ago

The former would avoid smaller storefiles, the latter would put a limit on how stale an entry
in the memstore can be.

                
> Periodically flush the Memstore?
> --------------------------------
>
>                 Key: HBASE-5930
>                 URL: https://issues.apache.org/jira/browse/HBASE-5930
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Lars Hofhansl
>            Priority: Minor
>
> A colleague of mine ran into an interesting issue.
> He inserted some data with the WAL disabled, which happened to fit in the aggregate Memstores
memory.
> Two weeks later he a had problem with the HDFS cluster, which caused the region servers
to abort. He found that his data was lost. Looking at the log we found that the Memstores
were not flushed at all during these two weeks.
> Should we have an option to flush memstores periodically. There are obvious downsides
to this, like many small storefiles, etc.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message