hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Devaraj Das (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-5930) Periodically flush the Memstore?
Date Sat, 26 Jan 2013 01:08:12 GMT

    [ https://issues.apache.org/jira/browse/HBASE-5930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13563261#comment-13563261
] 

Devaraj Das commented on HBASE-5930:
------------------------------------

bq. Should upper bound for the sleep take length of MemStoreFlusher.flushQueue into consideration
?

[~yuzhihong@gmail.com], I think we don't have to worry about this one as much. The reason
being that there is a random delay before each flush is inserted in the queue (as opposed
to inserts coming in at a rate faster than what the flusher can handle).

bq. Also, the sleep should be bounded by the remaining time w.r.t. cacheFlushInterval - we
don't want the loop in chore() to outlast cacheFlushInterval.

This should be fine. I don't see issues with this one.
                
> Periodically flush the Memstore?
> --------------------------------
>
>                 Key: HBASE-5930
>                 URL: https://issues.apache.org/jira/browse/HBASE-5930
>             Project: HBase
>          Issue Type: Improvement
>            Reporter: Lars Hofhansl
>            Assignee: Devaraj Das
>            Priority: Minor
>             Fix For: 0.96.0
>
>         Attachments: 5930-1.patch, 5930-2.1.patch, 5930-wip.patch
>
>
> A colleague of mine ran into an interesting issue.
> He inserted some data with the WAL disabled, which happened to fit in the aggregate Memstores
memory.
> Two weeks later he a had problem with the HDFS cluster, which caused the region servers
to abort. He found that his data was lost. Looking at the log we found that the Memstores
were not flushed at all during these two weeks.
> Should we have an option to flush memstores periodically. There are obvious downsides
to this, like many small storefiles, etc.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message