Return-Path: X-Original-To: apmail-hbase-issues-archive@www.apache.org Delivered-To: apmail-hbase-issues-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 5F998E7CD for ; Sat, 26 Jan 2013 00:13:14 +0000 (UTC) Received: (qmail 25465 invoked by uid 500); 26 Jan 2013 00:13:14 -0000 Delivered-To: apmail-hbase-issues-archive@hbase.apache.org Received: (qmail 25422 invoked by uid 500); 26 Jan 2013 00:13:14 -0000 Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@hbase.apache.org Received: (qmail 25413 invoked by uid 99); 26 Jan 2013 00:13:14 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 26 Jan 2013 00:13:14 +0000 Date: Sat, 26 Jan 2013 00:13:14 +0000 (UTC) From: "Lars Hofhansl (JIRA)" To: issues@hbase.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HBASE-5930) Periodically flush the Memstore? MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HBASE-5930?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13563193#comment-13563193 ] Lars Hofhansl commented on HBASE-5930: -------------------------------------- Absolutely not use SecureRandom here. We're not using this to generate cryptographics keys, but just some jitter for memstore flush timing, right? SecureRandom will exhaust your locally generated entropy that is much better used in case where it is actually needed (and it can hang - on Linux at least - if not enough entropy has been collected) > Periodically flush the Memstore? > -------------------------------- > > Key: HBASE-5930 > URL: https://issues.apache.org/jira/browse/HBASE-5930 > Project: HBase > Issue Type: Improvement > Reporter: Lars Hofhansl > Assignee: Devaraj Das > Priority: Minor > Fix For: 0.96.0 > > Attachments: 5930-1.patch, 5930-2.1.patch, 5930-wip.patch > > > A colleague of mine ran into an interesting issue. > He inserted some data with the WAL disabled, which happened to fit in the aggregate Memstores memory. > Two weeks later he a had problem with the HDFS cluster, which caused the region servers to abort. He found that his data was lost. Looking at the log we found that the Memstores were not flushed at all during these two weeks. > Should we have an option to flush memstores periodically. There are obvious downsides to this, like many small storefiles, etc. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira