Return-Path: X-Original-To: apmail-hbase-issues-archive@www.apache.org Delivered-To: apmail-hbase-issues-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 183491837D for ; Wed, 10 Jun 2015 05:34:04 +0000 (UTC) Received: (qmail 29946 invoked by uid 500); 10 Jun 2015 05:34:03 -0000 Delivered-To: apmail-hbase-issues-archive@hbase.apache.org Received: (qmail 29879 invoked by uid 500); 10 Jun 2015 05:34:03 -0000 Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@hbase.apache.org Received: (qmail 29866 invoked by uid 99); 10 Jun 2015 05:34:03 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 10 Jun 2015 05:34:03 +0000 Date: Wed, 10 Jun 2015 05:34:03 +0000 (UTC) From: "Enis Soztutar (JIRA)" To: issues@hbase.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HBASE-13877) Interrupt to flush from TableFlushProcedure causes dataloss in ITBLL MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HBASE-13877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14580046#comment-14580046 ] Enis Soztutar commented on HBASE-13877: --------------------------------------- bq. He added nice tests over in HBASE-13811 so we don't regress. Any chance of our adding one here given it is torturous to figure We can cook up something I guess. Not sure how to test the condition that HRegion.flush() should not be aborted, or it if throws DroppedSnapshotException will be properly handled. Do you have anything in mind? bq. though as said above, the "Enis the Detective" novel above was a good read Hehe. That is the next title of my book. > Interrupt to flush from TableFlushProcedure causes dataloss in ITBLL > -------------------------------------------------------------------- > > Key: HBASE-13877 > URL: https://issues.apache.org/jira/browse/HBASE-13877 > Project: HBase > Issue Type: Bug > Reporter: Enis Soztutar > Assignee: Enis Soztutar > Priority: Blocker > Fix For: 2.0.0, 1.2.0, 1.1.1 > > Attachments: hbase-13877_v1.patch, hbase-13877_v2-branch-1.1.patch > > > ITBLL with 1.25B rows failed for me (and Stack as reported in https://issues.apache.org/jira/browse/HBASE-13811?focusedCommentId=14577834&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14577834) > HBASE-13811 and HBASE-13853 fixed an issue with WAL edit filtering. > The root cause this time seems to be different. It is due to procedure based flush interrupting the flush request in case the procedure is cancelled from an exception elsewhere. This leaves the memstore snapshot intact without aborting the server. The next flush, then flushes the previous memstore with the current seqId (as opposed to seqId from the memstore snapshot). This creates an hfile with larger seqId than what its contents are. Previous behavior in 0.98 and 1.0 (I believe) is that after flush prepare and interruption / exception will cause RS abort. -- This message was sent by Atlassian JIRA (v6.3.4#6332)