Return-Path: X-Original-To: apmail-hbase-issues-archive@www.apache.org Delivered-To: apmail-hbase-issues-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id EF994178B9 for ; Wed, 3 Jun 2015 00:13:50 +0000 (UTC) Received: (qmail 1668 invoked by uid 500); 2 Jun 2015 23:13:50 -0000 Delivered-To: apmail-hbase-issues-archive@hbase.apache.org Received: (qmail 1619 invoked by uid 500); 2 Jun 2015 23:13:50 -0000 Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list issues@hbase.apache.org Received: (qmail 1608 invoked by uid 99); 2 Jun 2015 23:13:50 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 02 Jun 2015 23:13:50 +0000 Date: Tue, 2 Jun 2015 23:13:50 +0000 (UTC) From: "stack (JIRA)" To: issues@hbase.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (HBASE-13811) Splitting WALs, we are filtering out too many edits -> DATALOSS MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HBASE-13811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] stack updated HBASE-13811: -------------------------- Attachment: 13811.txt Mostly logging changes so we output less but with more density including detail like sequence id at critical junctures so it easier debugging these issues going forward. Patch includes a change log. Fix is in FSHLog getEarliestMemstoreSeqNum methods; look in the Map of currently flushing sequence ids first and then if none found here, look in the oldest sequence id map. Trying this patch against hadoopqa to see if I've broke anything. Trying on a cluster. Need to add a test for this particular case still. > Splitting WALs, we are filtering out too many edits -> DATALOSS > --------------------------------------------------------------- > > Key: HBASE-13811 > URL: https://issues.apache.org/jira/browse/HBASE-13811 > Project: HBase > Issue Type: Bug > Components: wal > Reporter: stack > Priority: Critical > Attachments: 13811.txt > > > I've been running ITBLLs against branch-1 around HBASE-13616 (move of ServerShutdownHandler to pv2). I have come across an instance of dataloss. My patch for HBASE-13616 was in place so can only think it the cause (but cannot see how). When we split the logs, we are skipping legit edits. Digging. -- This message was sent by Atlassian JIRA (v6.3.4#6332)