From dev-return-1528-apmail-tephra-dev-archive=tephra.apache.org@tephra.incubator.apache.org Mon Sep 11 05:14:03 2017 Return-Path: X-Original-To: apmail-tephra-dev-archive@minotaur.apache.org Delivered-To: apmail-tephra-dev-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 7EDCC1A932 for ; Mon, 11 Sep 2017 05:14:03 +0000 (UTC) Received: (qmail 82493 invoked by uid 500); 11 Sep 2017 05:14:03 -0000 Delivered-To: apmail-tephra-dev-archive@tephra.apache.org Received: (qmail 82457 invoked by uid 500); 11 Sep 2017 05:14:03 -0000 Mailing-List: contact dev-help@tephra.incubator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@tephra.incubator.apache.org Delivered-To: mailing list dev@tephra.incubator.apache.org Received: (qmail 82446 invoked by uid 99); 11 Sep 2017 05:14:03 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 11 Sep 2017 05:14:03 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id BD6541950E6 for ; Mon, 11 Sep 2017 05:14:02 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -100.002 X-Spam-Level: X-Spam-Status: No, score=-100.002 tagged_above=-999 required=6.31 tests=[RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id j8-SaBzY9svA for ; Mon, 11 Sep 2017 05:14:01 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id A43965FE16 for ; Mon, 11 Sep 2017 05:14:01 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 2EB2AE0EB1 for ; Mon, 11 Sep 2017 05:14:01 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 852622415B for ; Mon, 11 Sep 2017 05:14:00 +0000 (UTC) Date: Mon, 11 Sep 2017 05:14:00 +0000 (UTC) From: "ASF GitHub Bot (JIRA)" To: dev@tephra.incubator.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (TEPHRA-244) Invalid tx pruning does not handle deletion of tables well MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/TEPHRA-244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16160717#comment-16160717 ] ASF GitHub Bot commented on TEPHRA-244: --------------------------------------- Github user anew commented on a diff in the pull request: https://github.com/apache/incubator-tephra/pull/55#discussion_r137980868 --- Diff: tephra-hbase-compat-1.1-base/src/main/java/org/apache/tephra/hbase/txprune/HBaseTransactionPruningPlugin.java --- @@ -335,6 +351,25 @@ private long computePruneUpperBound(TimeRegions timeRegions) throws IOException return -1; } + private SortedSet filterDeletedRegions(final Set existingTables, --- End diff -- Better name this filterDeletedTables(), or filterDeletedTableRegions() - it does not filter out deleted regions - it does filter out all regions of deleted tables > Invalid tx pruning does not handle deletion of tables well > ---------------------------------------------------------- > > Key: TEPHRA-244 > URL: https://issues.apache.org/jira/browse/TEPHRA-244 > Project: Tephra > Issue Type: Bug > Components: core, manager > Affects Versions: 0.12.0-incubating > Reporter: Andreas Neumann > Assignee: Poorna Chandra > Fix For: 0.13.0-incubating > > > Suppose an application regularly creates and deletes tables (for example, temporary tables). In such a scenario, there will always be such a temporary table when pruning runs, and its regions will be recorded for that time. However, the region will be deleted before it ever compacts or flushes, and it will never record prune information. Because all prune times have such regions, there will never be a set of transactional regions that all have prune info, and pruning will never happen. > The fix is to exclude deleted tables from the list of regions at each time. This is all regions of deleted tables (tables that do not exist any more), not deleted regions: a region may disappear due to a split and its data will be in new regions. -- This message was sent by Atlassian JIRA (v6.4.14#64029)