Mailing-List: contact dev-help@tephra.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@tephra.incubator.apache.org
Date: Tue, 12 Jul 2016 07:51:20 +0000 (UTC)
From: "James Taylor (JIRA)" <jira@apache.org>
To: dev@tephra.incubator.apache.org
Message-ID: <JIRA.12982973.1409263454000.455.1468309880458@Atlassian.JIRA>
In-Reply-To: <JIRA.12982973.1409263454000@Atlassian.JIRA>
References: <JIRA.12982973.1409263454000@Atlassian.JIRA> <JIRA.12982973.1409263454000@arcas>
Subject: [jira] [Commented] (TEPHRA-35) Prune invalid transaction set once
 all data for a given invalid transaction has been dropped
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Tue, 12 Jul 2016 07:51:25 -0000


    [ https://issues.apache.org/jira/browse/TEPHRA-35?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15372470#comment-15372470 ] 

James Taylor commented on TEPHRA-35:
------------------------------------

bq. At the time a transaction client starts a transaction, if it knows in advance all the tables that will be used in the transaction (like in Phoenix's case)
Phoenix doesn't know all of the tables involved at the start of a transaction. It would at the end of the transaction, though (for example, during canCommit()).

I think it's fine to do this in a phased approach, starting with having to wait for all tables to major compact.


> Prune invalid transaction set once all data for a given invalid transaction has been dropped
> --------------------------------------------------------------------------------------------
>
>                 Key: TEPHRA-35
>                 URL: https://issues.apache.org/jira/browse/TEPHRA-35
>             Project: Tephra
>          Issue Type: New Feature
>            Reporter: Gary Helmling
>            Assignee: Poorna Chandra
>            Priority: Blocker
>         Attachments: ApacheTephraAutomaticInvalidListPruning-v2.pdf
>
>
> In addition to dropping the data from invalid transactions we need to be able to prune the invalid set of any transactions where data cleanup has been completely performed. Without this, the invalid set will grow indefinitely and become a greater and greater cost to in-progress transactions over time.
> To do this correctly, the TransactionDataJanitor coprocessor will need to maintain some bookkeeping for the transaction data that it removes, so that the transaction manager can reason about when all of a given transaction's data has been removed. Only at this point can the transaction manager safely drop the transaction ID from the invalid set.
> One approach would be for the TransactionDataJanitor to update a table marking when a major compaction was performed on a region and what transaction IDs were filtered out. Once all regions in a table containing the transaction data have been compacted, we can remove the filtered out transaction IDs from the invalid set. However, this will need to cope with changing region names due to splits, etc.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)