From issues-return-199832-archive-asf-public=cust-asf.ponee.io@hive.apache.org Sat Oct 3 10:44:02 2020 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mxout1-ec2-va.apache.org (mxout1-ec2-va.apache.org [3.227.148.255]) by mx-eu-01.ponee.io (Postfix) with ESMTPS id A56B018063F for ; Sat, 3 Oct 2020 12:44:02 +0200 (CEST) Received: from mail.apache.org (mailroute1-lw-us.apache.org [207.244.88.153]) by mxout1-ec2-va.apache.org (ASF Mail Server at mxout1-ec2-va.apache.org) with SMTP id E2C3F41C1B for ; Sat, 3 Oct 2020 10:44:01 +0000 (UTC) Received: (qmail 24400 invoked by uid 500); 3 Oct 2020 10:44:01 -0000 Mailing-List: contact issues-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list issues@hive.apache.org Received: (qmail 24381 invoked by uid 99); 3 Oct 2020 10:44:01 -0000 Received: from mailrelay1-us-west.apache.org (HELO mailrelay1-us-west.apache.org) (209.188.14.139) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 03 Oct 2020 10:44:01 +0000 Received: from jira-he-de.apache.org (static.172.67.40.188.clients.your-server.de [188.40.67.172]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id D55BD40E8A for ; Sat, 3 Oct 2020 10:44:00 +0000 (UTC) Received: from jira-he-de.apache.org (localhost.localdomain [127.0.0.1]) by jira-he-de.apache.org (ASF Mail Server at jira-he-de.apache.org) with ESMTP id 53BCD7801F2 for ; Sat, 3 Oct 2020 10:44:00 +0000 (UTC) Date: Sat, 3 Oct 2020 10:44:00 +0000 (UTC) From: "ASF GitHub Bot (Jira)" To: issues@hive.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Work logged] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/HIVE-21052?focusedWorklogId=494269&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-494269 ] ASF GitHub Bot logged work on HIVE-21052: ----------------------------------------- Author: ASF GitHub Bot Created on: 03/Oct/20 10:43 Start Date: 03/Oct/20 10:43 Worklog Time Spent: 10m Work Description: vpnvishv commented on pull request #1415: URL: https://github.com/apache/hive/pull/1415#issuecomment-703083771 @deniskuzZ Thanks for reviewing and porting to master. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: users@infra.apache.org Issue Time Tracking ------------------- Worklog Id: (was: 494269) Time Spent: 1h (was: 50m) > Make sure transactions get cleaned if they are aborted before addPartitions is called > ------------------------------------------------------------------------------------- > > Key: HIVE-21052 > URL: https://issues.apache.org/jira/browse/HIVE-21052 > Project: Hive > Issue Type: Bug > Components: Transactions > Affects Versions: 3.0.0, 3.1.1 > Reporter: Jaume M > Assignee: Jaume M > Priority: Critical > Labels: pull-request-available > Attachments: Aborted Txn w_Direct Write.pdf, HIVE-21052.1.patch, HIVE-21052.10.patch, HIVE-21052.11.patch, HIVE-21052.12.patch, HIVE-21052.2.patch, HIVE-21052.3.patch, HIVE-21052.4.patch, HIVE-21052.5.patch, HIVE-21052.6.patch, HIVE-21052.7.patch, HIVE-21052.8.patch, HIVE-21052.9.patch > > Time Spent: 1h > Remaining Estimate: 0h > > If the transaction is aborted between openTxn and addPartitions and data has been written on the table the transaction manager will think it's an empty transaction and no cleaning will be done. > This is currently an issue in the streaming API and in micromanaged tables. As proposed by [~ekoifman] this can be solved by: > * Writing an entry with a special marker to TXN_COMPONENTS at openTxn and when addPartitions is called remove this entry from TXN_COMPONENTS and add the corresponding partition entry to TXN_COMPONENTS. > * If the cleaner finds and entry with a special marker in TXN_COMPONENTS that specifies that a transaction was opened and it was aborted it must generate jobs for the worker for every possible partition available. > cc [~ewohlstadter] -- This message was sent by Atlassian Jira (v8.3.4#803005)