Mailing-List: contact dev-help@hive.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hive.apache.org
Content-Type: multipart/alternative;
 boundary="===============8964786901083672557=="
MIME-Version: 1.0
Subject: Re: Review Request 47353: HIVE-13616: Investigate renaming a table
 without invalidating the column stats
From: Ashutosh Chauhan <hashutosh@apache.org>
To: Chaoyu Tang <ctang.ma@gmail.com>
Cc: Ashutosh Chauhan <hashutosh@apache.org>, hive <dev@hive.apache.org>,
 Aihua Xu <axu@cloudera.com>
Date: Mon, 16 May 2016 21:37:14 -0000
Message-ID: <20160516213714.8624.646@reviews.apache.org>
Auto-Submitted: auto-generated
Sender: Ashutosh Chauhan <noreply@reviews.apache.org>
References: <20160516212257.8623.521@reviews.apache.org>
In-Reply-To: <20160516212257.8623.521@reviews.apache.org>
Reply-To: Ashutosh Chauhan <hashutosh@apache.org>
archived-at: Mon, 16 May 2016 21:37:18 -0000

--===============8964786901083672557==
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit


> On May 16, 2016, 9:22 p.m., Aihua Xu wrote:
> > metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java, lines 698-699
> > <https://reviews.apache.org/r/47353/diff/1/?file=1382994#file1382994line698>
> >
> >     Similar to openTransaction, commitTransaction will not do the real commit until openTrasactionCalls reaches 0. So deleteTableColumnStatistics() or updateTableColumnStatistics() will not be committed actually, until line 244 in which all the operations will be committed.
> >     
> >     
> >     Here is the commitTransaction() logic.
> >     
> >        openTrasactionCalls--;
> >         debugLog("Commit transaction: count = " + openTrasactionCalls + ", isactive "+ currentTransaction.isActive());
> >     
> >         if ((openTrasactionCalls == 0) && currentTransaction.isActive()) {
> >           transactionStatus = TXN_STATUS.COMMITED;
> >           currentTransaction.commit();
> >         }

I didn't verify in debugger. But I would expect following is whats gonna happen.
AlterHandler calls deleteTableStats() which will call openTransaction() which increases openTransactionCalls to 1. Before returning deleteTableStats call commitTransaction() which will set openTransactionCalls to 0. Later when AlterHandler calls updateTableColumnStats() it will start new transaction, since openTransactionCalls now is 0.

May be I am missing something obvious.


- Ashutosh


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47353/#review133447
-----------------------------------------------------------


On May 13, 2016, 7:17 p.m., Aihua Xu wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47353/
> -----------------------------------------------------------
> 
> (Updated May 13, 2016, 7:17 p.m.)
> 
> 
> Review request for hive and Chaoyu Tang.
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> HIVE-13616: Investigate renaming a table without invalidating the column stats
> 
> 
> Diffs
> -----
> 
>   metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 0652b9db1a4224262611ff9258e871647442cfeb 
>   ql/src/test/queries/clientpositive/rename_table_update_column_stats.q PRE-CREATION 
>   ql/src/test/results/clientpositive/rename_table_update_column_stats.q.out PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/47353/diff/
> 
> 
> Testing
> -------
> 
> Tests have been run and the failed tests don't seem to be related. 
> 
> To carry over the column stats during renaming the table, the patch 
> 1. Save affected column stats
> 2. Rename the table
> 3. Write the saved column stats for the new table.
> 
> In this order, column stats will find the foreign key when writing in 3).
> 
> 
> Thanks,
> 
> Aihua Xu
> 
>


--===============8964786901083672557==--