hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ashutosh Chauhan <hashut...@apache.org>
Subject Re: Review Request 47353: HIVE-13616: Investigate renaming a table without invalidating the column stats
Date Mon, 16 May 2016 21:37:14 GMT


> On May 16, 2016, 9:22 p.m., Aihua Xu wrote:
> > metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java, lines
698-699
> > <https://reviews.apache.org/r/47353/diff/1/?file=1382994#file1382994line698>
> >
> >     Similar to openTransaction, commitTransaction will not do the real commit until
openTrasactionCalls reaches 0. So deleteTableColumnStatistics() or updateTableColumnStatistics()
will not be committed actually, until line 244 in which all the operations will be committed.
> >     
> >     
> >     Here is the commitTransaction() logic.
> >     
> >        openTrasactionCalls--;
> >         debugLog("Commit transaction: count = " + openTrasactionCalls + ", isactive
"+ currentTransaction.isActive());
> >     
> >         if ((openTrasactionCalls == 0) && currentTransaction.isActive())
{
> >           transactionStatus = TXN_STATUS.COMMITED;
> >           currentTransaction.commit();
> >         }

I didn't verify in debugger. But I would expect following is whats gonna happen.
AlterHandler calls deleteTableStats() which will call openTransaction() which increases openTransactionCalls
to 1. Before returning deleteTableStats call commitTransaction() which will set openTransactionCalls
to 0. Later when AlterHandler calls updateTableColumnStats() it will start new transaction,
since openTransactionCalls now is 0.

May be I am missing something obvious.


- Ashutosh


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47353/#review133447
-----------------------------------------------------------


On May 13, 2016, 7:17 p.m., Aihua Xu wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/47353/
> -----------------------------------------------------------
> 
> (Updated May 13, 2016, 7:17 p.m.)
> 
> 
> Review request for hive and Chaoyu Tang.
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> HIVE-13616: Investigate renaming a table without invalidating the column stats
> 
> 
> Diffs
> -----
> 
>   metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 0652b9db1a4224262611ff9258e871647442cfeb

>   ql/src/test/queries/clientpositive/rename_table_update_column_stats.q PRE-CREATION

>   ql/src/test/results/clientpositive/rename_table_update_column_stats.q.out PRE-CREATION

> 
> Diff: https://reviews.apache.org/r/47353/diff/
> 
> 
> Testing
> -------
> 
> Tests have been run and the failed tests don't seem to be related. 
> 
> To carry over the column stats during renaming the table, the patch 
> 1. Save affected column stats
> 2. Rename the table
> 3. Write the saved column stats for the new table.
> 
> In this order, column stats will find the foreign key when writing in 3).
> 
> 
> Thanks,
> 
> Aihua Xu
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message