hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Marta Kuczora <kuczo...@cloudera.com>
Subject Re: Review Request 54065: HIVE-15282: Different modification times are used when an index is built and when its staleness is checked
Date Thu, 02 Mar 2017 15:31:36 GMT


> On Jan. 27, 2017, 4:18 p.m., Sergio Pena wrote:
> > ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java
> > Lines 972 (patched)
> > <https://reviews.apache.org/r/54065/diff/1/?file=1570137#file1570137line972>
> >
> >     If this condition does not happen ever, then lastModificationTime will end up
being null, and basePartTs will contain the null value. Should we use the dataLocation timestamp
in case of this condition is never called?

Yes, you are right! This issue is fixed by fixing the previous one.


- Marta


-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54065/#review163281
-----------------------------------------------------------


On Dec. 12, 2016, 1:04 p.m., Marta Kuczora wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54065/
> -----------------------------------------------------------
> 
> (Updated Dec. 12, 2016, 1:04 p.m.)
> 
> 
> Review request for hive, Aihua Xu, Chaoyu Tang, Peter Vary, and Sergio Pena.
> 
> 
> Bugs: HIVE-15282
>     https://issues.apache.org/jira/browse/HIVE-15282
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> Changed the way how the modification time is determined for partitions in the DDLTask.alterIndex
method to be the same as when the index staleness is checked. Instead of using the modification
date of the partition folder, go through the files in the folder and use the highest modification
time and save it as index property. With this we can avoid the issue when the folder and the
file is created when the second turns. So the modification time of the folder is in the previous
second compared to the modification time of the file.
> If the partition folder doesn't contain any files, then use the folder's modification
time, just as before.
> 
> 
> Diffs
> -----
> 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java cfece77 
> 
> 
> Diff: https://reviews.apache.org/r/54065/diff/1/
> 
> 
> Testing
> -------
> 
> Ran the index_auto_mult_tables_compact and index_auto_mult_tables q tests multiple times,
with hard-coded delay with which the test failure described in HIVE-15282 could be reproduced.
With the patch, the tests were always successful.
> Also ran all index related q tests.
> 
> 
> Thanks,
> 
> Marta Kuczora
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message