hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aihua Xu <...@cloudera.com>
Subject Re: Review Request 50525: HIVE-14341: Altered skewed location is not respected for list bucketing
Date Tue, 20 Sep 2016 16:24:14 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/50525/#review149671
-----------------------------------------------------------




ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java (line 234)
<https://reviews.apache.org/r/50525/#comment217351>

    Actually the original one is also not overwrite but move from a temp file to the final
location. While that would fail if an previous file already exists. Then we should try to
rename to '_copy_1' and so.



ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java (line 899)
<https://reviews.apache.org/r/50525/#comment217352>

    Yes. That's where this issue comes from. Without this change, even if you alter the skew
location which will be saved in locationMap, it would get the old location and save it to
overwrite the newer location. That is not correct.
    
    The new logic is: if we have the mapped location for the skew keys, then use it. Otherwise,
get the default skew location.


- Aihua Xu


On Sept. 19, 2016, 9:02 p.m., Aihua Xu wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/50525/
> -----------------------------------------------------------
> 
> (Updated Sept. 19, 2016, 9:02 p.m.)
> 
> 
> Review request for hive.
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> HIVE-14341: Altered skewed location is not respected for list bucketing
> 
> 
> Diffs
> -----
> 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/FileSinkOperator.java e386717 
>   ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java da46854 
>   ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatUtils.java
ba4f6a7 
>   ql/src/test/queries/clientpositive/create_alter_list_bucketing_table1.q bf89e8f 
>   ql/src/test/results/clientpositive/create_alter_list_bucketing_table1.q.out 216d3be

> 
> Diff: https://reviews.apache.org/r/50525/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Aihua Xu
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message