hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Szehon Ho" <sze...@cloudera.com>
Subject Re: Review Request 24284: HIVE-7441:Custom partition scheme gets rewritten with hive scheme upon concatenate
Date Tue, 05 Aug 2014 19:12:45 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/24284/#review49631
-----------------------------------------------------------



metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java
<https://reviews.apache.org/r/24284/#comment86881>

    Can we get rid of the space after the '!'?


- Szehon Ho


On Aug. 5, 2014, 1:12 a.m., Chaoyu Tang wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/24284/
> -----------------------------------------------------------
> 
> (Updated Aug. 5, 2014, 1:12 a.m.)
> 
> 
> Review request for hive.
> 
> 
> Bugs: HIVE-7441
>     https://issues.apache.org/jira/browse/HIVE-7441
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> Problem (See HIVE-7441 description):
> When Hive tries to concatenate the partition data files with hdfs cache is disabled (fs.hdfs.impl.disable.cache=true),
the concatenated data file will be created under a partition directory with a name in form
of /part=value (e.g. /b=part1).  If the original partition directory is different (e.g /part1),
querying the concatenated partition returns empty result since the partition location metastore
still points to the original partition directory (/part1).
> 
> Cause:
> When hdfs cache is disabled, filesystem equals comparison returns false for two fs objects
even from same filesystem, which causes the original partition directory (/part1) NOT to be
assigned and used as the concatenated partition destination directory.
> 
> Solution:
> Before HDFS-4321 is fixed, we compare two filesystems using their uri.
> 
> 
> Diffs
> -----
> 
>   common/src/java/org/apache/hadoop/hive/common/FileUtils.java 8dcd4cf4432b261dff8d549b07704d75f4f1c769

>   metastore/src/java/org/apache/hadoop/hive/metastore/HiveAlterHandler.java 3126880aa1a1c5ed93c9576e84c980511e86145b

>   ql/src/java/org/apache/hadoop/hive/ql/metadata/Hive.java a7e50adf02e795d192a29d918e3ac24665461e06

>   ql/src/test/queries/clientpositive/alter_merge_3.q PRE-CREATION 
>   ql/src/test/results/clientpositive/alter_merge_3.q.out PRE-CREATION 
> 
> Diff: https://reviews.apache.org/r/24284/diff/
> 
> 
> Testing
> -------
> 
> 1. Manual tests verified the solution fixed the reported issue
> 2. New added qtests passed
> 3. Submit the patch and run pre-commit tests to see if there is any regression.
> 
> 
> Thanks,
> 
> Chaoyu Tang
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message