hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amareshwari Sriramadasu" <amareshw...@apache.org>
Subject Re: Review Request 36939: HIVE-11376: CombineHiveInputFormat is falling back to HiveInputFormat in case codecs are found for one of the input files
Date Mon, 03 Aug 2015 06:10:56 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/36939/#review93869
-----------------------------------------------------------

Ship it!


Ship It!

- Amareshwari Sriramadasu


On July 30, 2015, 1:19 p.m., Rajat Khandelwal wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/36939/
> -----------------------------------------------------------
> 
> (Updated July 30, 2015, 1:19 p.m.)
> 
> 
> Review request for hive.
> 
> 
> Bugs: HIVE-11376
>     https://issues.apache.org/jira/browse/HIVE-11376
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/io/CombineHiveInputFormat.java#L379
> 
> This is the exact code snippet:
> 
> {noformat}
> 
> / Since there is no easy way of knowing whether MAPREDUCE-1597 is present in the tree
or not,
>       // we use a configuration variable for the same
>       if (this.mrwork != null && !this.mrwork.getHadoopSupportsSplittable())
{
>         // The following code should be removed, once
>         // https://issues.apache.org/jira/browse/MAPREDUCE-1597 is fixed.
>         // Hadoop does not handle non-splittable files correctly for CombineFileInputFormat,
>         // so don't use CombineFileInputFormat for non-splittable files
> 
>         //ie, dont't combine if inputformat is a TextInputFormat and has compression
turned on
> 
> {noformat}
> 
> 
> Diffs
> -----
> 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 33b67dd7b0fde41f81f8d86ea8c83d29c631e3d7

>   ql/src/java/org/apache/hadoop/hive/ql/io/CombineHiveInputFormat.java 1de7e4073f5eea4c7be8423a7ebe6a89cb51d9f1

>   ql/src/java/org/apache/hadoop/hive/ql/optimizer/GenMapRedUtils.java 693d8c7e9f956999b2da33593d780e37ddf2b3b8

>   ql/src/java/org/apache/hadoop/hive/ql/plan/MapWork.java 3217df27bb5731a1dcd5db1ae17c5bdff2e3fbfc

>   ql/src/java/org/apache/hadoop/hive/ql/plan/PlanUtils.java 76926e79729d9ea4823de0ffc9b1e5bac6364842

> 
> Diff: https://reviews.apache.org/r/36939/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> Rajat Khandelwal
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message