hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From John Pullokkaran <jpullokka...@hortonworks.com>
Subject Re: Review Request 42487: Use bit vector to track NDV
Date Fri, 22 Jan 2016 23:39:30 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/42487/#review115948
-----------------------------------------------------------




metastore/src/java/org/apache/hadoop/hive/metastore/NumDistinctValueEstimator.java (line 46)
<https://reviews.apache.org/r/42487/#comment176992>

    We should meassure, computational/space complexity of Bitwise opeartions & FastBitSet.
    
    It would be good to get a meassure for 1000, 10000, 500000 partitions



ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java (line 1563)
<https://reviews.apache.org/r/42487/#comment176980>

    Why is this needed?
    Seems like its not used.


- John Pullokkaran


On Jan. 22, 2016, 7:09 p.m., pengcheng xiong wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/42487/
> -----------------------------------------------------------
> 
> (Updated Jan. 22, 2016, 7:09 p.m.)
> 
> 
> Review request for hive, Alan Gates and John Pullokkaran.
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> HIVE-12763
> 
> 
> Diffs
> -----
> 
>   common/src/java/org/apache/hadoop/hive/common/HiveStatsUtils.java 9193f80 
>   metastore/if/hive_metastore.thrift 81837e6 
>   metastore/pom.xml a8e84a1 
>   metastore/src/gen/protobuf/gen-java/org/apache/hadoop/hive/metastore/hbase/HbaseMetastoreProto.java
39a7278 
>   metastore/src/gen/thrift/gen-cpp/hive_metastore_types.h ce1d7da 
>   metastore/src/gen/thrift/gen-cpp/hive_metastore_types.cpp 0203b06 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/BinaryColumnStatsData.java
84e393c 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/BooleanColumnStatsData.java
6aa4668 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/DateColumnStatsData.java
2ebb811 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/DecimalColumnStatsData.java
720176a 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/DoubleColumnStatsData.java
5d48b5d 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/LongColumnStatsData.java
2f41c5a 
>   metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/StringColumnStatsData.java
bd8a922 
>   metastore/src/gen/thrift/gen-php/metastore/Types.php 380e6d0 
>   metastore/src/gen/thrift/gen-py/hive_metastore/ttypes.py 409c247 
>   metastore/src/gen/thrift/gen-rb/hive_metastore_types.rb a473611 
>   metastore/src/java/org/apache/hadoop/hive/metastore/NumDistinctValueEstimator.java
PRE-CREATION 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/HBaseUtils.java f4df2e2 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/StatsCache.java 5ec60be 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/BinaryColumnStatsAggregator.java
bbd2c7b 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/BooleanColumnStatsAggregator.java
9047f68 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/ColumnStatsAggregator.java
217b654 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/ColumnStatsAggregatorFactory.java
a8dbc1f 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/DecimalColumnStatsAggregator.java
ec25b31 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/DoubleColumnStatsAggregator.java
71af0ac 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/LongColumnStatsAggregator.java
15b8cf7 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/StringColumnStatsAggregator.java
fe1a04c 
>   metastore/src/protobuf/org/apache/hadoop/hive/metastore/hbase/hbase_metastore_proto.proto
0d0ef89 
>   metastore/src/test/org/apache/hadoop/hive/metastore/hbase/TestHBaseStoreBitVector.java
PRE-CREATION 
>   ql/pom.xml 358cd2a 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/ColumnStatsTask.java 7914471 
>   ql/src/java/org/apache/hadoop/hive/ql/parse/ColumnStatsSemanticAnalyzer.java 1f30cbd

>   ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java b4cf58f 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFComputeStats.java 0e96f89

>   ql/src/test/queries/clientpositive/tez_aggr_part_stats.q PRE-CREATION 
>   ql/src/test/results/clientpositive/char_udf1.q.java1.7.out bfed116 
>   ql/src/test/results/clientpositive/columnstats_partlvl.q.out b7c9075 
>   ql/src/test/results/clientpositive/columnstats_partlvl_dp.q.out 9685202 
>   ql/src/test/results/clientpositive/compute_stats_date.q.out b57a862 
>   ql/src/test/results/clientpositive/compute_stats_decimal.q.out 35abb37 
>   ql/src/test/results/clientpositive/compute_stats_double.q.out f6b4052 
>   ql/src/test/results/clientpositive/compute_stats_empty_table.q.out f76c760 
>   ql/src/test/results/clientpositive/compute_stats_long.q.out 2c6171d 
>   ql/src/test/results/clientpositive/compute_stats_string.q.out bdf9d85 
>   ql/src/test/results/clientpositive/temp_table_display_colstats_tbllvl.q.out ae39d18

>   ql/src/test/results/clientpositive/tez/tez_aggr_part_stats.q.out PRE-CREATION 
>   ql/src/test/results/clientpositive/varchar_udf1.q.java1.7.out 853bc4a 
> 
> Diff: https://reviews.apache.org/r/42487/diff/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> pengcheng xiong
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message