hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From j.prasant...@gmail.com
Subject Re: Review Request 60753: Add HLL as an alternative to FM sketch to compute stats
Date Mon, 10 Jul 2017 22:02:47 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/60753/#review180119
-----------------------------------------------------------




common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
Line 1723 (original), 1723 (patched)
<https://reviews.apache.org/r/60753/#comment255142>

    I am not sure if we need this config. 
    Any reason to support storing FM sketch's bitvectors?
    
    If this is for 3.0.0 release only, we could even remove this config.
    
    If we need to support FM sketch based NDV then having a separate field in metastore will
be better as Ashutosh suggested.


- Prasanth_J


On July 10, 2017, 9:29 p.m., pengcheng xiong wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/60753/
> -----------------------------------------------------------
> 
> (Updated July 10, 2017, 9:29 p.m.)
> 
> 
> Review request for hive, Ashutosh Chauhan and Prasanth_J.
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> HIVE-16966
> 
> 
> Diffs
> -----
> 
>   common/pom.xml e6722babd8 
>   common/src/java/org/apache/hadoop/hive/common/HiveStatsUtils.java 7c9d72fbd2 
>   common/src/java/org/apache/hadoop/hive/common/ndv/NumDistinctValueEstimator.java PRE-CREATION

>   common/src/java/org/apache/hadoop/hive/common/ndv/NumDistinctValueEstimatorFactory.java
PRE-CREATION 
>   common/src/java/org/apache/hadoop/hive/common/ndv/hll/HLLConstants.java PRE-CREATION

>   common/src/java/org/apache/hadoop/hive/common/ndv/hll/HLLDenseRegister.java PRE-CREATION

>   common/src/java/org/apache/hadoop/hive/common/ndv/hll/HLLRegister.java PRE-CREATION

>   common/src/java/org/apache/hadoop/hive/common/ndv/hll/HLLSparseRegister.java PRE-CREATION

>   common/src/java/org/apache/hadoop/hive/common/ndv/hll/HyperLogLog.java PRE-CREATION

>   common/src/java/org/apache/hadoop/hive/common/ndv/hll/HyperLogLogUtils.java PRE-CREATION

>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java 5700fb9325 
>   common/src/java/org/apache/hadoop/hive/conf/HiveConf.java.orig da48a7ccbd 
>   metastore/src/java/org/apache/hadoop/hive/metastore/NumDistinctValueEstimator.java
92f9a845e3 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/DecimalColumnStatsAggregator.java
36b2c9c56b 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/DoubleColumnStatsAggregator.java
a88ef84e5c 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/LongColumnStatsAggregator.java
8ac6561aec 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/StringColumnStatsAggregator.java
2aa4046a46 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/ColumnStatsMerger.java
33c7e3e52c 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/ColumnStatsMergerFactory.java
fe890e4e27 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/DateColumnStatsMerger.java
3179b23438 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/DecimalColumnStatsMerger.java
c13add9d9c 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/DoubleColumnStatsMerger.java
fbdba24b0a 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/LongColumnStatsMerger.java
ac65590505 
>   metastore/src/java/org/apache/hadoop/hive/metastore/hbase/stats/merge/StringColumnStatsMerger.java
41587477d3 
>   pom.xml f9fae59a5d 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/generic/DecimalNumDistinctValueEstimator.java
a05906edfa 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/generic/DoubleNumDistinctValueEstimator.java
e76fc74dbc 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/generic/GenericUDAFComputeStats.java 2ebfcb2360

>   ql/src/java/org/apache/hadoop/hive/ql/udf/generic/LongNumDistinctValueEstimator.java
1c197a028a 
>   ql/src/java/org/apache/hadoop/hive/ql/udf/generic/NumDistinctValueEstimator.java fa70f49857

>   ql/src/java/org/apache/hadoop/hive/ql/udf/generic/StringNumDistinctValueEstimator.java
601901c163 
>   ql/src/test/queries/clientpositive/hll.q PRE-CREATION 
>   ql/src/test/results/clientpositive/hll.q.out PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/60753/diff/1/
> 
> 
> Testing
> -------
> 
> 
> Thanks,
> 
> pengcheng xiong
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message