hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Zoltan Haindrich <k...@rxd.hu>
Subject Re: Review Request 68109: HIVE-20260 NDV of a column shouldn't be scaled when row count is changed by filter on another column
Date Wed, 01 Aug 2018 21:29:38 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/68109/
-----------------------------------------------------------

(Updated Aug. 1, 2018, 9:29 p.m.)


Review request for hive and Ashutosh Chauhan.


Changes
-------

patch#01


Bugs: HIVE-20260
    https://issues.apache.org/jira/browse/HIVE-20260


Repository: hive-git


Description
-------

* keep track of used column; and only rescale affected columns
* much more conservative than old logic - possible too much...
* wip patch


Diffs (updated)
-----

  ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/AnnotateStatsProcCtx.java
47ee949fbc 
  ql/src/java/org/apache/hadoop/hive/ql/optimizer/stats/annotation/StatsRulesProcFactory.java
3c2b085426 
  ql/src/test/queries/clientpositive/groupby_groupingset_bug.q c6e9a1a590 
  ql/src/test/queries/clientpositive/reopt_semijoin.q 0eacb8a68a 
  ql/src/test/queries/clientpositive/stat_estimate_drill.q PRE-CREATION 
  ql/src/test/queries/clientpositive/stat_estimate_related_col.q 52da2f759a 
  ql/src/test/results/clientpositive/annotate_stats_deep_filters.q.out 83bb65ede4 
  ql/src/test/results/clientpositive/cbo_rp_auto_join1.q.out f937834651 
  ql/src/test/results/clientpositive/llap/auto_join29.q.out 9db7cb3e79 
  ql/src/test/results/clientpositive/llap/auto_smb_mapjoin_14.q.out 21273c17f3 
  ql/src/test/results/clientpositive/llap/auto_sortmerge_join_10.q.out 9e07f1ce2b 
  ql/src/test/results/clientpositive/llap/auto_sortmerge_join_9.q.out 4ffa7e274a 
  ql/src/test/results/clientpositive/llap/bucket_map_join_tez2.q.out 800bcaa7b0 
  ql/src/test/results/clientpositive/llap/bucketsortoptimize_insert_7.q.out c7f5b887b6 
  ql/src/test/results/clientpositive/llap/constprog_semijoin.q.out 2ccde41e29 
  ql/src/test/results/clientpositive/llap/correlationoptimizer1.q.out b156ce3938 
  ql/src/test/results/clientpositive/llap/correlationoptimizer2.q.out eb0dcad7a4 
  ql/src/test/results/clientpositive/llap/correlationoptimizer6.q.out 6c3e65dea3 
  ql/src/test/results/clientpositive/llap/dynpart_sort_opt_vectorization.q.out 3b72fc4f28

  ql/src/test/results/clientpositive/llap/explainuser_1.q.out 708fa17617 
  ql/src/test/results/clientpositive/llap/explainuser_2.q.out 71e1f29ff6 
  ql/src/test/results/clientpositive/llap/explainuser_4.q.out fe330727d4 
  ql/src/test/results/clientpositive/llap/filter_join_breaktask.q.out 79fb6caee3 
  ql/src/test/results/clientpositive/llap/groupby_groupingset_bug.q.out 5e18f33f67 
  ql/src/test/results/clientpositive/llap/hybridgrace_hashjoin_1.q.out 333b10323b 
  ql/src/test/results/clientpositive/llap/limit_pushdown.q.out 252831eed1 
  ql/src/test/results/clientpositive/llap/load_dyn_part1.q.out 680f82cf57 
  ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_3.q.out 2538ac1ba9

  ql/src/test/results/clientpositive/llap/materialized_view_create_rewrite_rebuild_dummy.q.out
193e95909e 
  ql/src/test/results/clientpositive/llap/mrr.q.out 314031f61d 
  ql/src/test/results/clientpositive/llap/multiMapJoin2.q.out 1c6bc941ad 
  ql/src/test/results/clientpositive/llap/orc_llap.q.out 7f8667e637 
  ql/src/test/results/clientpositive/llap/orc_predicate_pushdown.q.out e29c3d9915 
  ql/src/test/results/clientpositive/llap/parquet_predicate_pushdown.q.out eaa7e5dec3 
  ql/src/test/results/clientpositive/llap/reopt_semijoin.q.out e60b207ba9 
  ql/src/test/results/clientpositive/llap/sample10.q.out 66214ec11e 
  ql/src/test/results/clientpositive/llap/sample10_mm.q.out 28d0cd052f 
  ql/src/test/results/clientpositive/llap/semijoin.q.out eb171453fc 
  ql/src/test/results/clientpositive/llap/semijoin6.q.out 05a191a997 
  ql/src/test/results/clientpositive/llap/semijoin7.q.out 31df60a1da 
  ql/src/test/results/clientpositive/llap/skewjoin.q.out c3f70111cb 
  ql/src/test/results/clientpositive/llap/smb_mapjoin_14.q.out 3c6ba4e6ce 
  ql/src/test/results/clientpositive/llap/subquery_exists.q.out e5fcd97cfc 
  ql/src/test/results/clientpositive/llap/subquery_in.q.out 8007025e5b 
  ql/src/test/results/clientpositive/llap/subquery_in_having.q.out 20428e1444 
  ql/src/test/results/clientpositive/llap/subquery_multi.q.out 0c0682def3 
  ql/src/test/results/clientpositive/llap/subquery_notin.q.out 11facd1857 
  ql/src/test/results/clientpositive/llap/subquery_scalar.q.out 87681d1477 
  ql/src/test/results/clientpositive/llap/subquery_select.q.out bd569b3285 
  ql/src/test/results/clientpositive/llap/subquery_views.q.out 370accb94e 
  ql/src/test/results/clientpositive/llap/tez_dynpart_hashjoin_2.q.out 55c581fc3c 
  ql/src/test/results/clientpositive/llap/tez_vector_dynpart_hashjoin_2.q.out 55c581fc3c 
  ql/src/test/results/clientpositive/llap/unionDistinct_1.q.out 47a0be98cd 
  ql/src/test/results/clientpositive/llap/unionDistinct_3.q.out 998d497a56 
  ql/src/test/results/clientpositive/llap/vector_groupby_grouping_sets2.q.out f8220e1927 
  ql/src/test/results/clientpositive/llap/vector_mapjoin_reduce.q.out c31b75919c 
  ql/src/test/results/clientpositive/llap/vector_windowing_gby.q.out 5dffd1561b 
  ql/src/test/results/clientpositive/llap/vector_windowing_gby2.q.out 75c393e932 
  ql/src/test/results/clientpositive/llap/vectorization_0.q.out 7abdc3c9a4 
  ql/src/test/results/clientpositive/llap/vectorization_limit.q.out f7c00f8153 
  ql/src/test/results/clientpositive/llap/vectorization_short_regress.q.out 2fc659deca 
  ql/src/test/results/clientpositive/llap/vectorized_context.q.out 6570f745d9 
  ql/src/test/results/clientpositive/llap/vectorized_mapjoin.q.out 97e8be6973 
  ql/src/test/results/clientpositive/llap/vectorized_nested_mapjoin.q.out 27bab206f0 
  ql/src/test/results/clientpositive/llap/vectorized_shufflejoin.q.out 597c432d77 
  ql/src/test/results/clientpositive/llap/windowing_gby.q.out eef7562b2f 
  ql/src/test/results/clientpositive/spark/spark_explainuser_1.q.out c5d0d63f8c 
  ql/src/test/results/clientpositive/stat_estimate_drill.q.out PRE-CREATION 
  ql/src/test/results/clientpositive/stat_estimate_related_col.q.out 669adafda3 
  ql/src/test/results/clientpositive/tez/explainanalyze_1.q.out 3ad2ba26f0 
  ql/src/test/results/clientpositive/tez/explainanalyze_4.q.out d7d11d8a57 
  ql/src/test/results/clientpositive/tez/hybridgrace_hashjoin_1.q.out 029f79c308 


Diff: https://reviews.apache.org/r/68109/diff/3/

Changes: https://reviews.apache.org/r/68109/diff/2-3/


Testing
-------


Thanks,

Zoltan Haindrich


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message