hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ashutosh Chauhan <hashut...@apache.org>
Subject Re: Review Request 62424: HIVE-17536 StatsUtil::getBasicStatForTable doesn't distinguish b/w absence of statistics or zero stats
Date Wed, 20 Sep 2017 00:27:37 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/62424/#review185757
-----------------------------------------------------------



Seems like estimation is no longer happening when there are no basic stats. Can you investigate
that ?


ql/src/test/results/clientpositive/auto_sortmerge_join_5.q.out
Line 79 (original), 79 (patched)
<https://reviews.apache.org/r/62424/#comment262113>

    This looks a case where there are no basic stats. But we do have data size, using that
we should have had a good estimate. Looks estimate became worse. 
    Also, Basic stats: COMPLETE, is that correct?



ql/src/test/results/clientpositive/decimal_udf.q.out
Line 41 (original), 41 (patched)
<https://reviews.apache.org/r/62424/#comment262114>

    Is this changed for better?


- Ashutosh Chauhan


On Sept. 19, 2017, 11:46 p.m., Vineet Garg wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/62424/
> -----------------------------------------------------------
> 
> (Updated Sept. 19, 2017, 11:46 p.m.)
> 
> 
> Review request for hive and Ashutosh Chauhan.
> 
> 
> Bugs: HIVE_17536
>     https://issues.apache.org/jira/browse/HIVE_17536
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> This patch updates getBasicStatForTable to return -1 if numRows property doesn't exist
in metastore . This is done so that the caller of this method could distinguish b/w existence
of stats and actual zero rows.
> 
> 
> Diffs
> -----
> 
>   ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java 17d9f2df0a 
>   ql/src/test/results/clientpositive/auto_sortmerge_join_5.q.out fbbe64ee91 
>   ql/src/test/results/clientpositive/beeline/smb_mapjoin_1.q.out 52314d93a5 
>   ql/src/test/results/clientpositive/beeline/smb_mapjoin_2.q.out 56a38a25af 
>   ql/src/test/results/clientpositive/beeline/smb_mapjoin_3.q.out f33b8bbd5f 
>   ql/src/test/results/clientpositive/bucketcontext_5.q.out 5db684df83 
>   ql/src/test/results/clientpositive/bucketcontext_6.q.out b3f00a2485 
>   ql/src/test/results/clientpositive/bucketmapjoin5.q.out 4b989932ce 
>   ql/src/test/results/clientpositive/bucketmapjoin_negative.q.out 97cb1f10d3 
>   ql/src/test/results/clientpositive/bucketmapjoin_negative2.q.out 724df736ee 
>   ql/src/test/results/clientpositive/bucketmapjoin_negative3.q.out 8a74cd2527 
>   ql/src/test/results/clientpositive/cbo_rp_join1.q.out df6510c066 
>   ql/src/test/results/clientpositive/cbo_rp_udaf_percentile_approx_23.q.out 12d9c108b3

>   ql/src/test/results/clientpositive/columnstats_tbllvl.q.out 25892d8fa6 
>   ql/src/test/results/clientpositive/compute_stats_date.q.out e738c25fc9 
>   ql/src/test/results/clientpositive/correlationoptimizer5.q.out 00bdb4caa1 
>   ql/src/test/results/clientpositive/decimal_precision.q.out d559bfd6f7 
>   ql/src/test/results/clientpositive/decimal_udf.q.out 8bda27ae87 
>   ql/src/test/results/clientpositive/decimal_udf2.q.out 6ef4a89402 
>   ql/src/test/results/clientpositive/display_colstats_tbllvl.q.out 8cbed9accf 
>   ql/src/test/results/clientpositive/distinct_windowing.q.out 197687a93c 
>   ql/src/test/results/clientpositive/distinct_windowing_no_cbo.q.out 85d0777525 
>   ql/src/test/results/clientpositive/drop_table_with_index.q.out 87c6978e7b 
>   ql/src/test/results/clientpositive/encrypted/encryption_join_unencrypted_tbl.q.out
5d894abc79 
>   ql/src/test/results/clientpositive/filter_cond_pushdown2.q.out 88605139a9 
>   ql/src/test/results/clientpositive/gen_udf_example_add10.q.out 707d6decd2 
>   ql/src/test/results/clientpositive/groupby10.q.out 66832b02fc 
>   ql/src/test/results/clientpositive/groupby_grouping_id3.q.out 6670d9c9ee 
>   ql/src/test/results/clientpositive/groupby_grouping_sets_grouping.q.out 39dd731f34

>   ql/src/test/results/clientpositive/index_serde.q.out c1d7ea34b2 
>   ql/src/test/results/clientpositive/input22.q.out 687510670b 
>   ql/src/test/results/clientpositive/input3_limit.q.out 560de96131 
>   ql/src/test/results/clientpositive/input4.q.out 69843183a5 
>   ql/src/test/results/clientpositive/join_star.q.out cbb7a03d6a 
>   ql/src/test/results/clientpositive/llap/acid_bucket_pruning.q.out 61e3dfde2c 
>   ql/src/test/results/clientpositive/llap/auto_sortmerge_join_5.q.out 7bab86fa14 
>   ql/src/test/results/clientpositive/llap/bucketmapjoin1.q.out e1373c438a 
>   ql/src/test/results/clientpositive/llap/bucketmapjoin4.q.out dd67a55a72 
>   ql/src/test/results/clientpositive/llap/column_table_stats.q.out c397f80e71 
>   ql/src/test/results/clientpositive/llap/correlationoptimizer4.q.out edf8506e73 
>   ql/src/test/results/clientpositive/llap/count.q.out 81b94e72e6 
>   ql/src/test/results/clientpositive/llap/dynpart_sort_optimization.q.out 36898ceed9

>   ql/src/test/results/clientpositive/llap/dynpart_sort_optimization2.q.out 4a79503587

>   ql/src/test/results/clientpositive/llap/explainuser_1.q.out 03ebe375b8 
>   ql/src/test/results/clientpositive/llap/join_is_not_distinct_from.q.out ce903eae56

>   ql/src/test/results/clientpositive/llap/join_nullsafe.q.out ce399dac7d 
>   ql/src/test/results/clientpositive/llap/mapjoin_hint.q.out 7fa188383f 
>   ql/src/test/results/clientpositive/llap/orc_merge5.q.out 8be0f55cfc 
>   ql/src/test/results/clientpositive/llap/orc_merge6.q.out afba086cde 
>   ql/src/test/results/clientpositive/llap/orc_merge7.q.out 58e37c28d0 
>   ql/src/test/results/clientpositive/llap/orc_merge_incompat1.q.out f3d34cce05 
>   ql/src/test/results/clientpositive/llap/orc_merge_incompat2.q.out 1419f90553 
>   ql/src/test/results/clientpositive/llap/ptf_matchpath.q.out b4b249bd9b 
>   ql/src/test/results/clientpositive/llap/smb_mapjoin_4.q.out 9079538601 
>   ql/src/test/results/clientpositive/llap/smb_mapjoin_5.q.out 3c672c3da3 
>   ql/src/test/results/clientpositive/llap/stats11.q.out 387a454cb9 
>   ql/src/test/results/clientpositive/llap/subquery_multi.q.out 92786f7e80 
>   ql/src/test/results/clientpositive/llap/subquery_scalar.q.out ba5ccad500 
>   ql/src/test/results/clientpositive/llap/tez_join_result_complex.q.out 3e434751b4 
>   ql/src/test/results/clientpositive/llap/tez_nway_join.q.out 9a6393217a 
>   ql/src/test/results/clientpositive/llap/vector_windowing_navfn.q.out 6bbaeed6e2 
>   ql/src/test/results/clientpositive/nullscript.q.out 4220e6915d 
>   ql/src/test/results/clientpositive/orc_merge5.q.out e8451869f6 
>   ql/src/test/results/clientpositive/orc_merge6.q.out 5ece361bbc 
>   ql/src/test/results/clientpositive/orc_merge_incompat1.q.out fcf1c68f83 
>   ql/src/test/results/clientpositive/orc_merge_incompat2.q.out a27041fd9b 
>   ql/src/test/results/clientpositive/ptf_matchpath.q.out b7409f9827 
>   ql/src/test/results/clientpositive/smb_mapjoin_1.q.out 52314d93a5 
>   ql/src/test/results/clientpositive/smb_mapjoin_2.q.out 56a38a25af 
>   ql/src/test/results/clientpositive/smb_mapjoin_25.q.out f72c2a70d9 
>   ql/src/test/results/clientpositive/smb_mapjoin_3.q.out f33b8bbd5f 
>   ql/src/test/results/clientpositive/spark/bucketmapjoin1.q.out 724f9829db 
>   ql/src/test/results/clientpositive/spark/bucketmapjoin4.q.out 7fb61f0645 
>   ql/src/test/results/clientpositive/spark/bucketmapjoin5.q.out a9415135fb 
>   ql/src/test/results/clientpositive/spark/bucketmapjoin_negative.q.out 3e74c217e7 
>   ql/src/test/results/clientpositive/spark/bucketmapjoin_negative2.q.out a3acd5dd74 
>   ql/src/test/results/clientpositive/spark/bucketmapjoin_negative3.q.out 475b709aec 
>   ql/src/test/results/clientpositive/spark/count.q.out 8df0d68e89 
>   ql/src/test/results/clientpositive/spark/gen_udf_example_add10.q.out 6abdddc89d 
>   ql/src/test/results/clientpositive/spark/groupby10.q.out b572995b32 
>   ql/src/test/results/clientpositive/spark/join_nullsafe.q.out b8c4366506 
>   ql/src/test/results/clientpositive/spark/join_star.q.out e304225b72 
>   ql/src/test/results/clientpositive/spark/orc_merge5.q.out 1f8c869574 
>   ql/src/test/results/clientpositive/spark/orc_merge6.q.out be62faed0e 
>   ql/src/test/results/clientpositive/spark/orc_merge_incompat1.q.out 65790c4962 
>   ql/src/test/results/clientpositive/spark/orc_merge_incompat2.q.out 52973c87bb 
>   ql/src/test/results/clientpositive/spark/parallel_orderby.q.out 483e42d642 
>   ql/src/test/results/clientpositive/spark/ptf_matchpath.q.out 45b59f1e6c 
>   ql/src/test/results/clientpositive/spark/smb_mapjoin_1.q.out df29e99242 
>   ql/src/test/results/clientpositive/spark/smb_mapjoin_2.q.out f6294b5b5b 
>   ql/src/test/results/clientpositive/spark/smb_mapjoin_25.q.out c3cdff31c4 
>   ql/src/test/results/clientpositive/spark/smb_mapjoin_3.q.out 4f1353930b 
>   ql/src/test/results/clientpositive/spark/smb_mapjoin_4.q.out 432d4bcf67 
>   ql/src/test/results/clientpositive/spark/smb_mapjoin_5.q.out d001d6d8a2 
>   ql/src/test/results/clientpositive/spark/spark_explainuser_1.q.out 09af066ad0 
>   ql/src/test/results/clientpositive/subquery_in_having.q.out 627d1b4f88 
>   ql/src/test/results/clientpositive/tez/explainanalyze_3.q.out 7cb8335191 
>   ql/src/test/results/clientpositive/tez/explainanalyze_5.q.out ce943160d7 
>   ql/src/test/results/clientpositive/tez/explainuser_3.q.out d26a9a32cb 
>   ql/src/test/results/clientpositive/udf_trunc_number.q.out dfc9d700e9 
> 
> 
> Diff: https://reviews.apache.org/r/62424/diff/1/
> 
> 
> Testing
> -------
> 
> Updated existing tests.
> 
> 
> Thanks,
> 
> Vineet Garg
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message