Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id C98D7200C33 for ; Sat, 25 Feb 2017 01:49:29 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id C82B7160B79; Sat, 25 Feb 2017 00:49:29 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id EC1D2160B69 for ; Sat, 25 Feb 2017 01:49:28 +0100 (CET) Received: (qmail 73903 invoked by uid 500); 25 Feb 2017 00:49:28 -0000 Mailing-List: contact dev-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list dev@hive.apache.org Received: (qmail 73882 invoked by uid 99); 25 Feb 2017 00:49:27 -0000 Received: from reviews-vm.apache.org (HELO reviews.apache.org) (140.211.11.40) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 25 Feb 2017 00:49:27 +0000 Received: from reviews.apache.org (localhost [127.0.0.1]) by reviews.apache.org (Postfix) with ESMTP id 661B631B060; Sat, 25 Feb 2017 00:49:27 +0000 (UTC) Content-Type: multipart/alternative; boundary="===============8248116339302265271==" MIME-Version: 1.0 Subject: Re: Review Request 56810: Compute table stats when user computes column stats From: pengcheng xiong To: Ashutosh Chauhan Cc: pengcheng xiong , hive Date: Sat, 25 Feb 2017 00:49:27 -0000 Message-ID: <20170225004927.13824.60962@reviews.apache.org> X-ReviewBoard-URL: https://reviews.apache.org/ Auto-Submitted: auto-generated Sender: pengcheng xiong X-ReviewGroup: hive X-Auto-Response-Suppress: DR, RN, OOF, AutoReply X-ReviewRequest-URL: https://reviews.apache.org/r/56810/ X-Sender: pengcheng xiong References: <20170223185314.1738.8643@reviews.apache.org> In-Reply-To: <20170223185314.1738.8643@reviews.apache.org> X-ReviewBoard-Diff-For: ql/src/test/queries/clientpositive/column_table_stats.q X-ReviewBoard-Diff-For: ql/src/test/results/clientpositive/column_table_stats.q.out Reply-To: pengcheng xiong X-ReviewRequest-Repository: hive-git archived-at: Sat, 25 Feb 2017 00:49:30 -0000 --===============8248116339302265271== MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/56810/#review166791 ----------------------------------------------------------- ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java (line 10250) In collect column stats case, isAnalyzeCommand is false and analyzerewrite is not null ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java (line 10257) It is null at this point. ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java (line 10262) ok ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java (line 10320) ok ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java (lines 307 - 343) So you mean you only want this new feature to work on Tez, not other engines like MR or Spark? - pengcheng xiong On Feb. 23, 2017, 6:52 p.m., pengcheng xiong wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/56810/ > ----------------------------------------------------------- > > (Updated Feb. 23, 2017, 6:52 p.m.) > > > Review request for hive and Ashutosh Chauhan. > > > Repository: hive-git > > > Description > ------- > > HIVE-15903 > > > Diffs > ----- > > ql/src/java/org/apache/hadoop/hive/ql/parse/BaseSemanticAnalyzer.java 3e749eb > ql/src/java/org/apache/hadoop/hive/ql/parse/ColumnStatsSemanticAnalyzer.java 93b8183 > ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java 2430811 > ql/src/java/org/apache/hadoop/hive/ql/parse/TaskCompiler.java 5f9ccc8 > ql/src/test/queries/clientpositive/column_table_stats.q PRE-CREATION > ql/src/test/queries/clientpositive/exec_parallel_column_stats.q ceacc24 > ql/src/test/results/clientpositive/alter_table_update_status.q.out a3c4f1a > ql/src/test/results/clientpositive/analyze_tbl_date.q.out a0cdbca > ql/src/test/results/clientpositive/analyze_tbl_part.q.out ed90b6f > ql/src/test/results/clientpositive/annotate_stats_deep_filters.q.out 32644dc > ql/src/test/results/clientpositive/annotate_stats_filter.q.out e22c3ef > ql/src/test/results/clientpositive/annotate_stats_groupby.q.out a8e4854 > ql/src/test/results/clientpositive/annotate_stats_groupby2.q.out 31c4ed1 > ql/src/test/results/clientpositive/annotate_stats_join.q.out 5d4fe6c > ql/src/test/results/clientpositive/annotate_stats_join_pkfk.q.out b4d46d2 > ql/src/test/results/clientpositive/annotate_stats_limit.q.out ea181cb > ql/src/test/results/clientpositive/annotate_stats_select.q.out 873f1ab > ql/src/test/results/clientpositive/annotate_stats_table.q.out efc3c1f > ql/src/test/results/clientpositive/annotate_stats_union.q.out 059f261 > ql/src/test/results/clientpositive/autoColumnStats_3.q.out dca158b > ql/src/test/results/clientpositive/avro_decimal.q.out 64e65ca > ql/src/test/results/clientpositive/avro_decimal_native.q.out cebc342 > ql/src/test/results/clientpositive/cbo_rp_annotate_stats_groupby.q.out f260f03 > ql/src/test/results/clientpositive/cbo_rp_auto_join1.q.out fe4bc4f > ql/src/test/results/clientpositive/colstats_all_nulls.q.out f67f81b > ql/src/test/results/clientpositive/columnStatsUpdateForStatsOptimizer_2.q.out af21343 > ql/src/test/results/clientpositive/column_table_stats.q.out PRE-CREATION > ql/src/test/results/clientpositive/columnstats_partlvl.q.out 5c9f901 > ql/src/test/results/clientpositive/columnstats_partlvl_dp.q.out 08fb42e > ql/src/test/results/clientpositive/columnstats_quoting.q.out 52e3538 > ql/src/test/results/clientpositive/columnstats_tbllvl.q.out 60652e0 > ql/src/test/results/clientpositive/compustat_avro.q.out 50c03bd > ql/src/test/results/clientpositive/compute_stats_date.q.out 73f08ad > ql/src/test/results/clientpositive/constGby.q.out fd8ecc2 > ql/src/test/results/clientpositive/constant_prop_3.q.out ecf6e57 > ql/src/test/results/clientpositive/decimal_stats.q.out cb02f76 > ql/src/test/results/clientpositive/deleteAnalyze.q.out 4382522 > ql/src/test/results/clientpositive/display_colstats_tbllvl.q.out e10edb4 > ql/src/test/results/clientpositive/distinct_stats.q.out 73b4add > ql/src/test/results/clientpositive/drop_table_with_stats.q.out 52aa10a > ql/src/test/results/clientpositive/exec_parallel_column_stats.q.out f256ec1 > ql/src/test/results/clientpositive/extrapolate_part_stats_date.q.out b12d3a1 > ql/src/test/results/clientpositive/llap/alter_table_invalidate_column_stats.q.out 85d7dc4 > ql/src/test/results/clientpositive/llap/columnStatsUpdateForStatsOptimizer_1.q.out 80ccddd > ql/src/test/results/clientpositive/llap/columnstats_part_coltype.q.out 3e28e58 > ql/src/test/results/clientpositive/llap/deleteAnalyze.q.out ba14835 > ql/src/test/results/clientpositive/llap/drop_partition_with_stats.q.out c6ab40d > ql/src/test/results/clientpositive/llap/explainuser_2.q.out 67f960a > ql/src/test/results/clientpositive/llap/llap_stats.q.out f6921f1 > ql/src/test/results/clientpositive/llap/llapdecider.q.out d514f42 > ql/src/test/results/clientpositive/llap/metadata_only_queries.q.out c8190bd > ql/src/test/results/clientpositive/llap/schema_evol_stats.q.out be8ca4a > ql/src/test/results/clientpositive/llap/special_character_in_tabnames_1.q.out d01b373 > ql/src/test/results/clientpositive/llap/stats_only_null.q.out c905ceb > ql/src/test/results/clientpositive/llap/union_remove_26.q.out 18b9aa5 > ql/src/test/results/clientpositive/llap/vector_outer_join1.q.out 2ea8c8f > ql/src/test/results/clientpositive/llap/vector_outer_join2.q.out 2077dce > ql/src/test/results/clientpositive/llap/vector_outer_join3.q.out dbbfd34 > ql/src/test/results/clientpositive/llap/vector_outer_join4.q.out ffce9e6 > ql/src/test/results/clientpositive/llap/vector_outer_join5.q.out 4f25253 > ql/src/test/results/clientpositive/llap/vectorized_dynamic_semijoin_reduction.q.out c9eec63 > ql/src/test/results/clientpositive/llap/vectorized_dynamic_semijoin_reduction2.q.out 062fef6 > ql/src/test/results/clientpositive/metadata_only_queries.q.out 57b59dd > ql/src/test/results/clientpositive/partial_column_stats.q.out 59b52b0 > ql/src/test/results/clientpositive/pcs.q.out c422a3d > ql/src/test/results/clientpositive/remove_exprs_stats.q.out 8fe688d > ql/src/test/results/clientpositive/rename_table_update_column_stats.q.out c025cfa > ql/src/test/results/clientpositive/spark/annotate_stats_join.q.out 31ccc5c > ql/src/test/results/clientpositive/spark/avro_decimal_native.q.out cebc342 > ql/src/test/results/clientpositive/spark/limit_pushdown.q.out c81240d > ql/src/test/results/clientpositive/spark/metadata_only_queries.q.out 543d0ef > ql/src/test/results/clientpositive/spark/stats_only_null.q.out 359eea3 > ql/src/test/results/clientpositive/spark/vector_elt.q.out b49462a > ql/src/test/results/clientpositive/spark/vector_left_outer_join.q.out 91af229 > ql/src/test/results/clientpositive/spark/vector_outer_join1.q.out ca4ce15 > ql/src/test/results/clientpositive/spark/vector_outer_join2.q.out 8a9f90f > ql/src/test/results/clientpositive/spark/vector_outer_join3.q.out dbbfd34 > ql/src/test/results/clientpositive/spark/vector_outer_join4.q.out ffce9e6 > ql/src/test/results/clientpositive/spark/vector_outer_join5.q.out 4f25253 > ql/src/test/results/clientpositive/spark/vectorization_0.q.out 5086f53 > ql/src/test/results/clientpositive/spark/vectorization_13.q.out a6ef031 > ql/src/test/results/clientpositive/spark/vectorization_14.q.out 1541908 > ql/src/test/results/clientpositive/spark/vectorization_15.q.out 1d925c5 > ql/src/test/results/clientpositive/spark/vectorization_16.q.out e731c2d > ql/src/test/results/clientpositive/spark/vectorization_17.q.out a8f401b > ql/src/test/results/clientpositive/spark/vectorization_9.q.out e731c2d > ql/src/test/results/clientpositive/spark/vectorization_div0.q.out baea88f > ql/src/test/results/clientpositive/spark/vectorization_pushdown.q.out 1f1bb30 > ql/src/test/results/clientpositive/spark/vectorization_short_regress.q.out 47664ce > ql/src/test/results/clientpositive/spark/vectorized_case.q.out 67ed3dd > ql/src/test/results/clientpositive/spark/vectorized_mapjoin.q.out 030a71b > ql/src/test/results/clientpositive/spark/vectorized_math_funcs.q.out 90ef576 > ql/src/test/results/clientpositive/spark/vectorized_nested_mapjoin.q.out 996021f > ql/src/test/results/clientpositive/spark/vectorized_shufflejoin.q.out 5930057 > ql/src/test/results/clientpositive/spark/vectorized_string_funcs.q.out 26aab1c > ql/src/test/results/clientpositive/special_character_in_tabnames_2.q.out 8cb04c7 > ql/src/test/results/clientpositive/stats_invalidation.q.out a0e7663 > ql/src/test/results/clientpositive/stats_missing_warning.q.out 0ed70a0 > ql/src/test/results/clientpositive/stats_only_null.q.out 88c2114 > ql/src/test/results/clientpositive/stats_partial_size.q.out c779741 > ql/src/test/results/clientpositive/stats_ppr_all.q.out d4060c6 > ql/src/test/results/clientpositive/temp_table_display_colstats_tbllvl.q.out c7ee93d > ql/src/test/results/clientpositive/tez/explainanalyze_3.q.out dd8849d > ql/src/test/results/clientpositive/tez/explainanalyze_5.q.out ee9affb > ql/src/test/results/clientpositive/tez/explainuser_3.q.out ef71d73 > ql/src/test/results/clientpositive/vector_outer_join1.q.out 3df63dc > ql/src/test/results/clientpositive/vector_outer_join2.q.out ec7de23 > ql/src/test/results/clientpositive/vector_outer_join3.q.out 1d2abee > ql/src/test/results/clientpositive/vector_outer_join4.q.out a73a5e1 > > Diff: https://reviews.apache.org/r/56810/diff/ > > > Testing > ------- > > > Thanks, > > pengcheng xiong > > --===============8248116339302265271==--