hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From pengcheng xiong <pxi...@hortonworks.com>
Subject Re: Review Request 57307: Display aggr stats for partitioned table in describe statement.
Date Tue, 07 Mar 2017 23:32:25 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/57307/#review168213
-----------------------------------------------------------




ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java
Lines 3277 (patched)
<https://reviews.apache.org/r/57307/#comment240410>

    This will be expensive. Shall we open a new jira to consider multi-threading approach?



ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java
Lines 3304 (patched)
<https://reviews.apache.org/r/57307/#comment240411>

    Each time when desc is called here, we need to do collect the column stats for the partition
column and also do the aggregation for the non-partition column. Shall we store it in the
metastore like the other table stats? We can then invalidate them when situation changes.
I think we can open a new jira for this as well.


- pengcheng xiong


On March 7, 2017, 3:58 p.m., Ashutosh Chauhan wrote:
> 
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/57307/
> -----------------------------------------------------------
> 
> (Updated March 7, 2017, 3:58 p.m.)
> 
> 
> Review request for hive and Gunther Hagleitner.
> 
> 
> Bugs: HIVE-16098
>     https://issues.apache.org/jira/browse/HIVE-16098
> 
> 
> Repository: hive-git
> 
> 
> Description
> -------
> 
> Currently desc statement lists stats for named partition. after this change if partitions
are not specified stats are aggregated and displayed.
> 
> 
> Diffs
> -----
> 
>   ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java c11ba97 
>   ql/src/java/org/apache/hadoop/hive/ql/metadata/PartitionIterable.java 25a7b57 
>   ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatUtils.java
03f74dd 
>   ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/TextMetaDataFormatter.java
b990bda 
>   ql/src/java/org/apache/hadoop/hive/ql/stats/StatsUtils.java e48b609 
>   ql/src/test/queries/clientpositive/describe_table.q c388207 
>   ql/src/test/queries/clientpositive/partition_coltype_literals.q 0c2365a 
>   ql/src/test/results/clientnegative/alter_view_as_select_with_partition.q.out 9e07416

>   ql/src/test/results/clientpositive/acid_table_stats.q.out 5d8a000 
>   ql/src/test/results/clientpositive/alter_numbuckets_partitioned_table2_h23.q.out 53ed194

>   ql/src/test/results/clientpositive/alter_numbuckets_partitioned_table_h23.q.out 578bad7

>   ql/src/test/results/clientpositive/alter_partition_clusterby_sortby.q.out 29fbe6f 
>   ql/src/test/results/clientpositive/alter_table_add_partition.q.out d459f3b 
>   ql/src/test/results/clientpositive/alter_table_serde2.q.out a1e1fb2 
>   ql/src/test/results/clientpositive/alter_table_update_status.q.out a3c4f1a 
>   ql/src/test/results/clientpositive/ambiguitycheck.q.out 3574573 
>   ql/src/test/results/clientpositive/autoColumnStats_3.q.out dca158b 
>   ql/src/test/results/clientpositive/autoColumnStats_9.q.out a90d180 
>   ql/src/test/results/clientpositive/avro_decimal.q.out 64e65ca 
>   ql/src/test/results/clientpositive/avro_decimal_native.q.out cebc342 
>   ql/src/test/results/clientpositive/avro_schema_evolution_native.q.out c92c751 
>   ql/src/test/results/clientpositive/beeline/escape_comments.q.out 0cbc8d6 
>   ql/src/test/results/clientpositive/colstats_all_nulls.q.out f67f81b 
>   ql/src/test/results/clientpositive/columnstats_partlvl.q.out 5c9f901 
>   ql/src/test/results/clientpositive/columnstats_tbllvl.q.out 60652e0 
>   ql/src/test/results/clientpositive/compustat_avro.q.out 50c03bd 
>   ql/src/test/results/clientpositive/compute_stats_date.q.out 73f08ad 
>   ql/src/test/results/clientpositive/confirm_initial_tbl_stats.q.out f0d8ff2 
>   ql/src/test/results/clientpositive/create_like_view.q.out 9e6dcd1 
>   ql/src/test/results/clientpositive/create_or_replace_view.q.out 98e0ca2 
>   ql/src/test/results/clientpositive/create_table_like_stats.q.out 97ee25a 
>   ql/src/test/results/clientpositive/create_view_partitioned.q.out 46a81c4 
>   ql/src/test/results/clientpositive/decimal_stats.q.out cb02f76 
>   ql/src/test/results/clientpositive/describe_syntax.q.out 34f0816 
>   ql/src/test/results/clientpositive/describe_table.q.out ad3bfc1 
>   ql/src/test/results/clientpositive/describe_xpath.q.out 4d40fc2 
>   ql/src/test/results/clientpositive/display_colstats_tbllvl.q.out e10edb4 
>   ql/src/test/results/clientpositive/escape_comments.q.out 111c103 
>   ql/src/test/results/clientpositive/exim_hidden_files.q.out 9b04a6d 
>   ql/src/test/results/clientpositive/llap/alter_table_invalidate_column_stats.q.out 85d7dc4

>   ql/src/test/results/clientpositive/llap/autoColumnStats_1.q.out 712beba 
>   ql/src/test/results/clientpositive/llap/autoColumnStats_2.q.out 9bff427 
>   ql/src/test/results/clientpositive/llap/orc_create.q.out 1ae93c6 
>   ql/src/test/results/clientpositive/llap/schema_evol_stats.q.out be8ca4a 
>   ql/src/test/results/clientpositive/llap/stats_noscan_1.q.out 2fba3e2 
>   ql/src/test/results/clientpositive/parquet_mixed_partition_formats.q.out 20fcb93 
>   ql/src/test/results/clientpositive/parquet_partitioned.q.out 7eace7d 
>   ql/src/test/results/clientpositive/parquet_serde.q.out 43c9bd0 
>   ql/src/test/results/clientpositive/partial_column_stats.q.out 59b52b0 
>   ql/src/test/results/clientpositive/partition_coltype_literals.q.out fad937d 
>   ql/src/test/results/clientpositive/rename_table_update_column_stats.q.out c025cfa 
>   ql/src/test/results/clientpositive/spark/avro_decimal_native.q.out cebc342 
>   ql/src/test/results/clientpositive/spark/stats10.q.out a0c5d40 
>   ql/src/test/results/clientpositive/spark/stats12.q.out f7cdab9 
>   ql/src/test/results/clientpositive/spark/stats13.q.out 304a488 
>   ql/src/test/results/clientpositive/spark/stats14.q.out e8fe776 
>   ql/src/test/results/clientpositive/spark/stats15.q.out e8fe776 
>   ql/src/test/results/clientpositive/spark/stats2.q.out 404e34b 
>   ql/src/test/results/clientpositive/spark/stats3.q.out 14eaf78 
>   ql/src/test/results/clientpositive/spark/stats6.q.out f2a756f 
>   ql/src/test/results/clientpositive/spark/stats7.q.out 3df7f06 
>   ql/src/test/results/clientpositive/spark/stats8.q.out dc8b5c1 
>   ql/src/test/results/clientpositive/spark/stats_noscan_1.q.out 9e1652b 
>   ql/src/test/results/clientpositive/spark/union_remove_15.q.out 658edea 
>   ql/src/test/results/clientpositive/spark/union_remove_16.q.out 6ddc9e2 
>   ql/src/test/results/clientpositive/spark/union_remove_17.q.out 3f16384 
>   ql/src/test/results/clientpositive/spark/union_remove_18.q.out 87d3eca 
>   ql/src/test/results/clientpositive/spark/vector_between_in.q.out 4de2eb6 
>   ql/src/test/results/clientpositive/stats10.q.out d7519af 
>   ql/src/test/results/clientpositive/stats12.q.out 9d13315 
>   ql/src/test/results/clientpositive/stats13.q.out cb90c35 
>   ql/src/test/results/clientpositive/stats14.q.out e8fe776 
>   ql/src/test/results/clientpositive/stats15.q.out e8fe776 
>   ql/src/test/results/clientpositive/stats2.q.out ac1d5cb 
>   ql/src/test/results/clientpositive/stats3.q.out 14eaf78 
>   ql/src/test/results/clientpositive/stats4.q.out 6105017 
>   ql/src/test/results/clientpositive/stats6.q.out f2a756f 
>   ql/src/test/results/clientpositive/stats7.q.out 86adcea 
>   ql/src/test/results/clientpositive/stats8.q.out 7c8ce8c 
>   ql/src/test/results/clientpositive/stats_noscan_1.q.out 9e1652b 
>   ql/src/test/results/clientpositive/unicode_comments.q.out ae50c67 
>   ql/src/test/results/clientpositive/union_remove_15.q.out 8109837 
>   ql/src/test/results/clientpositive/union_remove_16.q.out e8f3e47 
>   ql/src/test/results/clientpositive/union_remove_17.q.out e1c1692 
>   ql/src/test/results/clientpositive/union_remove_18.q.out 8c3ccb4 
> 
> 
> Diff: https://reviews.apache.org/r/57307/diff/3/
> 
> 
> Testing
> -------
> 
> Regression suite.
> 
> 
> Thanks,
> 
> Ashutosh Chauhan
> 
>


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message