hive-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "chengxiang li" <chengxiang...@intel.com>
Subject Review Request 27327: HIVE-8539 enable table statistics collection based on Spark Counter[Spark Branch]
Date Wed, 29 Oct 2014 13:34:49 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/27327/
-----------------------------------------------------------

Review request for hive, Rui Li and Xuefu Zhang.


Bugs: HIVE-8539
    https://issues.apache.org/jira/browse/HIVE-8539


Repository: hive-git


Description
-------

check the detail info in design doc.


Diffs
-----

  common/src/java/org/apache/hadoop/hive/common/StatsSetupConst.java cd4beeb 
  data/conf/spark/hive-site.xml df1dc9c 
  ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkClient.java ccf2038 
  ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkMapRecordHandler.java 0bb554a 
  ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkRecordHandler.java ec51864 
  ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkTask.java e954153 
  ql/src/java/org/apache/hadoop/hive/ql/exec/spark/SparkUtilities.java 37761c9 
  ql/src/java/org/apache/hadoop/hive/ql/exec/spark/counter/SparkCounters.java 447d964 
  ql/src/java/org/apache/hadoop/hive/ql/exec/spark/session/SparkSession.java 02456fc 
  ql/src/java/org/apache/hadoop/hive/ql/exec/spark/session/SparkSessionImpl.java eefd4be 
  ql/src/java/org/apache/hadoop/hive/ql/exec/spark/status/SparkJobRef.java PRE-CREATION 
  ql/src/java/org/apache/hadoop/hive/ql/exec/spark/status/SparkJobStatus.java 25cb60f 
  ql/src/java/org/apache/hadoop/hive/ql/exec/spark/status/impl/SimpleSparkJobStatus.java 6570b0c

  ql/src/java/org/apache/hadoop/hive/ql/plan/SparkWork.java 26428e8 
  ql/src/java/org/apache/hadoop/hive/ql/stats/CounterStatsAggregatorSpark.java PRE-CREATION

  ql/src/java/org/apache/hadoop/hive/ql/stats/CounterStatsPublisher.java 0b648aa 
  ql/src/test/results/clientpositive/spark/add_part_multiple.q.out 264a08d 
  ql/src/test/results/clientpositive/spark/alter_merge_orc.q.out 0fe395c 
  ql/src/test/results/clientpositive/spark/alter_merge_stats_orc.q.out 3e64d36 
  ql/src/test/results/clientpositive/spark/auto_smb_mapjoin_14.q.out e64d4fb 
  ql/src/test/results/clientpositive/spark/avro_decimal_native.q.out db5b0f2 
  ql/src/test/results/clientpositive/spark/bucket2.q.out b1b2997 
  ql/src/test/results/clientpositive/spark/bucket3.q.out 019c11a 
  ql/src/test/results/clientpositive/spark/bucket4.q.out 2cbab11 
  ql/src/test/results/clientpositive/spark/column_access_stats.q.out 7ffe3e3 
  ql/src/test/results/clientpositive/spark/create_merge_compressed.q.out 7c27c6a 
  ql/src/test/results/clientpositive/spark/custom_input_output_format.q.out d7b763a 
  ql/src/test/results/clientpositive/spark/date_udf.q.out 4018826 
  ql/src/test/results/clientpositive/spark/disable_merge_for_bucketing.q.out 590b265 
  ql/src/test/results/clientpositive/spark/enforce_order.q.out 95fd12a 
  ql/src/test/results/clientpositive/spark/groupby1.q.out 847f45c 
  ql/src/test/results/clientpositive/spark/groupby10.q.out 2095843 
  ql/src/test/results/clientpositive/spark/groupby11.q.out 70db5a5 
  ql/src/test/results/clientpositive/spark/groupby2.q.out 86e2f2a 
  ql/src/test/results/clientpositive/spark/groupby3.q.out 13a5fab 
  ql/src/test/results/clientpositive/spark/groupby3_map.q.out dac2824 
  ql/src/test/results/clientpositive/spark/groupby3_map_multi_distinct.q.out d2c054a 
  ql/src/test/results/clientpositive/spark/groupby3_map_skew.q.out ec6439a 
  ql/src/test/results/clientpositive/spark/groupby3_noskew.q.out 0c9a7e1 
  ql/src/test/results/clientpositive/spark/groupby3_noskew_multi_distinct.q.out 42fbb8c 
  ql/src/test/results/clientpositive/spark/groupby4.q.out 318c5a3 
  ql/src/test/results/clientpositive/spark/groupby7.q.out 01af804 
  ql/src/test/results/clientpositive/spark/groupby7_map.q.out 22a05b5 
  ql/src/test/results/clientpositive/spark/groupby7_map_multi_single_reducer.q.out bc453c6

  ql/src/test/results/clientpositive/spark/groupby7_map_skew.q.out 2a07f2a 
  ql/src/test/results/clientpositive/spark/groupby7_noskew.q.out 00a0707 
  ql/src/test/results/clientpositive/spark/groupby7_noskew_multi_single_reducer.q.out 36640ef

  ql/src/test/results/clientpositive/spark/groupby8.q.out d8295ce 
  ql/src/test/results/clientpositive/spark/groupby8_map.q.out b9aa597 
  ql/src/test/results/clientpositive/spark/groupby8_map_skew.q.out b9aa597 
  ql/src/test/results/clientpositive/spark/groupby8_noskew.q.out b9aa597 
  ql/src/test/results/clientpositive/spark/groupby9.q.out bec2346 
  ql/src/test/results/clientpositive/spark/groupby_complex_types.q.out 16fadea 
  ql/src/test/results/clientpositive/spark/groupby_complex_types_multi_single_reducer.q.out
7470843 
  ql/src/test/results/clientpositive/spark/groupby_cube1.q.out 169c4ac 
  ql/src/test/results/clientpositive/spark/groupby_multi_insert_common_distinct.q.out d3457da

  ql/src/test/results/clientpositive/spark/groupby_multi_single_reducer.q.out 3abd0e3 
  ql/src/test/results/clientpositive/spark/groupby_multi_single_reducer2.q.out 7f74c62 
  ql/src/test/results/clientpositive/spark/groupby_multi_single_reducer3.q.out c4b7419 
  ql/src/test/results/clientpositive/spark/groupby_position.q.out 9e58189 
  ql/src/test/results/clientpositive/spark/groupby_ppr.q.out 860aa58 
  ql/src/test/results/clientpositive/spark/groupby_rollup1.q.out 0aeff6b 
  ql/src/test/results/clientpositive/spark/groupby_sort_1_23.q.out 61dd2be 
  ql/src/test/results/clientpositive/spark/groupby_sort_skew_1_23.q.out 99da734 
  ql/src/test/results/clientpositive/spark/innerjoin.q.out 5e1a2aa 
  ql/src/test/results/clientpositive/spark/input12.q.out 8ef35c4 
  ql/src/test/results/clientpositive/spark/input14.q.out e7d4db6 
  ql/src/test/results/clientpositive/spark/input17.q.out 0882a29 
  ql/src/test/results/clientpositive/spark/input18.q.out 802fb0a 
  ql/src/test/results/clientpositive/spark/input1_limit.q.out 33ecd07 
  ql/src/test/results/clientpositive/spark/input_part2.q.out b9e85fe 
  ql/src/test/results/clientpositive/spark/insert1.q.out 49fb1d4 
  ql/src/test/results/clientpositive/spark/insert_into1.q.out e9be658 
  ql/src/test/results/clientpositive/spark/insert_into2.q.out 5c8e9c7 
  ql/src/test/results/clientpositive/spark/insert_into3.q.out 6c0111d 
  ql/src/test/results/clientpositive/spark/join1.q.out da63bee 
  ql/src/test/results/clientpositive/spark/join14.q.out f44c5dd 
  ql/src/test/results/clientpositive/spark/join17.q.out 6d68ed3 
  ql/src/test/results/clientpositive/spark/join2.q.out 56e0fe9 
  ql/src/test/results/clientpositive/spark/join24.q.out 0de60d0 
  ql/src/test/results/clientpositive/spark/join25.q.out f2c8457 
  ql/src/test/results/clientpositive/spark/join26.q.out 3b010a0 
  ql/src/test/results/clientpositive/spark/join27.q.out 2e1f61a 
  ql/src/test/results/clientpositive/spark/join3.q.out e61f724 
  ql/src/test/results/clientpositive/spark/join4.q.out 1d48393 
  ql/src/test/results/clientpositive/spark/join5.q.out 247750e 
  ql/src/test/results/clientpositive/spark/join6.q.out 966fcf2 
  ql/src/test/results/clientpositive/spark/join7.q.out d7e8bd1 
  ql/src/test/results/clientpositive/spark/join8.q.out 31115c3 
  ql/src/test/results/clientpositive/spark/join9.q.out 5553ad2 
  ql/src/test/results/clientpositive/spark/join_nullsafe.q.out c8d4413 
  ql/src/test/results/clientpositive/spark/load_dyn_part1.q.out 5d2622d 
  ql/src/test/results/clientpositive/spark/load_dyn_part10.q.out 9b70e4c 
  ql/src/test/results/clientpositive/spark/load_dyn_part11.q.out aa77469 
  ql/src/test/results/clientpositive/spark/load_dyn_part12.q.out 0be0c69 
  ql/src/test/results/clientpositive/spark/load_dyn_part13.q.out eb3e01a 
  ql/src/test/results/clientpositive/spark/load_dyn_part14.q.out 5dd5fad 
  ql/src/test/results/clientpositive/spark/load_dyn_part15.q.out d85fd69 
  ql/src/test/results/clientpositive/spark/load_dyn_part2.q.out f8f8971 
  ql/src/test/results/clientpositive/spark/load_dyn_part3.q.out b6ae54c 
  ql/src/test/results/clientpositive/spark/load_dyn_part4.q.out d8c1412 
  ql/src/test/results/clientpositive/spark/load_dyn_part5.q.out b5c83cb 
  ql/src/test/results/clientpositive/spark/load_dyn_part6.q.out f23dbe5 
  ql/src/test/results/clientpositive/spark/load_dyn_part7.q.out 6a12103 
  ql/src/test/results/clientpositive/spark/load_dyn_part8.q.out a12c75c 
  ql/src/test/results/clientpositive/spark/load_dyn_part9.q.out d158180 
  ql/src/test/results/clientpositive/spark/mapreduce1.q.out 1824126 
  ql/src/test/results/clientpositive/spark/mapreduce2.q.out 792a0c8 
  ql/src/test/results/clientpositive/spark/merge1.q.out c50a80b 
  ql/src/test/results/clientpositive/spark/merge2.q.out aec97a3 
  ql/src/test/results/clientpositive/spark/multi_insert.q.out 2b9f90e 
  ql/src/test/results/clientpositive/spark/multi_insert_gby.q.out 7d6d58b 
  ql/src/test/results/clientpositive/spark/multi_insert_gby2.q.out fca3e1d 
  ql/src/test/results/clientpositive/spark/multi_insert_gby3.q.out ce78fba 
  ql/src/test/results/clientpositive/spark/multi_insert_mixed.q.out 819b265 
  ql/src/test/results/clientpositive/spark/optimize_nullscan.q.out d9de8d9 
  ql/src/test/results/clientpositive/spark/parallel.q.out 32d7ff1 
  ql/src/test/results/clientpositive/spark/parallel_join1.q.out 8ba3abb 
  ql/src/test/results/clientpositive/spark/pcr.q.out 4e9244f 
  ql/src/test/results/clientpositive/spark/sample1.q.out e6c5e62 
  ql/src/test/results/clientpositive/spark/sample10.q.out c511152 
  ql/src/test/results/clientpositive/spark/sample2.q.out 2b4fcd6 
  ql/src/test/results/clientpositive/spark/sample4.q.out 6b12cd6 
  ql/src/test/results/clientpositive/spark/sample5.q.out a605c92 
  ql/src/test/results/clientpositive/spark/sample6.q.out f6256f5 
  ql/src/test/results/clientpositive/spark/sample7.q.out f99d898 
  ql/src/test/results/clientpositive/spark/scriptfile1.q.out 5d91e90 
  ql/src/test/results/clientpositive/spark/skewjoin.q.out d674d04 
  ql/src/test/results/clientpositive/spark/smb_mapjoin_13.q.out 0319137 
  ql/src/test/results/clientpositive/spark/smb_mapjoin_15.q.out 7849e78 
  ql/src/test/results/clientpositive/spark/smb_mapjoin_16.q.out 11ffefd 
  ql/src/test/results/clientpositive/spark/subquery_multiinsert.q.out 2bedd37 
  ql/src/test/results/clientpositive/spark/table_access_keys_stats.q.out 844d36f 
  ql/src/test/results/clientpositive/spark/timestamp_1.q.out 2d48500 
  ql/src/test/results/clientpositive/spark/timestamp_2.q.out f41ecbe 
  ql/src/test/results/clientpositive/spark/timestamp_3.q.out 7bb6b19 
  ql/src/test/results/clientpositive/spark/timestamp_lazy.q.out 640ca6b 
  ql/src/test/results/clientpositive/spark/timestamp_udf.q.out 4cfca4c 
  ql/src/test/results/clientpositive/spark/transform1.q.out a657d01 
  ql/src/test/results/clientpositive/spark/union10.q.out 59ebb0c 
  ql/src/test/results/clientpositive/spark/union18.q.out f1c69bf 
  ql/src/test/results/clientpositive/spark/union19.q.out c86afb0 
  ql/src/test/results/clientpositive/spark/union25.q.out 07ba875 
  ql/src/test/results/clientpositive/spark/union28.q.out f668ff8 
  ql/src/test/results/clientpositive/spark/union29.q.out f82d561 
  ql/src/test/results/clientpositive/spark/union3.q.out ba21367 
  ql/src/test/results/clientpositive/spark/union30.q.out ee0daf4 
  ql/src/test/results/clientpositive/spark/union33.q.out ca08e0c 
  ql/src/test/results/clientpositive/spark/union4.q.out 2e46204 
  ql/src/test/results/clientpositive/spark/union6.q.out 01f044e 
  ql/src/test/results/clientpositive/spark/vector_cast_constant.q.out 2dd7aab 
  ql/src/test/results/clientpositive/spark/vector_char_4.q.out f0d0a1a 
  ql/src/test/results/clientpositive/spark/vector_count_distinct.q.out 8b6a226 
  ql/src/test/results/clientpositive/spark/vector_data_types.q.out 5758c4b 
  ql/src/test/results/clientpositive/spark/vector_orderby_5.q.out 3b89885 
  ql/src/test/results/clientpositive/spark/vector_string_concat.q.out b45f042 
  ql/src/test/results/clientpositive/spark/vector_varchar_4.q.out 37be6bc 
  ql/src/test/results/clientpositive/spark/vectorization_part.q.out 44e43c1 
  ql/src/test/results/clientpositive/spark/vectorization_part_project.q.out aa87dd9 
  ql/src/test/results/clientpositive/spark/vectorized_bucketmapjoin1.q.out 17d8235 
  ql/src/test/results/clientpositive/spark/vectorized_ptf.q.out 1d11b30 
  ql/src/test/results/clientpositive/spark/vectorized_rcfile_columnar.q.out 984d904 
  ql/src/test/results/clientpositive/spark/vectorized_timestamp_funcs.q.out cd43197 
  ql/src/test/results/clientpositive/spark/windowing.q.out 424c444 

Diff: https://reviews.apache.org/r/27327/diff/


Testing
-------


Thanks,

chengxiang li


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message