Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 6F6FD200C01 for ; Thu, 19 Jan 2017 16:49:34 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 6E08D160B54; Thu, 19 Jan 2017 15:49:34 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 8DCBD160B42 for ; Thu, 19 Jan 2017 16:49:33 +0100 (CET) Received: (qmail 30841 invoked by uid 500); 19 Jan 2017 15:49:32 -0000 Mailing-List: contact issues-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list issues@hive.apache.org Received: (qmail 30832 invoked by uid 99); 19 Jan 2017 15:49:32 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd3-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 19 Jan 2017 15:49:32 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd3-us-west.apache.org (ASF Mail Server at spamd3-us-west.apache.org) with ESMTP id 5EB1E185D86 for ; Thu, 19 Jan 2017 15:49:32 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd3-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 0.002 X-Spam-Level: X-Spam-Status: No, score=0.002 tagged_above=-999 required=6.31 tests=[KAM_BADIPHTTP=2, KAM_LAZY_DOMAIN_SECURITY=1, NORMAL_HTTP_TO_IP=0.001, RP_MATCHES_RCVD=-2.999] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd3-us-west.apache.org [10.40.0.10]) (amavisd-new, port 10024) with ESMTP id lM-SWMqJRDW4 for ; Thu, 19 Jan 2017 15:49:30 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id 8C7195F47E for ; Thu, 19 Jan 2017 15:49:29 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 0B2CDE8683 for ; Thu, 19 Jan 2017 15:49:28 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 0D80F24E02 for ; Thu, 19 Jan 2017 15:49:27 +0000 (UTC) Date: Thu, 19 Jan 2017 15:49:27 +0000 (UTC) From: "Hive QA (JIRA)" To: issues@hive.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HIVE-15489) Alternatively use table scan stats for HoS MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Thu, 19 Jan 2017 15:49:34 -0000 [ https://issues.apache.org/jira/browse/HIVE-15489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15830130#comment-15830130 ] Hive QA commented on HIVE-15489: -------------------------------- Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12848121/HIVE-15489.1.patch {color:red}ERROR:{color} -1 due to no test(s) being added or modified. {color:red}ERROR:{color} -1 due to 80 failed/errored test(s), 10946 tests executed *Failed tests:* {noformat} TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out) (batchId=234) org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_joins] (batchId=218) org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_predicate_pushdown] (batchId=218) org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_single_sourced_multi_insert] (batchId=218) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[ctas] (batchId=230) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_into_dynamic_partitions] (batchId=230) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_into_table] (batchId=230) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_directory] (batchId=230) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions] (batchId=230) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_table] (batchId=230) org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[write_final_output_blobstore] (batchId=230) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_table_stats] (batchId=48) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[analyze_tbl_part] (batchId=44) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input19] (batchId=79) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[insert_overwrite_directory] (batchId=25) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample5] (batchId=52) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[serde_opencsv] (batchId=68) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[specialChar] (batchId=22) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorized_math_funcs] (batchId=19) org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver[dboutput] (batchId=221) org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver[fileformat_base64] (batchId=221) org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver[udf_row_sequence] (batchId=221) org.apache.hadoop.hive.cli.TestContribCliDriver.testCliDriver[url_hook] (batchId=221) org.apache.hadoop.hive.cli.TestContribNegativeCliDriver.testCliDriver[case_with_row_sequence] (batchId=224) org.apache.hadoop.hive.cli.TestContribNegativeCliDriver.testCliDriver[invalid_row_sequence] (batchId=224) org.apache.hadoop.hive.cli.TestContribNegativeCliDriver.testCliDriver[serde_regex] (batchId=224) org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_insert_partition_dynamic] (batchId=158) org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_insert_partition_static] (batchId=156) org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_insert_values] (batchId=157) org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_unencrypted_tbl] (batchId=159) org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys] (batchId=159) org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_load_data_to_encrypted_tables] (batchId=157) org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl] (batchId=157) org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_select_read_only_encrypted_tbl] (batchId=159) org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_select_read_only_unencrypted_tbl] (batchId=159) org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_unencrypted_nonhdfs_external_tables] (batchId=157) org.apache.hadoop.hive.cli.TestHBaseNegativeCliDriver.testCliDriver[cascade_dbdrop] (batchId=225) org.apache.hadoop.hive.cli.TestHBaseNegativeCliDriver.testCliDriver[generatehfiles_require_family_path] (batchId=225) org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_llap_counters] (batchId=137) org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_basic] (batchId=135) org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_schema_evol_3a] (batchId=136) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[escape1] (batchId=139) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[escape2] (batchId=154) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[offset_limit_ppd_optimizer] (batchId=151) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_text_vec_part] (batchId=149) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_char_simple] (batchId=147) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_decimal_expressions] (batchId=148) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_math_funcs] (batchId=142) org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_4] (batchId=94) org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_5] (batchId=93) org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[hybridgrace_hashjoin_1] (batchId=93) org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[hybridgrace_hashjoin_2] (batchId=93) org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[orc_vectorization_ppd] (batchId=93) org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[tez_union_with_udf] (batchId=94) org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[unionDistinct_2] (batchId=93) org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[update_orig_table] (batchId=93) org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[vectorization_div0] (batchId=94) org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[vectorization_limit] (batchId=93) org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver[bucket_num_reducers2] (batchId=84) org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver[bucket_num_reducers] (batchId=84) org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver[index_bitmap3] (batchId=83) org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver[index_bitmap_auto] (batchId=83) org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver[infer_bucket_sort_dyn_part] (batchId=83) org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver[infer_bucket_sort_map_operators] (batchId=83) org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver[infer_bucket_sort_merge] (batchId=84) org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver[infer_bucket_sort_num_buckets] (batchId=83) org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver[infer_bucket_sort_reducers_power_two] (batchId=83) org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver[parallel_orderby] (batchId=83) org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver[scriptfile1] (batchId=84) org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[fs_default_name1] (batchId=85) org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[fs_default_name2] (batchId=86) org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[cluster_tasklog_retrieval] (batchId=87) org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[minimr_broken_pipe] (batchId=87) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query33] (batchId=222) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query60] (batchId=222) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query83] (batchId=222) org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver (batchId=126) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[sample5] (batchId=118) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vectorized_math_funcs] (batchId=104) org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1 (batchId=205) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3041/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3041/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3041/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 80 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12848121 - PreCommit-HIVE-Build > Alternatively use table scan stats for HoS > ------------------------------------------ > > Key: HIVE-15489 > URL: https://issues.apache.org/jira/browse/HIVE-15489 > Project: Hive > Issue Type: Improvement > Components: Spark, Statistics > Affects Versions: 2.2.0 > Reporter: Chao Sun > Assignee: Chao Sun > Attachments: HIVE-15489.1.patch, HIVE-15489.wip.patch > > > For MapJoin in HoS, we should provide an option to only use stats in the TS rather than the populated stats in each of the join branch. This could be pretty conservative but more reliable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)