Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id C9574200C6E for ; Mon, 8 May 2017 16:20:12 +0200 (CEST) Received: by cust-asf.ponee.io (Postfix) id C7F06160B99; Mon, 8 May 2017 14:20:12 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id C458F160BA5 for ; Mon, 8 May 2017 16:20:11 +0200 (CEST) Received: (qmail 35998 invoked by uid 500); 8 May 2017 14:20:10 -0000 Mailing-List: contact issues-help@hive.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hive.apache.org Delivered-To: mailing list issues@hive.apache.org Received: (qmail 35877 invoked by uid 99); 8 May 2017 14:20:10 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd4-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 08 May 2017 14:20:10 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd4-us-west.apache.org (ASF Mail Server at spamd4-us-west.apache.org) with ESMTP id D3E36C047C for ; Mon, 8 May 2017 14:20:09 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd4-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -98.001 X-Spam-Level: X-Spam-Status: No, score=-98.001 tagged_above=-999 required=6.31 tests=[KAM_BADIPHTTP=2, NORMAL_HTTP_TO_IP=0.001, RP_MATCHES_RCVD=-0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd4-us-west.apache.org [10.40.0.11]) (amavisd-new, port 10024) with ESMTP id mrwvPeqODL5R for ; Mon, 8 May 2017 14:20:07 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id ACF095F24C for ; Mon, 8 May 2017 14:20:06 +0000 (UTC) Received: from jira-lw-us.apache.org (unknown [207.244.88.139]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 0AE23E06C6 for ; Mon, 8 May 2017 14:20:05 +0000 (UTC) Received: from jira-lw-us.apache.org (localhost [127.0.0.1]) by jira-lw-us.apache.org (ASF Mail Server at jira-lw-us.apache.org) with ESMTP id 1B3C321DF5 for ; Mon, 8 May 2017 14:20:05 +0000 (UTC) Date: Mon, 8 May 2017 14:20:05 +0000 (UTC) From: "Hive QA (JIRA)" To: issues@hive.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (HIVE-16602) Implement shared scans with Tez MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 archived-at: Mon, 08 May 2017 14:20:13 -0000 [ https://issues.apache.org/jira/browse/HIVE-16602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16000794#comment-16000794 ] Hive QA commented on HIVE-16602: -------------------------------- Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12866868/HIVE-16602.patch {color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 86 failed/errored test(s), 10564 tests executed *Failed tests:* {noformat} TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=147) [list_bucket_dml_10.q,mapjoin_emit_interval.q,acid_globallimit.q,delete_where_no_match.q,semijoin_hint.q,vector_groupby_grouping_sets4.q,tez_vector_dynpart_hashjoin_2.q,vectorization_part.q,delete_all_non_partitioned.q,delete_all_partitioned.q,auto_join30.q,dynamic_partition_pruning.q,cluster.q,bucketmapjoin7.q,cbo_rp_unionDistinct_2.q,subquery_nested_subquery.q,limit_pushdown3.q,vector_outer_join2.q,smb_mapjoin_18.q,vector_varchar_4.q,ppd_union_view.q,union6.q,vector_decimal_4.q,schema_evol_orc_acid_part.q,cbo_subq_in.q,cross_join.q,parquet_predicate_pushdown.q,llap_vector_nohybridgrace.q,tez_smb_main.q,vectorized_timestamp_funcs.q] TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=148) [vector_auto_smb_mapjoin_14.q,deleteAnalyze.q,multi_insert.q,tez_dml.q,database.q,vector_bround.q,orc_ppd_schema_evol_1b.q,vector_join30.q,smb_mapjoin_6.q,vector_groupby_grouping_sets2.q,cte_3.q,vector_reduce_groupby_decimal.q,vectorized_dynamic_partition_pruning.q,cbo_views.q,vector_ptf_part_simple.q,vector_decimal_cast.q,groupby_grouping_id2.q,tez_smb_empty.q,orc_merge6.q,cte_mat_1.q,vector_char_mapjoin1.q,cte_5.q,subquery_shared_alias.q,vector_decimal_2.q,columnStatsUpdateForStatsOptimizer_1.q,vector_outer_join3.q,vector_string_concat.q,vectorized_context.q,metadata_only_queries.q,auto_sortmerge_join_12.q] TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=155) [vector_interval_2.q,schema_evol_orc_acid_part_update.q,orc_ppd_varchar.q,metadataonly1.q,join_nullsafe.q,vectorized_shufflejoin.q,limit_pushdown.q,auto_join_nulls.q,metadata_only_queries_with_filters.q,vector_inner_join.q,subquery_notin.q,schema_evol_text_nonvec_part_all_complex.q,vector_coalesce_2.q,vector_between_columns.q,vector_groupby_grouping_sets6.q,table_access_keys_stats.q,udaf_collect_set_2.q,subquery_null_agg.q,update_after_multiple_inserts.q,offset_limit_ppd_optimizer.q,mapjoin_decimal.q,orc_merge_incompat1.q,columnstats_part_coltype.q,vectorized_parquet_types.q,vector_char_2.q,stats_based_fetch_decision.q,auto_sortmerge_join_10.q,extrapolate_part_stats_partial_ndv.q,auto_sortmerge_join_9.q,vector_groupby_grouping_id2.q] org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[cte_2] (batchId=139) org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[except_distinct] (batchId=140) org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[explainuser_2] (batchId=140) org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[intersect_all] (batchId=139) org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[intersect_distinct] (batchId=140) org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[intersect_merge] (batchId=138) org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_nullscan] (batchId=138) org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[tez_union_dynamic_partition] (batchId=139) org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1] (batchId=139) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_join0] (batchId=158) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_join_filters] (batchId=153) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_smb_mapjoin_14] (batchId=153) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1] (batchId=158) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[column_access_stats] (batchId=153) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[correlationoptimizer2] (batchId=151) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[correlationoptimizer3] (batchId=158) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[correlationoptimizer6] (batchId=151) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[explainuser_1] (batchId=149) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[join46] (batchId=142) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[limit_join_transpose] (batchId=146) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[mapjoin46] (batchId=153) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[mrr] (batchId=144) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[multiMapJoin1] (batchId=151) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[multiMapJoin2] (batchId=156) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[orc_ppd_date] (batchId=150) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[orc_ppd_decimal] (batchId=143) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[orc_ppd_timestamp] (batchId=157) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[ppr_pushdown] (batchId=150) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_in] (batchId=154) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_multi] (batchId=145) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_scalar] (batchId=149) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_select] (batchId=149) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_views] (batchId=144) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_join_tests] (batchId=157) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_joins_explain] (batchId=149) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union8] (batchId=153) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_stats] (batchId=146) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_top_level] (batchId=154) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_groupby_mapjoin] (batchId=156) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr] (batchId=143) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_join_filters] (batchId=154) org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_1] (batchId=96) org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2] (batchId=97) org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_5] (batchId=96) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] (batchId=230) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query16] (batchId=230) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query17] (batchId=230) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query1] (batchId=230) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23] (batchId=230) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query25] (batchId=230) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query29] (batchId=230) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query30] (batchId=230) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query31] (batchId=230) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query32] (batchId=230) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query33] (batchId=230) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query38] (batchId=230) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query39] (batchId=230) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query46] (batchId=230) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query51] (batchId=230) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query56] (batchId=230) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query58] (batchId=230) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query5] (batchId=230) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query60] (batchId=230) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query64] (batchId=230) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query65] (batchId=230) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query66] (batchId=230) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query68] (batchId=230) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query69] (batchId=230) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query6] (batchId=230) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query70] (batchId=230) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query75] (batchId=230) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query76] (batchId=230) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query80] (batchId=230) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query81] (batchId=230) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query83] (batchId=230) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query85] (batchId=230) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query87] (batchId=230) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query90] (batchId=230) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query92] (batchId=230) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query95] (batchId=230) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query97] (batchId=230) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query9] (batchId=230) org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgressParallel (batchId=218) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/5111/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/5111/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-5111/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 86 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12866868 - PreCommit-HIVE-Build > Implement shared scans with Tez > ------------------------------- > > Key: HIVE-16602 > URL: https://issues.apache.org/jira/browse/HIVE-16602 > Project: Hive > Issue Type: New Feature > Components: Physical Optimizer > Affects Versions: 3.0.0 > Reporter: Jesus Camacho Rodriguez > Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-16602.patch > > > Given a query plan, the goal is to identify scans on input tables that can be merged so the data is read only once. Optimization will be carried out at the physical level. > In the longer term, identification of equivalent expressions and reutilization of intermediary results should be done at the logical layer via Spool operator. -- This message was sent by Atlassian JIRA (v6.3.15#6346)