hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-16602) Implement shared scans with Tez
Date Mon, 08 May 2017 14:20:05 GMT

    [ https://issues.apache.org/jira/browse/HIVE-16602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16000794#comment-16000794
] 

Hive QA commented on HIVE-16602:
--------------------------------



Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12866868/HIVE-16602.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 86 failed/errored test(s), 10564 tests executed
*Failed tests:*
{noformat}
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=147)
	[list_bucket_dml_10.q,mapjoin_emit_interval.q,acid_globallimit.q,delete_where_no_match.q,semijoin_hint.q,vector_groupby_grouping_sets4.q,tez_vector_dynpart_hashjoin_2.q,vectorization_part.q,delete_all_non_partitioned.q,delete_all_partitioned.q,auto_join30.q,dynamic_partition_pruning.q,cluster.q,bucketmapjoin7.q,cbo_rp_unionDistinct_2.q,subquery_nested_subquery.q,limit_pushdown3.q,vector_outer_join2.q,smb_mapjoin_18.q,vector_varchar_4.q,ppd_union_view.q,union6.q,vector_decimal_4.q,schema_evol_orc_acid_part.q,cbo_subq_in.q,cross_join.q,parquet_predicate_pushdown.q,llap_vector_nohybridgrace.q,tez_smb_main.q,vectorized_timestamp_funcs.q]
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=148)
	[vector_auto_smb_mapjoin_14.q,deleteAnalyze.q,multi_insert.q,tez_dml.q,database.q,vector_bround.q,orc_ppd_schema_evol_1b.q,vector_join30.q,smb_mapjoin_6.q,vector_groupby_grouping_sets2.q,cte_3.q,vector_reduce_groupby_decimal.q,vectorized_dynamic_partition_pruning.q,cbo_views.q,vector_ptf_part_simple.q,vector_decimal_cast.q,groupby_grouping_id2.q,tez_smb_empty.q,orc_merge6.q,cte_mat_1.q,vector_char_mapjoin1.q,cte_5.q,subquery_shared_alias.q,vector_decimal_2.q,columnStatsUpdateForStatsOptimizer_1.q,vector_outer_join3.q,vector_string_concat.q,vectorized_context.q,metadata_only_queries.q,auto_sortmerge_join_12.q]
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=155)
	[vector_interval_2.q,schema_evol_orc_acid_part_update.q,orc_ppd_varchar.q,metadataonly1.q,join_nullsafe.q,vectorized_shufflejoin.q,limit_pushdown.q,auto_join_nulls.q,metadata_only_queries_with_filters.q,vector_inner_join.q,subquery_notin.q,schema_evol_text_nonvec_part_all_complex.q,vector_coalesce_2.q,vector_between_columns.q,vector_groupby_grouping_sets6.q,table_access_keys_stats.q,udaf_collect_set_2.q,subquery_null_agg.q,update_after_multiple_inserts.q,offset_limit_ppd_optimizer.q,mapjoin_decimal.q,orc_merge_incompat1.q,columnstats_part_coltype.q,vectorized_parquet_types.q,vector_char_2.q,stats_based_fetch_decision.q,auto_sortmerge_join_10.q,extrapolate_part_stats_partial_ndv.q,auto_sortmerge_join_9.q,vector_groupby_grouping_id2.q]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[cte_2] (batchId=139)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[except_distinct] (batchId=140)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[explainuser_2] (batchId=140)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[intersect_all] (batchId=139)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[intersect_distinct] (batchId=140)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[intersect_merge] (batchId=138)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_nullscan] (batchId=138)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[tez_union_dynamic_partition]
(batchId=139)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1] (batchId=139)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_join0] (batchId=158)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_join_filters] (batchId=153)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_smb_mapjoin_14] (batchId=153)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=158)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[column_access_stats] (batchId=153)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[correlationoptimizer2]
(batchId=151)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[correlationoptimizer3]
(batchId=158)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[correlationoptimizer6]
(batchId=151)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[explainuser_1] (batchId=149)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[join46] (batchId=142)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[limit_join_transpose]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[mapjoin46] (batchId=153)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[mrr] (batchId=144)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[multiMapJoin1] (batchId=151)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[multiMapJoin2] (batchId=156)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[orc_ppd_date] (batchId=150)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[orc_ppd_decimal] (batchId=143)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[orc_ppd_timestamp] (batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[ppr_pushdown] (batchId=150)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_in] (batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_multi] (batchId=145)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_scalar] (batchId=149)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_select] (batchId=149)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_views] (batchId=144)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_join_tests] (batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_joins_explain] (batchId=149)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union8] (batchId=153)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_stats] (batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_top_level] (batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_groupby_mapjoin]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr] (batchId=143)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_join_filters] (batchId=154)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_1] (batchId=96)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2] (batchId=97)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_5] (batchId=96)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] (batchId=230)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query16] (batchId=230)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query17] (batchId=230)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query1] (batchId=230)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23] (batchId=230)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query25] (batchId=230)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query29] (batchId=230)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query30] (batchId=230)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query31] (batchId=230)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query32] (batchId=230)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query33] (batchId=230)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query38] (batchId=230)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query39] (batchId=230)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query46] (batchId=230)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query51] (batchId=230)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query56] (batchId=230)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query58] (batchId=230)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query5] (batchId=230)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query60] (batchId=230)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query64] (batchId=230)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query65] (batchId=230)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query66] (batchId=230)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query68] (batchId=230)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query69] (batchId=230)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query6] (batchId=230)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query70] (batchId=230)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query75] (batchId=230)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query76] (batchId=230)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query80] (batchId=230)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query81] (batchId=230)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query83] (batchId=230)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query85] (batchId=230)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query87] (batchId=230)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query90] (batchId=230)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query92] (batchId=230)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query95] (batchId=230)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query97] (batchId=230)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query9] (batchId=230)
org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgressParallel (batchId=218)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/5111/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/5111/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-5111/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 86 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12866868 - PreCommit-HIVE-Build

> Implement shared scans with Tez
> -------------------------------
>
>                 Key: HIVE-16602
>                 URL: https://issues.apache.org/jira/browse/HIVE-16602
>             Project: Hive
>          Issue Type: New Feature
>          Components: Physical Optimizer
>    Affects Versions: 3.0.0
>            Reporter: Jesus Camacho Rodriguez
>            Assignee: Jesus Camacho Rodriguez
>         Attachments: HIVE-16602.patch
>
>
> Given a query plan, the goal is to identify scans on input tables that can be merged
so the data is read only once. Optimization will be carried out at the physical level.
> In the longer term, identification of equivalent expressions and reutilization of intermediary
results should be done at the logical layer via Spool operator.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message