hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-19008) Improve Spark session id logging
Date Mon, 16 Apr 2018 04:02:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-19008?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16438965#comment-16438965
] 

Hive QA commented on HIVE-19008:
--------------------------------



Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12919034/HIVE-19008.1.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 115 failed/errored test(s), 14192 tests executed
*Failed tests:*
{noformat}
TestDbNotificationListener - did not produce a TEST-*.xml file (likely timed out) (batchId=247)
TestHCatHiveCompatibility - did not produce a TEST-*.xml file (likely timed out) (batchId=247)
TestNonCatCallsWithCatalog - did not produce a TEST-*.xml file (likely timed out) (batchId=217)
TestSequenceFileReadWrite - did not produce a TEST-*.xml file (likely timed out) (batchId=247)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_0] (batchId=17)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb] (batchId=153)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[mm_all] (batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=174)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez_empty]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb] (batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_1] (batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_main] (batchId=160)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_5] (batchId=105)
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver[infer_bucket_sort_dyn_part] (batchId=93)
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver[infer_bucket_sort_map_operators]
(batchId=93)
org.apache.hadoop.hive.cli.TestMinimrCliDriver.testCliDriver[infer_bucket_sort_num_buckets]
(batchId=93)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.org.apache.hadoop.hive.cli.TestNegativeCliDriver
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.org.apache.hadoop.hive.cli.TestNegativeCliDriver
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[alter_notnull_constraint_violation]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[default_constraint_invalid_default_value_type]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[insert_into_acid_notnull] (batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[insert_into_notnull_constraint]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[insert_multi_into_notnull]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[insert_overwrite_notnull_constraint]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[insertsel_fail] (batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[joinneg] (batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[ptf_negative_JoinWithAmbigousAlias]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[smb_bucketmapjoin] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[smb_mapjoin_14] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[sortmerge_mapjoin_mismatch_1]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[stats_aggregator_error_1] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[stats_aggregator_error_2] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[stats_publisher_error_1] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[stats_publisher_error_2] (batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[subq_insert] (batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[subquery_corr_in_agg] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[subquery_in_implicit_gby] (batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[subquery_notin_implicit_gby]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[truncate_bucketed_column] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[truncate_column_list_bucketing]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[truncate_column_seqfile] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[truncate_nonexistant_column]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[truncate_partition_column2]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[truncate_partition_column]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_assert_true2] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_elt_wrong_type] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_format_number_wrong3] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_format_number_wrong5] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_greatest_error_2] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_if_not_bool] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_if_wrong_args_len] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_in] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_instr_wrong_args_len] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_invalid] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_likeall_wrong1] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_likeany_wrong1] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_map_keys_arg_num] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_map_keys_arg_type] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_map_values_arg_type] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_max] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_min] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_next_day_error_1] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_next_day_error_2] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_nonexistent_resource] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_printf_wrong4] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_reflect_neg] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_sort_array_by_wrong1] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_sort_array_by_wrong2] (batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_sort_array_by_wrong3] (batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_sort_array_wrong1] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_sort_array_wrong2] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_sort_array_wrong3] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_test_error] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_test_error_reduce] (batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_trunc_error1] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_trunc_error2] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_trunc_error3] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udf_when_type_wrong] (batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udtf_explode_not_supported1]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udtf_explode_not_supported2]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udtf_explode_not_supported3]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udtf_explode_not_supported4]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udtf_invalid_place] (batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udtf_not_supported1] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[udtf_not_supported3] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[union22] (batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[union2] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[unionClusterBy] (batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[unionDistributeBy] (batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[unionLimit] (batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[unionOrderBy] (batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[unionSortBy] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[uniquejoin2] (batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[uniquejoin3] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[uniquejoin] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[unset_table_property] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[unset_view_property] (batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[updateBasicStats] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[update_bucket_col] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[update_no_such_table] (batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[update_non_acid_table] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[update_not_acid] (batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[update_notnull_constraint]
(batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[update_partition_col] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[view_update] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[windowing_after_orderby] (batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[windowing_invalid_udaf] (batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[windowing_leadlag_in_udaf]
(batchId=96)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[windowing_ll_no_neg] (batchId=95)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[windowing_ll_no_over] (batchId=95)
org.apache.hadoop.hive.druid.TestDruidStorageHandler.testCommitMultiInsertOverwriteTable (batchId=261)
org.apache.hadoop.hive.metastore.TestMetastoreVersion.testMetastoreVersion (batchId=227)
org.apache.hadoop.hive.metastore.TestMetastoreVersion.testVersionMatching (batchId=227)
org.apache.hadoop.hive.ql.TestMTQueries.testMTQueries1 (batchId=232)
org.apache.hive.service.cli.operation.TestOperationLoggingAPIWithTez.org.apache.hive.service.cli.operation.TestOperationLoggingAPIWithTez
(batchId=237)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/10238/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/10238/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-10238/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 115 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12919034 - PreCommit-HIVE-Build

> Improve Spark session id logging
> --------------------------------
>
>                 Key: HIVE-19008
>                 URL: https://issues.apache.org/jira/browse/HIVE-19008
>             Project: Hive
>          Issue Type: Sub-task
>          Components: Spark
>            Reporter: Sahil Takiar
>            Assignee: Sahil Takiar
>            Priority: Major
>         Attachments: HIVE-19008.1.patch
>
>
> HoS users have two session ids, one id for the Hive session and another id for the Spark
session, both are UUIDs.
> I think some improvements could be made here:
> The Spark session id could just be a counter that is incremented for each new Spark session
within a Hive session. Each Spark session is still globally identifiable by its associated
Hive session id + its own counter. This may make more sense since the Hive session - Spark
session has a 1-to-many relationship, as in a single Hive session can contain multiple Spark
sessions, and each Spark session must belong to a Hive session.
> Furthermore, we should include both the Hive session id and Spark session id in the console
logs + the Spark Web UI.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message