hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-12568) Use the same logic finding HS2 host name in Spark client [Spark Branch]
Date Sat, 05 Dec 2015 01:48:10 GMT

    [ https://issues.apache.org/jira/browse/HIVE-12568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15042565#comment-15042565
] 

Hive QA commented on HIVE-12568:
--------------------------------



Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12775875/HIVE-12568.2-spark.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 43 failed/errored test(s), 9295 tests executed
*Failed tests:*
{noformat}
TestCliDriver-alter_table_not_sorted.q-cbo_udf_max.q-groupby4_map.q-and-12-more - did not
produce a TEST-*.xml file
TestCliDriver-auto_sortmerge_join_7.q-exim_04_evolved_parts.q-query_with_semi.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-compute_stats_string.q-skewjoin_union_remove_2.q-avro_nullable_fields.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-create_func1.q-enforce_order.q-interval_comparison.q-and-12-more - did not produce
a TEST-*.xml file
TestCliDriver-groupby_grouping_id2.q-insert_values_nonascii.q-vectorization_13.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-groupby_sort_7.q-vector_decimal_10_0.q-partition_vs_table_metadata.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-input19.q-index_auth.q-input16.q-and-12-more - did not produce a TEST-*.xml
file
TestCliDriver-interval_udf.q-tez_schema_evolution.q-metadataonly1.q-and-12-more - did not
produce a TEST-*.xml file
TestCliDriver-join_cond_pushdown_unqual4.q-udf_var_samp.q-load_dyn_part2.q-and-12-more - did
not produce a TEST-*.xml file
TestCliDriver-join_filters.q-groupby8_map.q-exim_14_managed_location_over_existing.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-orc_split_elimination.q-udf_xpath_string.q-partition_wise_fileformat.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-partition_coltype_literals.q-index_auto_partitioned.q-udf_hash.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-ppd_join4.q-union27.q-show_indexes_edge_cases.q-and-12-more - did not produce
a TEST-*.xml file
TestCliDriver-ppd_union.q-udf_round_3.q-groupby12.q-and-12-more - did not produce a TEST-*.xml
file
TestCliDriver-push_or.q-infer_bucket_sort_list_bucket.q-vector_interval_2.q-and-12-more -
did not produce a TEST-*.xml file
TestCliDriver-show_conf.q-udaf_covar_samp.q-udf_md5.q-and-12-more - did not produce a TEST-*.xml
file
TestCliDriver-skewjoin_mapjoin4.q-groupby6_map.q-cbo_rp_union.q-and-12-more - did not produce
a TEST-*.xml file
TestCliDriver-skewjoinopt15.q-fileformat_text.q-load_dyn_part12.q-and-12-more - did not produce
a TEST-*.xml file
TestCliDriver-smb_mapjoin_15.q-mapreduce1.q-ppd_udf_case.q-and-12-more - did not produce a
TEST-*.xml file
TestCliDriver-sort_merge_join_desc_3.q-join2.q-vector_data_types.q-and-12-more - did not produce
a TEST-*.xml file
TestCliDriver-stats12.q-unionall_unbalancedppd.q-insert_overwrite_local_directory_1.q-and-12-more
- did not produce a TEST-*.xml file
TestCliDriver-tez_joins_explain.q-rename_column.q-index_compact.q-and-12-more - did not produce
a TEST-*.xml file
TestCliDriver-udf_explode.q-avro_add_column.q-bucketmapjoin_negative.q-and-12-more - did not
produce a TEST-*.xml file
TestCliDriver-union36.q-acid_join.q-part_inherit_tbl_props_empty.q-and-6-more - did not produce
a TEST-*.xml file
TestHWISessionManager - did not produce a TEST-*.xml file
TestMiniTezCliDriver-auto_join30.q-vector_decimal_10_0.q-schema_evol_orc_acidvec_mapwork_part.q-and-12-more
- did not produce a TEST-*.xml file
TestMiniTezCliDriver-cbo_windowing.q-tez_join.q-bucket_map_join_tez1.q-and-12-more - did not
produce a TEST-*.xml file
TestMiniTezCliDriver-update_orig_table.q-update_after_multiple_inserts.q-unionDistinct_2.q-and-12-more
- did not produce a TEST-*.xml file
TestMiniTezCliDriver-vector_interval_2.q-constprog_dpp.q-dynamic_partition_pruning.q-and-12-more
- did not produce a TEST-*.xml file
TestMiniTezCliDriver-vectorization_13.q-tez_bmj_schema_evolution.q-bucket3.q-and-12-more -
did not produce a TEST-*.xml file
TestParseNegative - did not produce a TEST-*.xml file
TestSparkCliDriver-bucketsortoptimize_insert_7.q-skewjoin_noskew.q-sample2.q-and-12-more -
did not produce a TEST-*.xml file
TestSparkCliDriver-groupby_complex_types.q-auto_join9.q-groupby_map_ppr.q-and-12-more - did
not produce a TEST-*.xml file
TestSparkCliDriver-join_casesensitive.q-decimal_join.q-mapjoin_addjar.q-and-12-more - did
not produce a TEST-*.xml file
TestSparkCliDriver-ppd_join4.q-skewjoinopt3.q-union27.q-and-12-more - did not produce a TEST-*.xml
file
TestSparkCliDriver-runtime_skewjoin_mapjoin_spark.q-union_remove_9.q-ppd_multi_insert.q-and-12-more
- did not produce a TEST-*.xml file
TestSparkCliDriver-stats12.q-skewjoinopt19.q-groupby10.q-and-12-more - did not produce a TEST-*.xml
file
TestSparkCliDriver-stats13.q-stats2.q-ppd_gby_join.q-and-12-more - did not produce a TEST-*.xml
file
TestSparkCliDriver-vectorization_10.q-list_bucket_dml_2.q-scriptfile1.q-and-12-more - did
not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_import
org.apache.hadoop.hive.metastore.TestHiveMetaStorePartitionSpecs.testGetPartitionSpecs_WithAndWithoutPartitionGrouping
org.apache.hive.hcatalog.hbase.TestPigHBaseStorageHandler.org.apache.hive.hcatalog.hbase.TestPigHBaseStorageHandler
org.apache.hive.jdbc.TestSSL.testSSLVersion
{noformat}

Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/1020/testReport
Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-SPARK-Build/1020/console
Test logs: http://ec2-50-18-27-0.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-SPARK-Build-1020/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 43 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12775875 - PreCommit-HIVE-SPARK-Build

> Use the same logic finding HS2 host name in Spark client [Spark Branch]
> -----------------------------------------------------------------------
>
>                 Key: HIVE-12568
>                 URL: https://issues.apache.org/jira/browse/HIVE-12568
>             Project: Hive
>          Issue Type: Bug
>          Components: Spark
>    Affects Versions: 1.1.0
>            Reporter: Xuefu Zhang
>            Assignee: Xuefu Zhang
>         Attachments: HIVE-12568.0-spark.patch, HIVE-12568.1-spark.patch, HIVE-12568.2-spark.patch
>
>
> Spark client sends a pair of host name and port number to the remote driver so that the
driver can connects back to HS2 where the user session is. Spark client has its own way determining
the host name, and pick one network interface if the host happens to have multiple network
interfaces. This can be problematic. For that, there is parameter, hive.spark.client.server.address,
which user can pick an interface. Unfortunately, this interface isn't exposed.
> Instead of exposing this parameter, we can use the same logic as Hive in determining
the host name. Therefore, the remote driver connecting to HS2 using the same network interface
as a HS2 client would do.
> There might be a case where user may want the remote driver to use a different network.
This is rare if at all. Thus, for now it should be sufficient to use the same network interface.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message