hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-14739) Replace runnables directly added to runtime shutdown hooks to avoid deadlock
Date Tue, 13 Sep 2016 21:45:21 GMT

    [ https://issues.apache.org/jira/browse/HIVE-14739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15488526#comment-15488526
] 

Hive QA commented on HIVE-14739:
--------------------------------



Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12828311/HIVE-14739.3.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 10545 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_mapjoin]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[stats0]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_join_part_col_char]
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[acid_bucket_pruning]
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3]
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testAddJarConstructorUnCaching
org.apache.hive.spark.client.TestSparkClient.testJobSubmission
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/1167/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-MASTER-Build/1167/console
Test logs: http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-MASTER-Build-1167/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12828311 - PreCommit-HIVE-MASTER-Build

> Replace runnables directly added to runtime shutdown hooks to avoid deadlock
> ----------------------------------------------------------------------------
>
>                 Key: HIVE-14739
>                 URL: https://issues.apache.org/jira/browse/HIVE-14739
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: 2.2.0
>            Reporter: Deepesh Khandelwal
>            Assignee: Prasanth Jayachandran
>         Attachments: HIVE-14739.1.patch, HIVE-14739.2.patch, HIVE-14739.3.patch
>
>
> [~deepesh] reported that a deadlock can occur when running queries through hive cli.
[~cnauroth] analyzed it and reported that hive adds shutdown hooks directly to java Runtime
which may execute in non-deterministic order causing deadlocks with hadoop's shutdown hooks.
In one case, hadoop shutdown locked FileSystem#Cache and FileSystem.close whereas hive shutdown
hook locked FileSystem.close and FileSystem#Cache order causing a deadlock. 
> Hive and Hadoop has ShutdownHookManager that runs the shutdown hooks in deterministic
order based on priority. We should use that to avoid deadlock throughout the code.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message