hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Weiwei Yang (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HADOOP-13837) Process check bug in hadoop_stop_daemon of hadoop-functions.sh
Date Mon, 28 Nov 2016 17:40:58 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15702543#comment-15702543
] 

Weiwei Yang edited comment on HADOOP-13837 at 11/28/16 5:40 PM:
----------------------------------------------------------------

You can use [^check_proc.sh] to test, run

{code}
bash check_proc.sh <hadoop-daemon-pid>
{code}

it gives output like 

{code}
check proc
process 20377 is still running
check proc
process is killed
{code}

To fix this issue, propose to wrap up a function to check process liveness by pid, it checks
the process by fixed interval with timeout. This will fix both issues
# it always waits for $HADOOP_STOP_TIMEOUT even it doesn't necessarily to
# it doesn't wait enough time until the process gets killed.





was (Author: cheersyang):
You can use [^check_proc.sh] to test, run

{code}
bash check_proc.sh <hadoop-daemon-pid>
{code}

it gives output like 

{code}
check proc
process 20377 is still running
check proc
process is killed
{code}

To fix this issue, propose to wrap up a function to check process liveness by pid, it checks
the process by fixed interval with timeout. This will fix the issue it always waits for $HADOOP_STOP_TIMEOUT
even it doesn't necessarily to, and the issue it doesn't wait enough time until the process
gets killed.




> Process check bug in hadoop_stop_daemon of hadoop-functions.sh
> --------------------------------------------------------------
>
>                 Key: HADOOP-13837
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13837
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: scripts
>            Reporter: Weiwei Yang
>            Assignee: Weiwei Yang
>         Attachments: check_proc.sh
>
>
> Always get {{ERROR: Unable to kill ...}} after {{Trying to kill with kill -9}}, see following
output of stop-yarn.sh
> {code}
> <NM_HOST>: WARNING: nodemanager did not stop gracefully after 5 seconds: Trying
to kill with kill -9
> <NM_HOST>: ERROR: Unable to kill 18097
> {code}
> hadoop_stop_daemon doesn't check process liveness correctly, this bug can be reproduced
by the script easily. kill -9 would need some time to be done, directly check process existence
right after mostly will fail.
> {code}
> function hadoop_stop_daemon
> {
>     ...
>       kill -9 "${pid}" >/dev/null 2>&1
>     fi
>     if ps -p "${pid}" > /dev/null 2>&1; then
>       hadoop_error "ERROR: Unable to kill ${pid}"
>     else
>       ...
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message