hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5382) RM does not audit log kill request for active applications
Date Mon, 18 Jul 2016 18:15:20 GMT

    [ https://issues.apache.org/jira/browse/YARN-5382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15382769#comment-15382769
] 

Jason Lowe commented on YARN-5382:
----------------------------------

Ah, I see.  The client is continuing to issue the kill request until the application is reported
as killed.  Unfortunately if the client does not linger until the kill is reported completely
successful then we can end up in a situation where the RM does end up killing the app based
on the initial request but it is never logged anywhere.

One thing we could do is no longer audit killing of completed applications (it's not that
interesting anyway) and only audit the killing of active apps.  We could check the app state
and avoid logging for any app that is completed or in the process of completing (i.e.: state
is FINAL_SAVING, KILLING, etc.).  Then for most cases we should only get a single audit log
unless the RM is slow at dispatching events.  Similarly we could set a flag on the application
as being killed and only log if it hasn't been flagged.

Looks like this was introduced in YARN-1446.  Pinging [~vinodkv] and [~jianhe] in case they
have opinions on how this should be fixed.


> RM does not audit log kill request for active applications
> ----------------------------------------------------------
>
>                 Key: YARN-5382
>                 URL: https://issues.apache.org/jira/browse/YARN-5382
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: resourcemanager
>    Affects Versions: 2.7.2
>            Reporter: Jason Lowe
>            Assignee: Vrushali C
>         Attachments: YARN-5382-branch-2.7.01.patch, YARN-5382-branch-2.7.02.patch
>
>
> ClientRMService will audit a kill request but only if it either fails to issue the kill
or if the kill is sent to an already finished application.  It does not create a log entry
when the application is active which is arguably the most important case to audit.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message