hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vrushali C (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-5382) RM does not audit log kill request for active applications
Date Thu, 21 Jul 2016 08:09:20 GMT

     [ https://issues.apache.org/jira/browse/YARN-5382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Vrushali C updated YARN-5382:
-----------------------------
    Attachment: YARN-5382-branch-2.7.03.patch

Uploading patch v3.

Changes made:
- Added a RMAppKillEvent that contains the callerUGI and the remote IP
- Added a method (log success) to print the ip  (I am wondering if we need a corresponding
log failure method?)
- Updated the diagnostic message be similar to YARN-5053
- Tested on local machine in a pseudo distributed setup as well through unit test TestRMAppTransitions.


I see the following in the RM log when I ran and killed a sleep job:
{code}

vrushali[hostname-vrushali hadoop-2.7.4-SNAPSHOT (branch-2.7)]$ grep -rni auditlogger logs/yarn-vvrushali-resourcemanager-hostname-vrushali.log 
| grep -i "kill app"
logs/yarn-vvrushali-resourcemanager-hostname-vrushali.log:204:2016-07-21 00:53:07,493 INFO
org.apache.hadoop.yarn.server.resourcemanager.RMAuditLogger: USER=vvrushali    IP=127.0.0.1   
OPERATION=Kill Application Request    TARGET=RMAppImpl    RESULT=SUCCESS    APPID=application_1469087565504_0001
[hostname-vrushali hadoop-2.7.4-SNAPSHOT (branch-2.7)]$
{code}

I see the following in the unit test log:
{code}

[hostname-vrushali hadoop-yarn-server-resourcemanager (branch-2.7)]$ grep  "Kill Application
" target/surefire-reports/org.apache.hadoop.yarn.server.resourcemanager.rmapp.TestRMAppTransitions-output.txt
2016-07-21 00:54:02,761 INFO  [Thread-5] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(192))
- USER=fooTestAppNewKill    IP=127.0.0.1    OPERATION=Kill Application Request   
TARGET=RMAppImpl    RESULT=SUCCESS    APPID=application_1469087642647_0001
2016-07-21 00:54:03,086 INFO  [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(192))
- USER=fooTestAppSubmittedKill    IP=127.0.0.1    OPERATION=Kill Application Request   
TARGET=RMAppImpl    RESULT=SUCCESS    APPID=application_1469087642647_0007
2016-07-21 00:54:03,155 INFO  [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(192))
- USER=fooTestAppAcceptedKill    IP=127.0.0.1    OPERATION=Kill Application Request   
TARGET=RMAppImpl    RESULT=SUCCESS    APPID=application_1469087642647_0009
2016-07-21 00:54:03,258 INFO  [Thread-53] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(192))
- USER=fooTestAppNewSavingKill    IP=127.0.0.1    OPERATION=Kill Application Request   
TARGET=RMAppImpl    RESULT=SUCCESS    APPID=application_1469087642647_0012
2016-07-21 00:54:03,328 INFO  [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(192))
- USER=fooTestAppRunningKill    IP=127.0.0.1    OPERATION=Kill Application Request   
TARGET=RMAppImpl    RESULT=SUCCESS    APPID=application_1469087642647_0014
2016-07-21 00:54:03,587 INFO  [Thread-102] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(192))
- USER=fooTestAppNewKill    IP=127.0.0.1    OPERATION=Kill Application Request   
TARGET=RMAppImpl    RESULT=SUCCESS    APPID=application_1469087642647_0026
2016-07-21 00:54:03,754 INFO  [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(192))
- USER=fooTestAppSubmittedKill    IP=127.0.0.1    OPERATION=Kill Application Request   
TARGET=RMAppImpl    RESULT=SUCCESS    APPID=application_1469087642647_0032
2016-07-21 00:54:03,807 INFO  [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(192))
- USER=fooTestAppAcceptedKill    IP=127.0.0.1    OPERATION=Kill Application Request   
TARGET=RMAppImpl    RESULT=SUCCESS    APPID=application_1469087642647_0034
2016-07-21 00:54:03,882 INFO  [Thread-150] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(192))
- USER=fooTestAppNewSavingKill    IP=127.0.0.1    OPERATION=Kill Application Request   
TARGET=RMAppImpl    RESULT=SUCCESS    APPID=application_1469087642647_0037
2016-07-21 00:54:03,926 INFO  [main] resourcemanager.RMAuditLogger (RMAuditLogger.java:logSuccess(192))
- USER=fooTestAppRunningKill    IP=127.0.0.1    OPERATION=Kill Application Request   
TARGET=RMAppImpl    RESULT=SUCCESS    APPID=application_1469087642647_0039
[hostname-vrushali hadoop-yarn-server-resourcemanager (branch-2.7)]$

{code}


> RM does not audit log kill request for active applications
> ----------------------------------------------------------
>
>                 Key: YARN-5382
>                 URL: https://issues.apache.org/jira/browse/YARN-5382
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: resourcemanager
>    Affects Versions: 2.7.2
>            Reporter: Jason Lowe
>            Assignee: Vrushali C
>         Attachments: YARN-5382-branch-2.7.01.patch, YARN-5382-branch-2.7.02.patch, YARN-5382-branch-2.7.03.patch
>
>
> ClientRMService will audit a kill request but only if it either fails to issue the kill
or if the kill is sent to an already finished application.  It does not create a log entry
when the application is active which is arguably the most important case to audit.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message