hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Siddharth Seth (JIRA)" <j...@apache.org>
Subject [jira] [Created] (YARN-5193) For long running services, aggregate logs when a container completes instead of when the app completes
Date Thu, 02 Jun 2016 06:40:59 GMT
Siddharth Seth created YARN-5193:
------------------------------------

             Summary: For long running services, aggregate logs when a container completes
instead of when the app completes
                 Key: YARN-5193
                 URL: https://issues.apache.org/jira/browse/YARN-5193
             Project: Hadoop YARN
          Issue Type: Improvement
            Reporter: Siddharth Seth


For a long running service, containers will typically not complete very often. However, when
a container completes - it would be useful to aggregate the logs right then, instead of waiting
for the app to complete.
This will allow the command line log tool to lookup containers for an app from the log file
index itself, instead of having to go and talk to YARN. Talking to YARN really only works
if ATS is enabled, and YARN is configured to publish container information to ATS (That may
not always be the case - since this can overload ATS quite fast).

There's some added benefits like cleaning out local disk space early, instead of waiting till
the app completes. (There's probably a separate jira somewhere about cleanup of container
for long running services anyway)

cc [~vinodkv], [~xgong]



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message