hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xuan Gong (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3154) Should not upload partial logs for MR jobs or other "short-running' applications
Date Thu, 05 Mar 2015 05:24:38 GMT

    [ https://issues.apache.org/jira/browse/YARN-3154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14348169#comment-14348169
] 

Xuan Gong commented on YARN-3154:
---------------------------------

bq. how should an app handle stdout and stderr i.e. rotating of these files?

Yes, and this is the user's responsibility for this. All we need to do is to upload the logs
which exist.

bq.what regex format is supported? Will the NMClient do a sanity check to see if the format
is valid?

Support java regex. All the Include/Excludepatterns are defined in ASC.

> Should not upload partial logs for MR jobs or other "short-running' applications 
> ---------------------------------------------------------------------------------
>
>                 Key: YARN-3154
>                 URL: https://issues.apache.org/jira/browse/YARN-3154
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager, resourcemanager
>            Reporter: Xuan Gong
>            Assignee: Xuan Gong
>            Priority: Blocker
>         Attachments: YARN-3154.1.patch, YARN-3154.2.patch, YARN-3154.3.patch
>
>
> Currently, if we are running a MR job, and we do not set the log interval properly, we
will have their partial logs uploaded and then removed from the local filesystem which is
not right.
> We only upload the partial logs for LRS applications.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message