hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3089) LinuxContainerExecutor does not handle file arguments to deleteAsUser
Date Thu, 05 Feb 2015 23:26:35 GMT

    [ https://issues.apache.org/jira/browse/YARN-3089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14308238#comment-14308238
] 

Jason Lowe commented on YARN-3089:
----------------------------------

Looking closer at AppLogAggregatorImpl's rolling interval support, I'm worried enabling the
rolling interval on a cluster will easily lose logs.  For example if the cluster is configured
with a log aggregation rolling interval of 24 hours, is it true that any MapReduce job or
other typical job will have their partial logs uploaded and then removed from the local filesystem
after 24 hours?  It seems like one would have to set the rolling interval to be larger than
the worst-case runtime of any application that doesn't roll its own logs to avoid loss of
logs.

> LinuxContainerExecutor does not handle file arguments to deleteAsUser
> ---------------------------------------------------------------------
>
>                 Key: YARN-3089
>                 URL: https://issues.apache.org/jira/browse/YARN-3089
>             Project: Hadoop YARN
>          Issue Type: Bug
>    Affects Versions: 2.6.0
>            Reporter: Jason Lowe
>            Assignee: Eric Payne
>            Priority: Blocker
>         Attachments: YARN-3089.v1.txt, YARN-3089.v2.txt, YARN-3089.v3.txt
>
>
> YARN-2468 added the deletion of individual logs that are aggregated, but this fails to
delete log files when the LCE is being used.  The LCE native executable assumes the paths
being passed are paths and the delete fails.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message