hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhijie Shen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2748) Upload logs in the sub-folders under the local log dir when aggregating logs
Date Wed, 26 Nov 2014 21:31:13 GMT

    [ https://issues.apache.org/jira/browse/YARN-2748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14226829#comment-14226829

Zhijie Shen commented on YARN-2748:

[~varun_saxena], thanks for the patch. I mostly agree on your points. Given {{Log Root Dir/sub-dir1/sub-dir2/.../xxxx.log}},
we can use the relative path {{sub-dir1/sub-dir2/.../xxxx.log}} to uniquely identify a log.

One question is if we want to support a single-level sub folder or recursive sub folders.
The choice will make the solution significantly different. The use case of the single-level
sub folder is to put the all rolling log files of one particular log in a single folder. I'd
like to hear more from [~sumitmohanty] about the use case he met before.

> Upload logs in the sub-folders under the local log dir when aggregating logs
> ----------------------------------------------------------------------------
>                 Key: YARN-2748
>                 URL: https://issues.apache.org/jira/browse/YARN-2748
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: log-aggregation
>    Affects Versions: 2.6.0
>            Reporter: Zhijie Shen
>            Assignee: Varun Saxena
>             Fix For: 2.7.0
>         Attachments: YARN-2748.001.patch
> YARN-2734 has a temporal fix to skip sub folders to avoid exception. Ideally, if the
app is creating a sub folder and putting its rolling logs there, we need to upload these logs
as well.

This message was sent by Atlassian JIRA

View raw message