hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xuan Gong (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2468) Log handling for LRS
Date Tue, 21 Oct 2014 19:11:35 GMT

    [ https://issues.apache.org/jira/browse/YARN-2468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14178869#comment-14178869
] 

Xuan Gong commented on YARN-2468:
---------------------------------

bq. Thanks. But what about the case where the file was modified. Let's say 10 more lines were
added to the syslog file. Doesn't it upload the full file again?

This is the pre-requirement: We will rely on user’s log application (such as log4j) to do
the rollover for the logs. Users need to set up their log application properly. For our side,
we upload every logs in our log dirs.

> Log handling for LRS
> --------------------
>
>                 Key: YARN-2468
>                 URL: https://issues.apache.org/jira/browse/YARN-2468
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: log-aggregation, nodemanager, resourcemanager
>            Reporter: Xuan Gong
>            Assignee: Xuan Gong
>             Fix For: 2.6.0
>
>         Attachments: YARN-2468.1.patch, YARN-2468.10.patch, YARN-2468.11.patch, YARN-2468.2.patch,
YARN-2468.3.patch, YARN-2468.3.rebase.2.patch, YARN-2468.3.rebase.patch, YARN-2468.4.1.patch,
YARN-2468.4.patch, YARN-2468.5.1.patch, YARN-2468.5.1.patch, YARN-2468.5.2.patch, YARN-2468.5.3.patch,
YARN-2468.5.4.patch, YARN-2468.5.patch, YARN-2468.6.1.patch, YARN-2468.6.patch, YARN-2468.7.1.patch,
YARN-2468.7.patch, YARN-2468.8.patch, YARN-2468.9.1.patch, YARN-2468.9.patch
>
>
> Currently, when application is finished, NM will start to do the log aggregation. But
for Long running service applications, this is not ideal. The problems we have are:
> 1) LRS applications are expected to run for a long time (weeks, months).
> 2) Currently, all the container logs (from one NM) will be written into a single file.
The files could become larger and larger.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message