hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2583) Modify the LogDeletionService to support Log aggregation for LRS
Date Mon, 06 Oct 2014 23:59:34 GMT

    [ https://issues.apache.org/jira/browse/YARN-2583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14161238#comment-14161238
] 

Hadoop QA commented on YARN-2583:
---------------------------------

{color:red}-1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12673234/YARN-2583.3.patch
  against trunk revision 519e5a7.

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:green}+1 tests included{color}.  The patch appears to include 2 new or modified
test files.

      {color:red}-1 javac{color}.  The applied patch generated 1268 javac compiler warnings
(more than the trunk's current 1267 warnings).

    {color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

    {color:green}+1 eclipse:eclipse{color}.  The patch built with eclipse:eclipse.

    {color:green}+1 findbugs{color}.  The patch does not introduce any new Findbugs (version
2.0.3) warnings.

    {color:green}+1 release audit{color}.  The applied patch does not increase the total number
of release audit warnings.

    {color:green}+1 core tests{color}.  The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-api
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager.

    {color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: https://builds.apache.org/job/PreCommit-YARN-Build/5290//testReport/
Javac warnings: https://builds.apache.org/job/PreCommit-YARN-Build/5290//artifact/patchprocess/diffJavacWarnings.txt
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/5290//console

This message is automatically generated.

> Modify the LogDeletionService to support Log aggregation for LRS
> ----------------------------------------------------------------
>
>                 Key: YARN-2583
>                 URL: https://issues.apache.org/jira/browse/YARN-2583
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager, resourcemanager
>            Reporter: Xuan Gong
>            Assignee: Xuan Gong
>         Attachments: YARN-2583.1.patch, YARN-2583.2.patch, YARN-2583.3.1.patch, YARN-2583.3.patch
>
>
> Currently, AggregatedLogDeletionService will delete old logs from HDFS. It will check
the cut-off-time, if all logs for this application is older than this cut-off-time. The app-log-dir
from HDFS will be deleted. This will not work for LRS. We expect a LRS application can keep
running for a long time. 
> Two different scenarios: 
> 1) If we configured the rollingIntervalSeconds, the new log file will be always uploaded
to HDFS. The number of log files for this application will become larger and larger. And there
is no log files will be deleted.
> 2) If we did not configure the rollingIntervalSeconds, the log file can only be uploaded
to HDFS after the application is finished. It is very possible that the logs are uploaded
after the cut-off-time. It will cause problem because at that time the app-log-dir for this
application in HDFS has been deleted.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message