hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hadoop QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2724) If an unreadable file is encountered during log aggregation then aggregated file in HDFS badly formed
Date Thu, 23 Oct 2014 06:53:34 GMT

    [ https://issues.apache.org/jira/browse/YARN-2724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14181074#comment-14181074
] 

Hadoop QA commented on YARN-2724:
---------------------------------

{color:green}+1 overall{color}.  Here are the results of testing the latest attachment 
  http://issues.apache.org/jira/secure/attachment/12676535/YARN-2724.3.patch
  against trunk revision d71d40a.

    {color:green}+1 @author{color}.  The patch does not contain any @author tags.

    {color:green}+1 tests included{color}.  The patch appears to include 1 new or modified
test files.

    {color:green}+1 javac{color}.  The applied patch does not increase the total number of
javac compiler warnings.

    {color:green}+1 javadoc{color}.  There were no new javadoc warning messages.

    {color:green}+1 eclipse:eclipse{color}.  The patch built with eclipse:eclipse.

    {color:green}+1 findbugs{color}.  The patch does not introduce any new Findbugs (version
2.0.3) warnings.

    {color:green}+1 release audit{color}.  The applied patch does not increase the total number
of release audit warnings.

    {color:green}+1 core tests{color}.  The patch passed unit tests in hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common.

    {color:green}+1 contrib tests{color}.  The patch passed contrib unit tests.

Test results: https://builds.apache.org/job/PreCommit-YARN-Build/5516//testReport/
Console output: https://builds.apache.org/job/PreCommit-YARN-Build/5516//console

This message is automatically generated.

> If an unreadable file is encountered during log aggregation then aggregated file in HDFS
badly formed
> -----------------------------------------------------------------------------------------------------
>
>                 Key: YARN-2724
>                 URL: https://issues.apache.org/jira/browse/YARN-2724
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: log-aggregation
>    Affects Versions: 2.5.1
>            Reporter: Sumit Mohanty
>            Assignee: Xuan Gong
>         Attachments: YARN-2724.1.patch, YARN-2724.2.patch, YARN-2724.3.patch
>
>
> Look into the log output snippet. It looks like there is an issue during aggregation
when an unreadable file is encountered. Likely, this results in bad encoding.
> {noformat}
> LogType: command-13.json
> LogLength: 13934
> Log Contents:
> Error aggregating log file. Log file : /grid/0/yarn/log/application_1413865041660_0002/container_1413865041660_0002_01_000004/command-13.json/grid/0/yarn/log/application_1413865041660_0002/container_1413865041660_0002_01_000004/command-13.json
(Permission denied)command-3.json13983Error aggregating log file. Log file : /grid/0/yarn/log/application_1413865041660_0002/container_1413865041660_0002_01_000004/command-3.json/grid/0/yarn/log/application_1413865041660_0002/contaierrors-13.txt0660_0002_01_000004/command-3.json
(Permission denied)
>               errors-3.txt0gc.log-20141021044514484052014-10-21T04:45:12.046+0000: 5.134:
[GC2014-10-21T04:45:12.046+0000: 5.134: [ParNew: 163840K->15575K(184320K), 0.0488700 secs]
163840K->15575K(1028096K), 0.0492510 secs] [Times: user=0.06 sys=0.01, real=0.05 secs]
> 2014-10-21T04:45:14.939+0000: 8.027: [GC2014-10-21T04:45:14.939+0000: 8.027: [ParNew:
179415K->11865K(184320K), 0.0941310 secs] 179415K->17228K(1028096K), 0.0943140 secs]
[Times: user=0.13 sys=0.04, real=0.09 secs]
> 2014-10-21T04:46:42.099+0000: 95.187: [GC2014-10-21T04:46:42.099+0000: 95.187: [ParNew:
175705K->12802K(184320K), 0.0466420 secs] 181068K->18164K(1028096K), 0.0468490 secs]
[Times: user=0.06 sys=0.00, real=0.04 secs]
> {noformat}
> Specifically, look at the text after the exception text. There should be two more entries
for log files but none exist. This is likely due to the fact that command-13.json is expected
to be of length 13934 but its is not as the file was never read.
> I think, it should have been
> {noformat}
> LogType: command-13.json
> LogLength: <Length of the exception text>
> Log Contents:
> Error aggregating log file. Log file : /grid/0/yarn/log/application_1413865041660_0002/container_1413865041660_0002_01_000004/command-13.json/grid/0/yarn/log/application_1413865041660_0002/container_1413865041660_0002_01_000004/command-13.json
(Permission denied)command-3.json13983Error aggregating log file. Log file : /grid/0/yarn/log/application_1413865041660_0002/container_1413865041660_0002_01_000004/command-3.json/grid/0/yarn/log/application_1413865041660_0002/contaierrors-13.txt0660_0002_01_000004/command-3.json
(Permission denied)
> {noformat}
> {noformat}
> LogType: errors-3.txt
> LogLength:0
> Log Contents:
> {noformat}
> {noformat}
> LogType:gc.log
> LogLength:???
> Log Contents:
> ......-20141021044514484052014-10-21T04:45:12.046+0000: 5.134: [GC2014-10-21T04:45:12.046+0000:
5.134: [ParNew: 163840K- .......
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message