hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xuan Gong (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2788) yarn logs -applicationId on 2.6.0 should support logs written by 2.4.0
Date Sun, 02 Nov 2014 00:26:34 GMT

    [ https://issues.apache.org/jira/browse/YARN-2788?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14193616#comment-14193616

Xuan Gong commented on YARN-2788:

bq. AggregatedLogFormat.readAcontainerLogs(): There is no caller that passes a valid upload-time.
So I am not sure which tests are really validating the "upload-time" printing now.

Previous testcases should be enough to validate this. Such as TestLogsCLi::testFetchApplictionLogs
and TestAggregatedLogFormat::testReadAcontainerLogs1

bq. A related problem is that tests use different API for rendering compared to the main-line
code. This is very problematic. Can you find out when we introduced this divergence and see
if this can be addressed?

Yes, there are lots of duplicate codes between readAcontainerLogs and readAContainerLogsForALogType.
I am writing a common function and remove the duplicate code.

bq. IAC, if only tests are using readAcontainerLogs, no need to introduce upload-time in this

Wrote a common function and remove the duplicate code between readAcontainerLogs and readAContainerLogsForALogType.
So, let us still add upload-time in this api to keep consistency ?

> yarn logs -applicationId on 2.6.0 should support logs written by 2.4.0
> ----------------------------------------------------------------------
>                 Key: YARN-2788
>                 URL: https://issues.apache.org/jira/browse/YARN-2788
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: log-aggregation
>    Affects Versions: 2.6.0
>            Reporter: Gopal V
>            Assignee: Xuan Gong
>            Priority: Blocker
>         Attachments: YARN-2788.1.1.patch, YARN-2788.1.patch, YARN-2788.2.patch, YARN-2788.3.patch
> Log format version needs to be upped between 2.4.0 and 2.6.0
> {code}
>         at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65)
>         at java.lang.Long.parseLong(Long.java:589)
>         at java.lang.Long.parseLong(Long.java:631)
>         at org.apache.hadoop.yarn.logaggregation.AggregatedLogFormat$ContainerLogsReader.nextLog(AggregatedLogFormat.java:765)
>         at org.apache.hadoop.yarn.webapp.log.AggregatedLogsBlock.readContainerLogs(AggregatedLogsBlock.java:197)
>         at org.apache.hadoop.yarn.webapp.log.AggregatedLogsBlock.render(AggregatedLogsBlock.java:166)
>         at org.apache.hadoop.yarn.webapp.view.HtmlBlock.render(HtmlBlock.java:67)
>         at org.apache.hadoop.yarn.webapp.view.HtmlBlock.renderPartial(HtmlBlock.java:77)
>         at org.apache.hadoop.yarn.webapp.View.render(View.java:235)
>         at org.apache.hadoop.yarn.webapp.view.HtmlPage$Page.subView(HtmlPage.java:49)
>         at org.apache.hadoop.yarn.webapp.hamlet.HamletImpl$EImp._v(HamletImpl.java:117)
>         at org.apache.hadoop.yarn.webapp.hamlet.Hamlet$TD._(Hamlet.java:845)
>         at org.apache.hadoop.yarn.webapp.view.TwoColumnLayout.render(TwoColumnLayout.java:71)
>         at org.apache.hadoop.yarn.webapp.view.HtmlPage.render(HtmlPage.java:82)
>         at org.apache.hadoop.yarn.webapp.Controller.render(Controller.java:212)
>         at org.apache.hadoop.mapreduce.v2.hs.webapp.HsController.logs(HsController.java:178)
>         ... 40 more
> {code}

This message was sent by Atlassian JIRA

View raw message