hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xuan Gong (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7891) LogAggregationIndexedFileController should support HAR file
Date Fri, 02 Mar 2018 12:46:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16383542#comment-16383542

Xuan Gong commented on YARN-7891:

Thanks for the review.

Previous code does not work. It does not handle the har file correctly. So, I have to make
the change to make it work.

In general, the HAR file should not exist with the ifiles(tfiles). Because, when we call HAR
CLI, it would check whether the application finishes and what is the log aggregation status.
But we do have the corner case. The only corner case that I can think is that the application
finishes while the log aggregation is still processing, but at this time, the RM failover/restart
happens. In this case, we would lose the log aggregation status, and some of the logs have
been aggregated. At this time, when we call HAR command, it is possible that we can have har
file as well as ifiles(tfiles).


But, this issue can be solved by https://issues.apache.org/jira/browse/YARN-7952.


bq.  In {{readAggregatedLogs}}/{{readAggregatedLogsMeta}}, following logics are mostly
identical, could you merge common logics if possible?


We can. But the temporary variable we are created here will be used later. In order to re-use
them, we have to make lots of re-factory (maybe, create a common object includes all the variables)
which i think, is too much work for this jira. I would like to solve it later.

> LogAggregationIndexedFileController should support HAR file
> -----------------------------------------------------------
>                 Key: YARN-7891
>                 URL: https://issues.apache.org/jira/browse/YARN-7891
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Xuan Gong
>            Assignee: Xuan Gong
>            Priority: Major
>         Attachments: YARN-7891.1.patch

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message