hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7072) Add a new log aggregation file format controller
Date Tue, 29 Aug 2017 18:52:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145893#comment-16145893

Wangda Tan commented on YARN-7072:

Offline discussed with [~xgong]. 

bq. I suggest to add a separator to this patch (a hard coded and randomly generated 128 bits
string) if it is just a trivial change. This might be important for the future file format,
update implementation is relatively simple but update format is painful.
We decided to remove uuid from file and append it after each partial aggregation completed.
It will be a const value. 

Some rename suggestions: 
1) PerContainerLogFileInfo => ContainerLogFileInfo
2) LogMeta => IndexedPerAggregationLogMeta
3) {{private Map<String, List<FileLogMeta>> logMetas = new HashMap<>();}},
use ContainerId as key. 
4) FileLogMeta => IndexedFileLogMeta

Beyond this, I don't have more comments, I hope [~djp] could help to take a look at the patch
as well.

> Add a new log aggregation file format controller
> ------------------------------------------------
>                 Key: YARN-7072
>                 URL: https://issues.apache.org/jira/browse/YARN-7072
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Xuan Gong
>            Assignee: Xuan Gong
>         Attachments: YARN-7072-trunk.001.patch, YARN-7072.trunk.002.patch, YARN-7072-trunk.003.patch

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message