hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7072) Add a new log aggregation file format controller
Date Tue, 29 Aug 2017 18:52:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145893#comment-16145893
] 

Wangda Tan commented on YARN-7072:
----------------------------------

Offline discussed with [~xgong]. 

bq. I suggest to add a separator to this patch (a hard coded and randomly generated 128 bits
string) if it is just a trivial change. This might be important for the future file format,
update implementation is relatively simple but update format is painful.
We decided to remove uuid from file and append it after each partial aggregation completed.
It will be a const value. 

Some rename suggestions: 
1) PerContainerLogFileInfo => ContainerLogFileInfo
2) LogMeta => IndexedPerAggregationLogMeta
3) {{private Map<String, List<FileLogMeta>> logMetas = new HashMap<>();}},
use ContainerId as key. 
4) FileLogMeta => IndexedFileLogMeta

Beyond this, I don't have more comments, I hope [~djp] could help to take a look at the patch
as well.

> Add a new log aggregation file format controller
> ------------------------------------------------
>
>                 Key: YARN-7072
>                 URL: https://issues.apache.org/jira/browse/YARN-7072
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Xuan Gong
>            Assignee: Xuan Gong
>         Attachments: YARN-7072-trunk.001.patch, YARN-7072.trunk.002.patch, YARN-7072-trunk.003.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message