hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Li Lu (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-5018) Online aggregation logic should not run immediately after collectors got started
Date Fri, 20 May 2016 00:27:12 GMT

     [ https://issues.apache.org/jira/browse/YARN-5018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Li Lu updated YARN-5018:
------------------------
    Attachment: YARN-5018-YARN-2928.001.patch

OK let me do a first-aid style fix to this problem. The problem occurs because we tried to
launch the first aggregation call immediately after the collector got started. However, some
context information is not set until postPut. This would cause the aggregation method sees
null values within the context.

This fix is a "first-aid" style fix because I simply introduced a wait (15 secs) between the
start of a collector and the first aggregation call. A more comprehensive fix would introduce
a callback function from the collector manager to collectors so that we can tell the collectors
all context data should be ready. Let me know if you have a strong preference on that one.
Thanks! 

> Online aggregation logic should not run immediately after collectors got started
> --------------------------------------------------------------------------------
>
>                 Key: YARN-5018
>                 URL: https://issues.apache.org/jira/browse/YARN-5018
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Li Lu
>            Assignee: Li Lu
>              Labels: yarn-2928-1st-milestone
>         Attachments: YARN-5018-YARN-2928.001.patch
>
>
> In app level collector, we launch the aggregation logic immediately after the collector
got started. However, at this time, important context data has yet to be published to the
container. Also, if the aggregation result is empty, we do not need to publish them.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message