hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Li Lu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5018) Online aggregation logic should not run immediately after collectors got started
Date Tue, 24 May 2016 00:15:13 GMT

    [ https://issues.apache.org/jira/browse/YARN-5018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15297397#comment-15297397

Li Lu commented on YARN-5018:

The current situation is a little bit nasty... In the original version we firstly called collector.start,
then put it in the collector map, because we do not want to publish any containers that are
not yet started (although "started" is only a trivial status change). If we start a collector
after putting it into the collector map, then anyone getting collectors may get collectors
that are not yet started. 

So the problem is that without a separate collector.postPut, we have the following dependencies
on TimelineCollectorManager#putIfAbsent:
1. Collector start happens before putting the collector into the map
2. When postPut is called, the collector is in the map (I'm not 100% sure about this assumption)
3. Aggregation should only run after postPut is called. 

Now I'm not very sure about 2. Breaking this assumption seems to be the best bet for us if
we don't like the extra postPut (I don't like it, frankly speaking. ). 

> Online aggregation logic should not run immediately after collectors got started
> --------------------------------------------------------------------------------
>                 Key: YARN-5018
>                 URL: https://issues.apache.org/jira/browse/YARN-5018
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Li Lu
>            Assignee: Li Lu
>              Labels: yarn-2928-1st-milestone
>         Attachments: YARN-5018-YARN-2928.001.patch, YARN-5018-YARN-2928.002.patch, YARN-5018-YARN-2928.003.patch,
> In app level collector, we launch the aggregation logic immediately after the collector
got started. However, at this time, important context data has yet to be published to the
container. Also, if the aggregation result is empty, we do not need to publish them.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message