hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Naganarasimha G R (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3367) Replace starting a separate thread for post entity with event loop in TimelineClient
Date Wed, 23 Sep 2015 01:10:05 GMT

    [ https://issues.apache.org/jira/browse/YARN-3367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14903754#comment-14903754

Naganarasimha G R commented on YARN-3367:

Thanks [~gtCarrera9], 
For looking into this, There were lot of open questions for this jira from my end which i
mentioned [earlier|https://issues.apache.org/jira/browse/YARN-3367?focusedCommentId=14732065&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14732065].
Hence just started with initial approach path. Mainly to use the refactor and reuse the existing
{{AsyncDispatcher}} we need to discuss on below specific points : 
* 3 Is it important to maintain the order of events which are sent from sync and async ? i.e.
Is it req to ensure all the async events are also pushed along with the current sync event
or is it ok to send only the sync ? (current patch just ensures async events are in order)
* 4 Whether its req to merge entities of multiple async calls as they belong to same application
If concluded then i can analyze further and inform.

> Replace starting a separate thread for post entity with event loop in TimelineClient
> ------------------------------------------------------------------------------------
>                 Key: YARN-3367
>                 URL: https://issues.apache.org/jira/browse/YARN-3367
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Junping Du
>            Assignee: Naganarasimha G R
>         Attachments: YARN-3367.YARN-2928.001.patch
> Since YARN-3039, we add loop in TimelineClient to wait for collectorServiceAddress ready
before posting any entity. In consumer of  TimelineClient (like AM), we are starting a new
thread for each call to get rid of potential deadlock in main thread. This way has at least
3 major defects:
> 1. The consumer need some additional code to wrap a thread before calling putEntities()
in TimelineClient.
> 2. It cost many thread resources which is unnecessary.
> 3. The sequence of events could be out of order because each posting operation thread
get out of waiting loop randomly.
> We should have something like event loop in TimelineClient side, putEntities() only put
related entities into a queue of entities and a separated thread handle to deliver entities
in queue to collector via REST call.

This message was sent by Atlassian JIRA

View raw message