hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Naganarasimha G R (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-3367) Replace starting a separate thread for post entity with event loop in TimelineClient
Date Sun, 31 Jan 2016 18:26:39 GMT

     [ https://issues.apache.org/jira/browse/YARN-3367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Naganarasimha G R updated YARN-3367:
    Attachment: YARN-3367-YARN-2928.v1.007.patch

Thanks [~sjlee0], thanks for providing the patch having changes with *FutureTask*, it was
nice to know how to use it. As earlier you had provided with a option to either use *CountdownLatch*
or *FutureTask* , and as i was aware of {{CountdownLatch}} went ahead with that. It was nice
to know how to use *FutureTask* and it was fitting aptly for this scenario.
bq. IMO I don't think it is an overkill. Sync or async, 
Earlier had just skimmed through the *FutureTask* class and found a reference to a *Thread*
with name *runner*, hence i thought it was creating a thread to publish it and mentioned it
as *overhead*. Walked through the *FutureTask* code thoroughly, to realize that it was not
spawning a thread but just holds the reference hence no problem.

Apart from other few comments which you had suggested and not included in your trial patch
have added a new Test class for V2.

Also would like to get further feedbacks on
* UGI <refer my earlier comments> or can it be handled later with Security fixes ? 
* {{TimelineClient}} class becoming bulky and little clumsy. So wanted to fork a new Class
{{TimelineClientV2Impl}} and have some sort of hierarchy with {{TimelineClientImpl}}. For
the EndUser there will be no change but just would like to split the V2 part of {{TimelineClientImpl}}
to {{TimelineClientV2Impl}} , may be after this jira goes in can fork a new jira for this
else can handle in the same jira

> Replace starting a separate thread for post entity with event loop in TimelineClient
> ------------------------------------------------------------------------------------
>                 Key: YARN-3367
>                 URL: https://issues.apache.org/jira/browse/YARN-3367
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Junping Du
>            Assignee: Naganarasimha G R
>              Labels: yarn-2928-1st-milestone
>         Attachments: YARN-3367-YARN-2928.v1.005.patch, YARN-3367-YARN-2928.v1.006.patch,
YARN-3367-YARN-2928.v1.007.patch, YARN-3367-feature-YARN-2928.003.patch, YARN-3367-feature-YARN-2928.v1.002.patch,
YARN-3367-feature-YARN-2928.v1.004.patch, YARN-3367.YARN-2928.001.patch, sjlee-suggestion.patch
> Since YARN-3039, we add loop in TimelineClient to wait for collectorServiceAddress ready
before posting any entity. In consumer of  TimelineClient (like AM), we are starting a new
thread for each call to get rid of potential deadlock in main thread. This way has at least
3 major defects:
> 1. The consumer need some additional code to wrap a thread before calling putEntities()
in TimelineClient.
> 2. It cost many thread resources which is unnecessary.
> 3. The sequence of events could be out of order because each posting operation thread
get out of waiting loop randomly.
> We should have something like event loop in TimelineClient side, putEntities() only put
related entities into a queue of entities and a separated thread handle to deliver entities
in queue to collector via REST call.

This message was sent by Atlassian JIRA

View raw message