hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sangjin Lee (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3367) Replace starting a separate thread for post entity with event loop in TimelineClient
Date Mon, 08 Feb 2016 18:30:39 GMT

    [ https://issues.apache.org/jira/browse/YARN-3367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15137425#comment-15137425

Sangjin Lee commented on YARN-3367:

OK I think it's real close now. I ran all the tests that are timing out locally, and they
all seem to pass.

In {{TimelineClientImpl.stop()}}, for the same reason, it would be good to add {{Thread.currentThread().interrupt()}}
to preserve the interrupt status. It's not very likely the {{stop()}} call chain is done on
an off-main thread, but it's a good practice to make it portable.

Also, the literal 2000 milliseconds are used in {{TimelineClientImpl.stop()}} and inside the
{{TimelineEntityDispatcher}} runnable. Can we define a constant for this so they never get
out of sync?

> Replace starting a separate thread for post entity with event loop in TimelineClient
> ------------------------------------------------------------------------------------
>                 Key: YARN-3367
>                 URL: https://issues.apache.org/jira/browse/YARN-3367
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Junping Du
>            Assignee: Naganarasimha G R
>              Labels: yarn-2928-1st-milestone
>         Attachments: YARN-3367-YARN-2928.v1.005.patch, YARN-3367-YARN-2928.v1.006.patch,
YARN-3367-YARN-2928.v1.007.patch, YARN-3367-YARN-2928.v1.008.patch, YARN-3367-YARN-2928.v1.009.patch,
YARN-3367-YARN-2928.v1.010.patch, YARN-3367-YARN-2928.v1.011.patch, YARN-3367-YARN-2928.v1.012.patch,
YARN-3367-feature-YARN-2928.003.patch, YARN-3367-feature-YARN-2928.v1.002.patch, YARN-3367-feature-YARN-2928.v1.004.patch,
YARN-3367.YARN-2928.001.patch, sjlee-suggestion.patch
> Since YARN-3039, we add loop in TimelineClient to wait for collectorServiceAddress ready
before posting any entity. In consumer of  TimelineClient (like AM), we are starting a new
thread for each call to get rid of potential deadlock in main thread. This way has at least
3 major defects:
> 1. The consumer need some additional code to wrap a thread before calling putEntities()
in TimelineClient.
> 2. It cost many thread resources which is unnecessary.
> 3. The sequence of events could be out of order because each posting operation thread
get out of waiting loop randomly.
> We should have something like event loop in TimelineClient side, putEntities() only put
related entities into a queue of entities and a separated thread handle to deliver entities
in queue to collector via REST call.

This message was sent by Atlassian JIRA

View raw message