Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: yarn-issues@hadoop.apache.org
Date: Fri, 5 Feb 2016 17:06:40 +0000 (UTC)
From: "Naganarasimha G R (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.12783032.1426714643000.313262.1454692000000@Atlassian.JIRA>
In-Reply-To: <JIRA.12783032.1426714643000@Atlassian.JIRA>
References: <JIRA.12783032.1426714643000@Atlassian.JIRA>
 <JIRA.12783032.1426714643415@arcas>
Subject: [jira] [Updated] (YARN-3367) Replace starting a separate thread for
 post entity with event loop in TimelineClient
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/YARN-3367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Naganarasimha G R updated YARN-3367:
------------------------------------
    Attachment: YARN-3367-YARN-2928.v1.011.patch

Hi [~sjlee0] & [~varun_saxena],
I have attached a patch which fixes the draining issue and have added a testcase for the same. I have done slight modification in the runnable instead of {{Thread.currentThread().isInterrupted()}} have used {{executor.isShutdown()}} as some how interrupt state was not getting set in the executor's worker thread. Hope this approach is also fine or else i am open for suggestions! (Even if we replace the test passes but the draining logs will not come !)

> Replace starting a separate thread for post entity with event loop in TimelineClient
> ------------------------------------------------------------------------------------
>
>                 Key: YARN-3367
>                 URL: https://issues.apache.org/jira/browse/YARN-3367
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>    Affects Versions: YARN-2928
>            Reporter: Junping Du
>            Assignee: Naganarasimha G R
>              Labels: yarn-2928-1st-milestone
>         Attachments: YARN-3367-YARN-2928.v1.005.patch, YARN-3367-YARN-2928.v1.006.patch, YARN-3367-YARN-2928.v1.007.patch, YARN-3367-YARN-2928.v1.008.patch, YARN-3367-YARN-2928.v1.009.patch, YARN-3367-YARN-2928.v1.010.patch, YARN-3367-YARN-2928.v1.011.patch, YARN-3367-feature-YARN-2928.003.patch, YARN-3367-feature-YARN-2928.v1.002.patch, YARN-3367-feature-YARN-2928.v1.004.patch, YARN-3367.YARN-2928.001.patch, sjlee-suggestion.patch
>
>
> Since YARN-3039, we add loop in TimelineClient to wait for collectorServiceAddress ready before posting any entity. In consumer of  TimelineClient (like AM), we are starting a new thread for each call to get rid of potential deadlock in main thread. This way has at least 3 major defects:
> 1. The consumer need some additional code to wrap a thread before calling putEntities() in TimelineClient.
> 2. It cost many thread resources which is unnecessary.
> 3. The sequence of events could be out of order because each posting operation thread get out of waiting loop randomly.
> We should have something like event loop in TimelineClient side, putEntities() only put related entities into a queue of entities and a separated thread handle to deliver entities in queue to collector via REST call.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)