hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsuyoshi OZAWA (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-2517) Implement TimelineClientAsync
Date Mon, 08 Sep 2014 14:43:29 GMT

     [ https://issues.apache.org/jira/browse/YARN-2517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Tsuyoshi OZAWA updated YARN-2517:
    Attachment: YARN-2517.1.patch

Attached a first patch for review. The differences between TimelineClientAsync and TimelineClient
are as follows:

* TimelineClientAsyncImpl has 2 blocking queues and 2 threads: {{requestQueue}} is for queuing
requests from {{TimelineClientAsync#putEntities}}. {{responseQueue}} is for queueing responses
and errors from {{TimelineClientImpl#putEntities}}. {{dispatcherThread}} deques requests from
{{requestQueue}} and dispatches requests to TimelineServer. {{handlerThread}} deques results
of {{TimelineClient#putEntities}} and callback user-defined methods defined in CallbackHandler.
* CallbackHandler has two APIs for users: onEntitiesPut is a API for receiving results of
putEntities and onError is a API for handling errors. If Configuration#TIMELINE_SERVICE_ENABLED
is false, results of putEntities are returned via Callback#onEntitiesPut.
* {{void TimelineClientAsync#putEntities}} can throw InterruptedException because it uses
{{BlockingQueue#put}} in {{TimelineClientAsyncImpl}}, though I think it's not blocked basically
because the queue length is configured as Integer.MAX_VALUE. We can add a configuration for
controlling memory consumption of the queues.

> Implement TimelineClientAsync
> -----------------------------
>                 Key: YARN-2517
>                 URL: https://issues.apache.org/jira/browse/YARN-2517
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Zhijie Shen
>            Assignee: Tsuyoshi OZAWA
>         Attachments: YARN-2517.1.patch
> In some scenarios, we'd like to put timeline entities in another thread no to block the
current one.
> It's good to have a TimelineClientAsync like AMRMClientAsync and NMClientAsync. It can
buffer entities, put them in a separate thread, and have callback to handle the responses.

This message was sent by Atlassian JIRA

View raw message