hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhijie Shen (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-934) Defining Writing Interface of HistoryStorage
Date Thu, 25 Jul 2013 00:45:49 GMT

     [ https://issues.apache.org/jira/browse/YARN-934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Zhijie Shen updated YARN-934:

    Attachment: YARN-934.2.patch

Updated the patch:

1. The abstract methods have the protobuf classes as the parameters. Therefore, the implementations
of them just need to persist the protobuf objects. HistoryWriter already has the logic to
extract the required information from RMApp, RMAppAttempt and RMContainer to fill the fields
of these protobuf objects. Please note that RMContainer now doesn't have all the required
information to persist. We'll work on it a separate ticket.

2. Change from multi-thread model to using the dispatcher on a single thread. Thought about
it again. IMHO, users know the problem best. If users think the single dispatcher thread is
the bottleneck of persisting history data, they can fork more worker threads in the implementations
of the the abstract methods.

3. Add the test file for HistoryWriter
> Defining Writing Interface of HistoryStorage
> --------------------------------------------
>                 Key: YARN-934
>                 URL: https://issues.apache.org/jira/browse/YARN-934
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Zhijie Shen
>            Assignee: Zhijie Shen
>         Attachments: YARN-934.1.patch, YARN-934.2.patch

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message