hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhijie Shen (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-975) Adding HDFS implementation for Writing Interface of History Storage
Date Wed, 31 Jul 2013 07:21:53 GMT

     [ https://issues.apache.org/jira/browse/YARN-975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Zhijie Shen updated YARN-975:
-----------------------------

    Attachment: YARN-975.1.patch

Talked to Vinod. We should created a history storage that groups the reader and the writer
interfaces to be consistent. Uploaded a patch of HDFS-based implementation.

There're some addition points:

1. The reader APIs should allows throwing Throwable. For example, these APIs will also fail
due to the abnormal I/O.

2. Need to update the patch when YARN-956 is committed.

3. Need to add a test case for FileSystemHistoryStorage.

4. In this patch, we aim to make the end-to-end system work. We need to optimize the performance
of reading/writing the history data against HDFS later.
                
> Adding HDFS implementation for Writing Interface of History Storage
> -------------------------------------------------------------------
>
>                 Key: YARN-975
>                 URL: https://issues.apache.org/jira/browse/YARN-975
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Zhijie Shen
>            Assignee: Zhijie Shen
>         Attachments: YARN-975.1.patch
>
>
> HDFS implementation should be a standard persistence strategy of history storage

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message