hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsuyoshi OZAWA (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-1307) Rethink znode structure for RM HA
Date Wed, 13 Nov 2013 10:21:31 GMT

     [ https://issues.apache.org/jira/browse/YARN-1307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Tsuyoshi OZAWA updated YARN-1307:

    Attachment: YARN-1307.5.patch

Thank you for review, [~jianhe]. Updated patch based on your review comments.
1. Removed version info integer. This should be done in YARN-1239.
2. Removed needless class and methods - createRootDir(String rootPath, byte[] data) and ZNode.
3. Fixed to close Stream correctly
4. Refactored current state store internal API to take ApplicationId/ApplicationAttemptId
instead of its string value. 

This new method iterates the whole app directory again while loadRMAppState method is already
iterating the whole app directory, unnecessary overhead.

IIUC, loadRMAppState just read application id's nodes under RM_APP_ROOT. loadApplicationAttemptState
read application attempt id's node under the appId.

     |--- VERSION_INFO
     |--- RM_APP_ROOT
     |     |----- (#ApplicationId1)
     |     |        |----- (#ApplicationAttemptIds)
     |     |
     |     |----- (#ApplicationId2)
     |     |       |----- (#ApplicationAttemptIds)
     |     ....

> Rethink znode structure for RM HA
> ---------------------------------
>                 Key: YARN-1307
>                 URL: https://issues.apache.org/jira/browse/YARN-1307
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>            Reporter: Tsuyoshi OZAWA
>            Assignee: Tsuyoshi OZAWA
>         Attachments: YARN-1307.1.patch, YARN-1307.2.patch, YARN-1307.3.patch, YARN-1307.4-2.patch,
YARN-1307.4-3.patch, YARN-1307.4.patch, YARN-1307.5.patch
> Rethink for znode structure for RM HA is proposed in some JIRAs(YARN-659, YARN-1222).
The motivation of this JIRA is quoted from Bikas' comment in YARN-1222:
> {quote}
> We should move to creating a node hierarchy for apps such that all znodes for an app
are stored under an app znode instead of the app root znode. This will help in removeApplication
and also in scaling better on ZK. The earlier code was written this way to ensure create/delete
happens under a root znode for fencing. But given that we have moved to multi-operations globally,
this isnt required anymore.
> {quote}

This message was sent by Atlassian JIRA

View raw message