hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tsuyoshi OZAWA (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-1307) Rethink znode structure for RM HA
Date Tue, 22 Oct 2013 08:51:42 GMT

     [ https://issues.apache.org/jira/browse/YARN-1307?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Tsuyoshi OZAWA updated YARN-1307:
---------------------------------

    Attachment: YARN-1307.1.patch

Created a first patch for this JIRA with new znode structure. If we should merge YARN-1222
at first, I'll rebase this patch on YARN-1222's patch. If this JIRA is first to merge, I'll
help Karthik's work to rebase.

I found some discussion points:
1. Serialization format. Should we use protobuf to define to represent contents of znodes?
For now, I use DataOutputStream#write*().
2. Contents of znodes. I'm using a format as follows:

versionInfo format:
{code}
----------------
int    # version number
----------------
{code}


RMDelegationKey format:
{code}
----------------
MasterKey 1 # Internal format is defined in DelegationKey#write()
----------------
MasterKey 2
----------------
….
----------------
MasterKey n
----------------
{code}


RMDelegationToken format:
{code}
----------------
rmDTIdentifier 1 # Internal format is defined in RMDelegationTokenIdentifier#write()
----------------
renew date1 # long
----------------
rmDTIdentifier 2
----------------
renew date 2
----------------
….
----------------
rmDTIdentifier 3
----------------
renew date 3
----------------
{code}

RMDTSequenceNumber format:
{code}
----------------
int # sequence number
----------------
{code}

> Rethink znode structure for RM HA
> ---------------------------------
>
>                 Key: YARN-1307
>                 URL: https://issues.apache.org/jira/browse/YARN-1307
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>            Reporter: Tsuyoshi OZAWA
>            Assignee: Tsuyoshi OZAWA
>         Attachments: YARN-1307.1.patch
>
>
> Rethink for znode structure for RM HA is proposed in some JIRAs(YARN-659, YARN-1222).
The motivation of this JIRA is quoted from Bikas' comment in YARN-1222:
> {quote}
> We should move to creating a node hierarchy for apps such that all znodes for an app
are stored under an app znode instead of the app root znode. This will help in removeApplication
and also in scaling better on ZK. The earlier code was written this way to ensure create/delete
happens under a root znode for fencing. But given that we have moved to multi-operations globally,
this isnt required anymore.
> {quote}



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message