hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bikas Saha (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-353) Add Zookeeper-based store implementation for RMStateStore
Date Thu, 19 Sep 2013 20:35:54 GMT

    [ https://issues.apache.org/jira/browse/YARN-353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13772275#comment-13772275
] 

Bikas Saha commented on YARN-353:
---------------------------------

We should be storing the information inside the data of the znode instead of encoding it in
the name of the znode. The rename pattern was followed as an optimization in FSStore but is
not necessary in ZKStore give set-with-overwrite operations. Its also better to avoid encoding
the info in the name since doing "ls" in ZK is more relaxed wrt permissions than it is in
FileSystem.
{code}
+      if (childNodeName.startsWith(DELEGATION_TOKEN_SEQUENCE_NUMBER_PREFIX)) {
+        rmState.rmSecretManagerState.dtSequenceNumber =
+            Integer.parseInt(childNodeName.split("_")[1])
{code}

We should move to creating a node hierarchy for apps such that all znodes for an app are stored
under an app znode instead of the app root znode. This will help in removeApplication and
also in scaling better on ZK. The earlier code was written this way to ensure create/delete
happens under a root znode for fencing. But given that we have moved to multi-operations globally,
this isnt required anymore.

storeRMDTMasterKeyState() is not using multi-operation. We should be issuing every action
on ZK using a multi-operation for fencing to work.

I am fine with the patch as is since it puts the framework in place. The above comments may
be addressed in YARN-1222 thats opened under YARN-149 for HA. YARN-1222 blocks YARN-1026.

                
> Add Zookeeper-based store implementation for RMStateStore
> ---------------------------------------------------------
>
>                 Key: YARN-353
>                 URL: https://issues.apache.org/jira/browse/YARN-353
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>            Reporter: Hitesh Shah
>            Assignee: Karthik Kambatla
>         Attachments: YARN-353.10.patch, YARN-353.11.patch, YARN-353.12.patch, yarn-353-12-wip.patch,
YARN-353.13.patch, YARN-353.14.patch, YARN-353.15.patch, YARN-353.16.1.patch, YARN-353.16.patch,
YARN-353.1.patch, YARN-353.2.patch, YARN-353.3.patch, YARN-353.4.patch, YARN-353.5.patch,
YARN-353.6.patch, YARN-353.7.patch, YARN-353.8.patch, YARN-353.9.patch
>
>
> Add store that write RM state data to ZK

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message