hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bikas Saha (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1222) Make improvements in ZKRMStateStore for fencing
Date Tue, 05 Nov 2013 18:23:17 GMT

    [ https://issues.apache.org/jira/browse/YARN-1222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13814092#comment-13814092

Bikas Saha commented on YARN-1222:

bq. The root-node ACLs are per RM instance. They need to be different for it to work. The
documentation in yarn-default.xml explains this - we might have to make it even more clear?
Clarifying it, possibly with an example would be good.

bq. The number of ACLs in the list is always bounded by (user-configured-for-store + 1). Am
I missing something?
I missed that the patch is modifying the base acl from config and not the actual acl from
the znode. The latter would have increased the count. The former is fine. The current code
is good.

Where is the shared rm-admin-acl being set such that both RMs have admin access to the root
znode? This probably works because the default is world:all. But if that is not the case,
and we are using internally generated acls, then the rm has to give shared admin access to
the other rm when it creates the root znode, right?

bq. Do you think we should make it aware of fencing - have something like a StoreFencedException?
I think it should be aware of when the store is not available to it because it has been fenced
out. There are/were comments in state store error handling to differentiate between exceptions
when we have such a differentiation. So we should create a Fenced exception (look at HDFS
code for an example). This way all state store should be able to return this incident for
identical handling in the upper layers. We would like to avoid state store impls (which are
technically runtime pluggable pieces) to have to understand internal Hadoop code patterns
for HA etc.

bq. ZKRMStateStore itself is @Private @Unstable. Should we still label the methods @Private?
At some point ZKRMStateStore will become public/stable but these methods should remain private
for testing, right?

> Make improvements in ZKRMStateStore for fencing
> -----------------------------------------------
>                 Key: YARN-1222
>                 URL: https://issues.apache.org/jira/browse/YARN-1222
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Bikas Saha
>            Assignee: Karthik Kambatla
>         Attachments: yarn-1222-1.patch, yarn-1222-2.patch, yarn-1222-3.patch, yarn-1222-4.patch
> Using multi-operations for every ZK interaction. 
> In every operation, automatically creating/deleting a lock znode that is the child of
the root znode. This is to achieve fencing by modifying the create/delete permissions on the
root znode.

This message was sent by Atlassian JIRA

View raw message