hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Varun Saxena (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2962) ZKRMStateStore: Limit the number of znodes under a znode
Date Tue, 07 Jun 2016 19:29:21 GMT

    [ https://issues.apache.org/jira/browse/YARN-2962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15319183#comment-15319183
] 

Varun Saxena commented on YARN-2962:
------------------------------------

Thanks [~asuresh]. 
Seems patch is not applying. Will upload a patch after rebasing to trunk and fixing Daniel's
comments by tomorrow(as its late night here). Maybe you can have a look then.
It would be good if we can get this into 3.0.0-alpha because we were thinking of including
this fix in our private code.

I think one part which mainly needed a discussion on, was how do we delete the parent application
node(application is now split into 2 nodes) if it contains no children. This check is currently
done when application is being removed. We are not having the whole operation under a single
fencing as we have to check number children after deletion. If 2 RMs' can ever become active
at same time, this can potentially lead to a race. Maybe we can just swallow NotEmptyException
during deletion of parent.

> ZKRMStateStore: Limit the number of znodes under a znode
> --------------------------------------------------------
>
>                 Key: YARN-2962
>                 URL: https://issues.apache.org/jira/browse/YARN-2962
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: resourcemanager
>    Affects Versions: 2.6.0
>            Reporter: Karthik Kambatla
>            Assignee: Varun Saxena
>            Priority: Critical
>         Attachments: YARN-2962.01.patch, YARN-2962.04.patch, YARN-2962.2.patch, YARN-2962.3.patch
>
>
> We ran into this issue where we were hitting the default ZK server message size configs,
primarily because the message had too many znodes even though they individually they were
all small.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message