hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rakesh R (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2962) ZKRMStateStore: Limit the number of znodes under a znode
Date Tue, 16 Dec 2014 08:08:13 GMT

    [ https://issues.apache.org/jira/browse/YARN-2962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14247916#comment-14247916
] 

Rakesh R commented on YARN-2962:
--------------------------------

Hi [~kasha], I'm interested to know more about this. Could you tell me any exception messages
you are getting. From the description I'm thinking you are talking about the "jute.maxbuffer"
java system property of ZooKeeper(defaulting to 1MB data size). As I know this property is
used in the following cases:
# znode data size - user added large data on a znode and submitting request to the ZooKeeper
server.
# say if there are n children under one znode(say /myapp). Now when user call zkc#getChildren(/myapp),
server will get all the children and put into a list. This list will be added to the response
object. If the list exceeds 1MB size (characters of all the znode names) will validate against
the size and throws exception. 

I hope you are talking about 2nd case?. Thanks!

> ZKRMStateStore: Limit the number of znodes under a znode
> --------------------------------------------------------
>
>                 Key: YARN-2962
>                 URL: https://issues.apache.org/jira/browse/YARN-2962
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: resourcemanager
>    Affects Versions: 2.6.0
>            Reporter: Karthik Kambatla
>            Assignee: Varun Saxena
>            Priority: Critical
>
> We ran into this issue where we were hitting the default ZK server message size configs,
primarily because the message had too many znodes even though they individually they were
all small.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message