hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Varun Saxena (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2962) ZKRMStateStore: Limit the number of znodes under a znode
Date Wed, 26 Aug 2015 21:09:46 GMT

    [ https://issues.apache.org/jira/browse/YARN-2962?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14715519#comment-14715519

Varun Saxena commented on YARN-2962:

bq. how many applications did you have in the RM store before this became a problem
Will have to check. I think it was more than 10000 apps in our case. Will let you know.

bq. switching the zk max messages size via -Djute.maxbuffer=<bytes> a viable workaround?
Yes, that works. Also we can set a lower config value for number of completed apps to be stored
in state store. Even 0 can be set.

bq.  Also, is there a sense of how close this ticket is to being merged? 
The patches currently here have to be rebased because of recent changes. Had put this on the
back burner as this will go in trunk and not in branch-2. If its required to be handled earlier,
I will focus on it. Plan to take this up in the coming month anyways.

> ZKRMStateStore: Limit the number of znodes under a znode
> --------------------------------------------------------
>                 Key: YARN-2962
>                 URL: https://issues.apache.org/jira/browse/YARN-2962
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: resourcemanager
>    Affects Versions: 2.6.0
>            Reporter: Karthik Kambatla
>            Assignee: Varun Saxena
>            Priority: Critical
>         Attachments: YARN-2962.01.patch, YARN-2962.2.patch, YARN-2962.3.patch
> We ran into this issue where we were hitting the default ZK server message size configs,
primarily because the message had too many znodes even though they individually they were
all small.

This message was sent by Atlassian JIRA

View raw message