hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bibin A Chundatt (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6825) RM quit due to ApplicationStateData exceed the limit size of znode in zk
Date Fri, 14 Jul 2017 13:19:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-6825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16087303#comment-16087303
] 

Bibin A Chundatt commented on YARN-6825:
----------------------------------------

{quote}
I think 80% of configured jute buffer should be taken implicitly rather than allowing admin
to configure 80% of jute buffer. 
{quote}
Since the default value of jute buffer size is currently 1MB , I agree we can make the default
value as {{.8MB}}. So does this solve the issues you have mentioned??
I had in mind the cases mentioned in this jira during 5006 missed to explicitly mention in
YARN-5006.

Do you find any other scenario which couldn't be handled by configuration??  
So are we good to go ahead with YARN-6819?  If its fine with you , will handle the  default
value change also in same jira..

> RM quit due to ApplicationStateData exceed the limit size of znode in zk
> ------------------------------------------------------------------------
>
>                 Key: YARN-6825
>                 URL: https://issues.apache.org/jira/browse/YARN-6825
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: resourcemanager
>            Reporter: Rohith Sharma K S
>
> YARN-5006 fixes this issue by strict validation for ApplicationStateData length against
1MB(default max jute buffer) during application submission only. There is possibility of thrashing
as dead zone was not properly defined/taken care.
> But it do not consider scenarios where ApplicationStateData can be increased later point
of time i.e 
> # If app is submitted with less than 1MB during submission, later updated like queue
name or life time value or priority is changed. The app update call will be sent to statestore
which cause same issue because ApplicationStateData length has increased.
> # Consider there is no app update, but final state are stored in ZK. This adds up several
fields such finishTime, finalState, finalApplicationState. This increases size of ApplicationStateData.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message