hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Templeton (JIRA)" <j...@apache.org>
Subject [jira] [Created] (YARN-6125) The application attempt's diagnostic message should have a maximum size
Date Thu, 26 Jan 2017 18:08:25 GMT
Daniel Templeton created YARN-6125:
--------------------------------------

             Summary: The application attempt's diagnostic message should have a maximum size
                 Key: YARN-6125
                 URL: https://issues.apache.org/jira/browse/YARN-6125
             Project: Hadoop YARN
          Issue Type: Improvement
          Components: resourcemanager
    Affects Versions: 2.7.0
            Reporter: Daniel Templeton
            Assignee: Daniel Templeton
            Priority: Critical


We've found through experience that the diagnostic message can grow unbounded.  I've seen
attempts that have diagnostic messages over 1MB.  Since the message is stored in the state
store, it's a bad idea to allow the message to grow unbounded.  Instead, there should be a
property that sets a maximum size on the message.

I suspect that some of the ZK state store issues we've seen in the past were due to the size
of the diagnostic messages and not to the size of the classpath, as is the current prevailing
opinion.

An open question is how best to prune the message once it grows too large.  Should we
# truncate the tail,
# truncate the head,
# truncate the middle,
# add another property to make the behavior selectable, or
# none of the above?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message