hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andras Piros (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (YARN-6125) The application attempt's diagnostic message should have a maximum size
Date Wed, 01 Feb 2017 21:41:51 GMT

    [ https://issues.apache.org/jira/browse/YARN-6125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15848968#comment-15848968
] 

Andras Piros edited comment on YARN-6125 at 2/1/17 9:40 PM:
------------------------------------------------------------

[~templedf] thanks for the review!

My thoughts on the comments:
# done
# done
# the {{pom.xml}} dependency reordering was necessary in order to get {{ExpectedException}}
working. There are other parts of {{hadoop}} that employ the same thing (notably, {{hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-nodemanager/pom.xml}}),
essentially to make sure the correct Hamcrest version is on the classpath. One day switching
to Mockito 2.x [*should solve this problem*|https://github.com/mockito/mockito/issues/324]
# {{Lists.newLinkedList()}} comes from Guava. But nevermind, changed to use {{new LinkedList<>()}}
# done
# done
# done
# I like {{final}} stuff, please specify where I should make variables / fields non-{{final}}
# WIP
# WIP
# done
# done
# done
# done


was (Author: andras.piros):
[~templedf] thanks for the review!

My thoughts on the comments:
# done
# done
# the {{pom.xml}} dependency reordering was necessary in order to get {{ExpectedException}}
working. There are other parts of {{hadoop}} that employ the same thing (notably, {{hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-nodemanager/pom.xml}}),
essentially to make sure the correct Hamcrest version is on the classpath. One day switching
to Mockito 2.x [*should solve this problem*|https://github.com/mockito/mockito/issues/324]
# {{Lists.newLinkedList()}} comes from Guava. But nevermind, changed to use {{new LinkedList{}()}}
# done
# done
# done
# I like {{final}} stuff, please specify where I should make variables / fields non-{{final}}
# WIP
# WIP
# done
# done
# done
# done

> The application attempt's diagnostic message should have a maximum size
> -----------------------------------------------------------------------
>
>                 Key: YARN-6125
>                 URL: https://issues.apache.org/jira/browse/YARN-6125
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: resourcemanager
>    Affects Versions: 2.7.0
>            Reporter: Daniel Templeton
>            Assignee: Andras Piros
>            Priority: Critical
>             Fix For: 3.0.0-alpha3
>
>         Attachments: YARN-6125.000.patch, YARN-6125.001.patch, YARN-6125.002.patch
>
>
> We've found through experience that the diagnostic message can grow unbounded.  I've
seen attempts that have diagnostic messages over 1MB.  Since the message is stored in the
state store, it's a bad idea to allow the message to grow unbounded.  Instead, there should
be a property that sets a maximum size on the message.
> I suspect that some of the ZK state store issues we've seen in the past were due to the
size of the diagnostic messages and not to the size of the classpath, as is the current prevailing
opinion.
> An open question is how best to prune the message once it grows too large.  Should we
> # truncate the tail,
> # truncate the head,
> # truncate the middle,
> # add another property to make the behavior selectable, or
> # none of the above?



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message