hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hitesh Shah (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-446) Container killed before hprof dumps profile.out
Date Mon, 04 Mar 2013 03:23:13 GMT

    [ https://issues.apache.org/jira/browse/YARN-446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13591958#comment-13591958

Hitesh Shah commented on YARN-446:

YARN does not know whether a container is being profiled. Are you suggesting an additional
api change in YARN to support the notion of a container being profiled?

In the scenario mentioned in the description, doesn't it make more sense for the MR application
to be changed such that the MR task process goes through a normal shutdown ( after work is
finished ) to enable itself to dump out the profiled information? Instead of the MR AM killing
the MR task/container via the ContainerManager protocol via a SIGTERM/SIGKILL?

> Container killed before hprof dumps profile.out
> -----------------------------------------------
>                 Key: YARN-446
>                 URL: https://issues.apache.org/jira/browse/YARN-446
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: client
>    Affects Versions: 2.0.3-alpha
>            Reporter: Radim Kolar
> If there is profiling enabled for mapper or reducer then hprof dumps profile.out at process
exit. It is dumped after task signaled to AM that work is finished.
> AM kills container with finished work without waiting for hprof to finish dumps. If hprof
is dumping larger outputs (such as with depth=4 while depth=3 works) , it could not finish
dump in time before being killed making entire dump unusable because cpu and heap stats are
> There needs to be better delay before container is killed if profiling is enabled.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message