tez-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "joeyli (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (TEZ-336) AMContainerImpl are stored even after container has completed
Date Wed, 28 Aug 2013 03:46:10 GMT

    [ https://issues.apache.org/jira/browse/TEZ-336?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13752080#comment-13752080

joeyli commented on TEZ-336:

Looks like the most heavy object ContainerLaunchContext has already been discarded in LaunchRequestTransition.
Do we still need to do this jira?
      container.clc = AMContainerHelpers.createContainerLaunchContext(
          container.taskAttemptListener, containerContext.getCredentials(),
          event.shouldProfile(), container.appContext);

      // Registering now, so that in case of delayed NM response, the child
      // task is not told to die since the TAL does not know about the container.
      LOG.info("Sending Launch Request for Container with id: " +
      // Forget about the clc to save resources. At some point, part of the clc
      // info may need to be exposed to the scheduler to figure out whether a
      // container can be used for a specific TaskAttempt.
      container.clc = null;
> AMContainerImpl are stored even after container has completed
> -------------------------------------------------------------
>                 Key: TEZ-336
>                 URL: https://issues.apache.org/jira/browse/TEZ-336
>             Project: Apache Tez
>          Issue Type: Bug
>            Reporter: Bikas Saha
>            Assignee: joeyli
>              Labels: TEZ-0.2.0
>             Fix For: 0.2.0
>         Attachments: TEZ-336.3.patch
> For a long running job thats allocated 100's of 1000's of containers this can be a memory
usage issue. Even if AMContainerImpls are small they contain full copies of container launch
context objects that look like heavy weights with multiple maps of local resources, env etc.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message