hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Haibo Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6277) Nodemanager heap memory leak
Date Fri, 31 Mar 2017 04:44:41 GMT

    [ https://issues.apache.org/jira/browse/YARN-6277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15950316#comment-15950316
] 

Haibo Chen commented on YARN-6277:
----------------------------------

Thanks [~Feng Yuan] for reporting the issue and working on a patch! If I follow you correctly,
the AllocatorPerContext instance for NM_LOCAL_DIR is global, and because ShuffleHandler and
NM are not sharing the same configuration object, ShuffleHandler does not see the change if
the local directory is changed in NM, resulting in ShuffleHandler and NM thinking of different
values for NM_LOCAL_DIR
{code}
    private Context confChanged(Configuration conf) throws IOException {
      ........
      if (!newLocalDirs.equals(ctx.savedLocalDirs)) {
          ctx = new Context();
          String[] dirStrings = StringUtils.getTrimmedStrings(newLocalDirs);
          ctx.localFS = FileSystem.getLocal(conf);
          ....
          ctx.savedLocalDirs = newLocalDirs;
      }
    }
    ...
{code}
The if statement will always evaluate to true if the other has executed confChanged() previously,
so we have this thrashing issue? But looking at FileSystem.getLocal() implementation though,
it seems like that only when caching for local file system is disabled, will massive number
of LocalFileSystem instances be created. Can you confirm is that your setting?

> Nodemanager heap memory leak
> ----------------------------
>
>                 Key: YARN-6277
>                 URL: https://issues.apache.org/jira/browse/YARN-6277
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: nodemanager
>    Affects Versions: 2.7.3, 2.8.1, 3.0.0-alpha2
>            Reporter: Feng Yuan
>            Assignee: Feng Yuan
>         Attachments: YARN-6277.branch-2.8.001.patch
>
>
> Because LocalDirHandlerService@LocalDirAllocator`s mechanism,they will create massive
LocalFileSystem.So lead to heap leak.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message