ambari-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AMBARI-20670) Node manager start extremely slow when YARN NM local dirs are very large
Date Mon, 10 Apr 2017 14:55:41 GMT

    [ https://issues.apache.org/jira/browse/AMBARI-20670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15962989#comment-15962989
] 

Hudson commented on AMBARI-20670:
---------------------------------

FAILURE: Integrated in Jenkins build Ambari-branch-2.5 #1377 (See [https://builds.apache.org/job/Ambari-branch-2.5/1377/])
AMBARI-20670. Node manager start extremely slow when YARN NM local dirs (echekanskiy: [http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=10da7925debade3610d1e500aeec08d125cdf2c8])
* (edit) ambari-server/src/test/python/stacks/2.0.6/YARN/test_nodemanager.py


> Node manager start extremely slow when YARN NM local dirs are very large
> ------------------------------------------------------------------------
>
>                 Key: AMBARI-20670
>                 URL: https://issues.apache.org/jira/browse/AMBARI-20670
>             Project: Ambari
>          Issue Type: Bug
>          Components: ambari-server
>    Affects Versions: 2.5.1
>            Reporter: Dmytro Grinenko
>            Assignee: Dmytro Grinenko
>            Priority: Critical
>             Fix For: trunk, 2.5.1
>
>         Attachments: AMBARI-20670-2.5.patch, AMBARI-20670.patch, AMBARI-20670-ut-fix.patch
>
>
> On the cluster with the YARN NM, where local dirs are 100 GB+ with lot of small files
- NM starts slow with timeouts
> Reason could be in this specific call in  yarn.py
> {code}
> def create_local_dir(dir_name):
>   import params
>   Directory(dir_name,
>             create_parents = True,
>             cd_access="a",
>             mode=0755,
>             owner=params.yarn_user,
>             group=params.user_group,
>             ignore_failures=True,
>             recursive_mode_flags = {'f': 'a+rw', 'd': 'a+rwx'},
>   )
> {code}
> was taking ~15 minutes per mount.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message