Mailing-List: contact issues-help@ambari.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@ambari.apache.org
Date: Mon, 10 Apr 2017 14:53:41 +0000 (UTC)
From: "Hudson (JIRA)" <jira@apache.org>
To: issues@ambari.apache.org
Message-ID: <JIRA.13061443.1491315867000.248356.1491836021673@Atlassian.JIRA>
In-Reply-To: <JIRA.13061443.1491315867000@Atlassian.JIRA>
References: <JIRA.13061443.1491315867000@Atlassian.JIRA> <JIRA.13061443.1491315867422@jira-lw-us.apache.org>
Subject: [jira] [Commented] (AMBARI-20670) Node manager start extremely slow
 when YARN NM local dirs are very large
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Mon, 10 Apr 2017 14:53:46 -0000


    [ https://issues.apache.org/jira/browse/AMBARI-20670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15962987#comment-15962987 ] 

Hudson commented on AMBARI-20670:
---------------------------------

FAILURE: Integrated in Jenkins build Ambari-trunk-Commit #7260 (See [https://builds.apache.org/job/Ambari-trunk-Commit/7260/])
AMBARI-20670. Node manager start extremely slow when YARN NM local dirs (echekanskiy: [http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=753b38c74723e2ef24ead5112dacb54b127c0d1a])
* (edit) ambari-server/src/test/python/stacks/2.0.6/YARN/test_nodemanager.py


> Node manager start extremely slow when YARN NM local dirs are very large
> ------------------------------------------------------------------------
>
>                 Key: AMBARI-20670
>                 URL: https://issues.apache.org/jira/browse/AMBARI-20670
>             Project: Ambari
>          Issue Type: Bug
>          Components: ambari-server
>    Affects Versions: 2.5.1
>            Reporter: Dmytro Grinenko
>            Assignee: Dmytro Grinenko
>            Priority: Critical
>             Fix For: trunk, 2.5.1
>
>         Attachments: AMBARI-20670-2.5.patch, AMBARI-20670.patch, AMBARI-20670-ut-fix.patch
>
>
> On the cluster with the YARN NM, where local dirs are 100 GB+ with lot of small files - NM starts slow with timeouts
> Reason could be in this specific call in  yarn.py
> {code}
> def create_local_dir(dir_name):
>   import params
>   Directory(dir_name,
>             create_parents = True,
>             cd_access="a",
>             mode=0755,
>             owner=params.yarn_user,
>             group=params.user_group,
>             ignore_failures=True,
>             recursive_mode_flags = {'f': 'a+rw', 'd': 'a+rwx'},
>   )
> {code}
> was taking ~15 minutes per mount.


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)