Mailing-List: contact issues-help@ambari.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@ambari.apache.org
Date: Wed, 9 Aug 2017 13:27:00 +0000 (UTC)
From: "Doroszlai, Attila (JIRA)" <jira@apache.org>
To: issues@ambari.apache.org
Message-ID: <JIRA.13074607.1495651854000.127733.1502285220381@Atlassian.JIRA>
In-Reply-To: <JIRA.13074607.1495651854000@Atlassian.JIRA>
References: <JIRA.13074607.1495651854000@Atlassian.JIRA> <JIRA.13074607.1495651854628@jira-lw-us.apache.org>
Subject: [jira] [Commented] (AMBARI-21116) Setting
 yarn.node-labels.fs-store.root-dir to a "path" changes the permission of
 the "root path"
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Wed, 09 Aug 2017 13:27:05 -0000


    [ https://issues.apache.org/jira/browse/AMBARI-21116?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16119892#comment-16119892 ] 

Doroszlai, Attila commented on AMBARI-21116:
--------------------------------------------

[~aonishuk], found one more problem with [this commit|https://github.com/apache/ambari/commit/4405d4982842a729ebfbd4c000c5ffb023abbbb5].  In HDP 2.X recursive permissions were eliminated for the wrong directory: {{mapreduce_jobhistory_done_dir}} instead of {{node_labels_dir}}.

Compare:

{noformat:title=YARN/2.1.0.2.0/package/scripts/yarn.py}
   params.HdfsResource(params.mapreduce_jobhistory_done_dir,
                        type="directory",
                         action="create_on_execute",
                         owner=params.mapred_user,
                         group=params.user_group,
 -                       change_permissions_for_parents=True,
                         mode=0777
    )
{noformat}

with:

{noformat:title=YARN/3.0.0.3.0/package/scripts/yarn.py}
     params.HdfsResource(params.node_labels_dir,
                           type="directory",
                           action="create_on_execute",
 -                         change_permissions_for_parents=True,
                           owner=params.yarn_user,
                           group=params.user_group,
                           mode=0700
     )
{noformat}

This causes MapReduce History Server to stop after startup, because it cannot create {{/mr-history/tmp}} due to wrong permissions on {{/mr-history}}:

{noformat:title=trunk}
drwxr-xr-x   - hdfs   hdfs            0 2017-08-09 11:54 /mr-history
drwxrwxrwx   - mapred hadoop          0 2017-08-09 11:54 /mr-history/done
{noformat}

{noformat:title=branch-2.5}
drwxrwxrwx   - mapred hadoop          0 2017-08-09 12:26 /mr-history
drwxrwxrwx   - mapred hadoop          0 2017-08-09 12:26 /mr-history/done
drwxrwxrwt   - mapred hadoop          0 2017-08-09 12:26 /mr-history/tmp
{noformat}

{noformat:title=mapred-mapred-historyserver.log}
2017-08-09 11:54:20,957 INFO  service.AbstractService (AbstractService.java:noteFailure(272)) - Service org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager failed in state INITED; cause: org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Error creating intermediate done directory
...
Caused by: org.apache.hadoop.security.AccessControlException: Permission denied: user=mapred, access=WRITE, inode="/mr-history/tmp":hdfs:hdfs:drwxr-xr-x
...
2017-08-09 11:54:20,971 INFO  util.ExitUtil (ExitUtil.java:terminate(124)) - Exiting with status -1
{noformat}

> Setting yarn.node-labels.fs-store.root-dir to a "path" changes the permission of the "root path"
> ------------------------------------------------------------------------------------------------
>
>                 Key: AMBARI-21116
>                 URL: https://issues.apache.org/jira/browse/AMBARI-21116
>             Project: Ambari
>          Issue Type: Bug
>            Reporter: Sumana Sathish
>            Assignee: Andrew Onischuk
>            Priority: Critical
>             Fix For: 3.0.0
>
>
> 1. Set the following configs to run nodeLabels test via Ambari Rest Call
> {code}
>  yarnProperties = {'yarn.acl.enable': 'true',
>                       'yarn.node-labels.enabled' : "True",
>                       'yarn.node-labels.fs-store.root-dir': NODE_LABEL_STORE_DIR,
>                       'yarn.admin.acl': yarn_user + ',' + qa_user}
> {code}
> where NODE_LABEL_STORE_DIR = "/tmp/node-labels"
> 2Restart ResourceManager and NodeManagers via ambari 
> 3. After running these commands,  /tmp directory's permission changes to 
> [drwx------   - yarn   hadoop] from [drwxrwxrwx   - hdfs   hadoop]  causing other tests to fail since it cannot access /tmp  


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)