hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Haibo Chen (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-5001) Aggregated Logs root directory is created with wrong group if nonexistent
Date Thu, 05 May 2016 19:11:12 GMT

     [ https://issues.apache.org/jira/browse/YARN-5001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Haibo Chen updated YARN-5001:
-----------------------------
    Description: The directory /tmp/logs, where the aggregated logs go, is supposed to be
read by JHS. But if it is not created beforehand, it will be created by NodeManager with group
being the superuser set in HDFS. This leads to JHS to fail to read the container logs files
under that directory if JHS is not running as a user that belongs to superuser.  (was: Usually,
the group owner for /tmp/logs, where the aggregated logs go, is "hadoop". Under that dir,
you then have <username>/logs/<application_id>/<files_per_node> with group
being "hadoop" all the way down. 

If you delete the /tmp/logs dir (when you want to clean up all the logs), the directory will
be created with a different group "superuser". The JHS runs as the mapred user, who is a member
of the hadoop group. With the new group, the JHS doesn't have permission to read the logs
any more.)

> Aggregated Logs root directory is created with wrong group if nonexistent 
> --------------------------------------------------------------------------
>
>                 Key: YARN-5001
>                 URL: https://issues.apache.org/jira/browse/YARN-5001
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: nodemanager
>    Affects Versions: 2.7.0
>            Reporter: Haibo Chen
>            Assignee: Haibo Chen
>         Attachments: yarn5001.001.patch
>
>
> The directory /tmp/logs, where the aggregated logs go, is supposed to be read by JHS.
But if it is not created beforehand, it will be created by NodeManager with group being the
superuser set in HDFS. This leads to JHS to fail to read the container logs files under that
directory if JHS is not running as a user that belongs to superuser.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message