hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-6550) archive-logs tool changes log ownership to the Yarn user when using DefaultContainerExecutor
Date Thu, 26 Nov 2015 04:55:11 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6550?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15028159#comment-15028159
] 

Hudson commented on MAPREDUCE-6550:
-----------------------------------

FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #645 (See [https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/645/])
MAPREDUCE-6550. archive-logs tool changes log ownership to the Yarn user (rkanter: rev 6d84cc16b3e0685fef01d0e3526b0f7556ceff51)
* hadoop-tools/hadoop-archive-logs/src/test/java/org/apache/hadoop/tools/TestHadoopArchiveLogsRunner.java
* hadoop-tools/hadoop-archive-logs/src/site/markdown/HadoopArchiveLogs.md
* hadoop-tools/hadoop-archive-logs/src/main/java/org/apache/hadoop/tools/HadoopArchiveLogs.java
* hadoop-tools/hadoop-archive-logs/src/main/java/org/apache/hadoop/tools/HadoopArchiveLogsRunner.java
* hadoop-mapreduce-project/CHANGES.txt
* hadoop-tools/hadoop-archive-logs/src/test/java/org/apache/hadoop/tools/TestHadoopArchiveLogs.java


> archive-logs tool changes log ownership to the Yarn user when using DefaultContainerExecutor
> --------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-6550
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6550
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>    Affects Versions: 2.8.0
>            Reporter: Robert Kanter
>            Assignee: Robert Kanter
>             Fix For: 2.8.0
>
>         Attachments: MAPREDUCE-6550.001.patch, MAPREDUCE-6550.002.patch, MAPREDUCE-6550.003.patch,
MAPREDUCE-6550.004.patch, MAPREDUCE-6550.005.patch
>
>
> The archive-logs tool added in MAPREDUCE-6415 leverages the Distributed Shell app.  When
using the DefaultContainerExecutor, this means that the job will actually run as the Yarn
user, so the resulting har files are owned by the Yarn user instead of the original owner.
The permissions are also now world-readable.
> In the below example, the archived logs are owned by 'yarn' instead of 'paul' and are
now world-readable:
> {noformat}
> [root@gs28-centos66-5 ~]# sudo -u hdfs hdfs dfs -ls -R /tmp/logs
> ...
> drwxrwx---   - paul  hadoop          0 2015-10-02 13:24 /tmp/logs/paul/logs/application_1443805425363_0005
> drwxr-xr-x   - yarn  hadoop          0 2015-10-02 13:24 /tmp/logs/paul/logs/application_1443805425363_0005/application_1443805425363_0005.har
> -rw-r--r--   3 yarn  hadoop          0 2015-10-02 13:24 /tmp/logs/paul/logs/application_1443805425363_0005/application_1443805425363_0005.har/_SUCCESS
> -rw-r--r--   3 yarn  hadoop       1256 2015-10-02 13:24 /tmp/logs/paul/logs/application_1443805425363_0005/application_1443805425363_0005.har/_index
> -rw-r--r--   3 yarn  hadoop         24 2015-10-02 13:24 /tmp/logs/paul/logs/application_1443805425363_0005/application_1443805425363_0005.har/_masterindex
> -rw-r--r--   3 yarn  hadoop    8451177 2015-10-02 13:24 /tmp/logs/paul/logs/application_1443805425363_0005/application_1443805425363_0005.har/part-0
> drwxrwx---   - paul  hadoop          0 2015-10-02 13:24 /tmp/logs/paul/logs/application_1443805425363_0006
> -rw-r-----   3 paul  hadoop       1155 2015-10-02 13:24 /tmp/logs/paul/logs/application_1443805425363_0006/gs-centos66-2.vpc.cloudera.com_8041
> -rw-r-----   3 paul  hadoop       4880 2015-10-02 13:24 /tmp/logs/paul/logs/application_1443805425363_0006/gs28-centos66-3.vpc.cloudera.com_8041
> ...
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message