Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: yarn-issues@hadoop.apache.org
Date: Mon, 8 Jun 2015 12:21:01 +0000 (UTC)
From: "Steve Loughran (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.12834473.1433222728000.17353.1433766061402@Atlassian.JIRA>
In-Reply-To: <JIRA.12834473.1433222728000@Atlassian.JIRA>
References: <JIRA.12834473.1433222728000@Atlassian.JIRA>
 <JIRA.12834473.1433222728486@arcas>
Subject: [jira] [Commented] (YARN-3755) Log the command of launching
 containers
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/YARN-3755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14577097#comment-14577097 ] 

Steve Loughran commented on YARN-3755:
--------------------------------------

I think this should be re-opened with YARN-3759 tagged as a duplicate.
# one of the problems of pushing it to the framework is "what if you can't control that framework"? Because I spent a lot of time last week having to add the logging inside my own fork of spark do to an "env > $LOG_DIR/env"
# your framework code doesn't know the CLI and env that is finally generated

I propose, then, one of two options
# that YARN has the option to log the env and CLI to the log directory
# that we have a specific YARN log for app & container launch, which can be set to INFO to get this log information
# that we start a section in the docs "troubleshooting YARN" which explains how to use this stuff.

This patch would be basis the no. 2 item on that list; we just create a specific log for those messages.


> Log the command of launching containers
> ---------------------------------------
>
>                 Key: YARN-3755
>                 URL: https://issues.apache.org/jira/browse/YARN-3755
>             Project: Hadoop YARN
>          Issue Type: Improvement
>    Affects Versions: 2.7.0
>            Reporter: Jeff Zhang
>            Assignee: Jeff Zhang
>         Attachments: YARN-3755-1.patch, YARN-3755-2.patch
>
>
> In the resource manager log, yarn would log the command for launching AM, this is very useful. But there's no such log in the NN log for launching containers. It would be difficult to diagnose when containers fails to launch due to some issue in the commands. Although user can look at the commands in the container launch script file, this is an internal things of yarn, usually user don't know that. In user's perspective, they only know what commands they specify when building yarn application. 
> {code}
> 2015-06-01 16:06:42,245 INFO org.apache.hadoop.yarn.server.resourcemanager.amlauncher.AMLauncher: Command to launch container container_1433145984561_0001_01_000001 : $JAVA_HOME/bin/java -server -Djava.net.preferIPv4Stack=true -Dhadoop.metrics.log.level=WARN  -Xmx1024m  -Dlog4j.configuratorClass=org.apache.tez.common.TezLog4jConfigurator -Dlog4j.configuration=tez-container-log4j.properties -Dyarn.app.container.log.dir=<LOG_DIR> -Dtez.root.logger=info,CLA -Dsun.nio.ch.bugLevel='' org.apache.tez.dag.app.DAGAppMaster 1><LOG_DIR>/stdout 2><LOG_DIR>/stderr
> {code}


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)