Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: yarn-issues@hadoop.apache.org
Date: Thu, 17 Dec 2015 16:35:46 +0000 (UTC)
From: "Naganarasimha G R (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.12760381.1418112332000.55360.1450370146899@Atlassian.JIRA>
In-Reply-To: <JIRA.12760381.1418112332000@Atlassian.JIRA>
References: <JIRA.12760381.1418112332000@Atlassian.JIRA>
 <JIRA.12760381.1418112332622@arcas>
Subject: [jira] [Updated] (YARN-2934) Improve handling of container's stderr
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/YARN-2934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Naganarasimha G R updated YARN-2934:
------------------------------------
    Attachment: YARN-2934.v2.001.patch

Thanks for the review comments and opinions [~jira.shegalov] & [~vvasudev],
bq.  Seeing as Gera Shegalov feels strongly about this, I'll defer to him unless you have an example app log file that a glob can't catch but a regex would.
Well i don't have a strong case to back my hunch and neither could i get a better example to show advantages of regex over glob. Most of the cases(MR, Distributed shell, Spark etc...) are all using *stderr*, so i am attaching a patch with modifications incorporating  [~jira.shegalov]'s suggestion along with fixes for other issues.
{code}
diagnosticInfo
              .append("Error files: ")
              .append(Arrays.toString(listStatus))
              .append("\n")
              .append("Last ").append(tailBytes).append(" bytes of ").append(listStatus[0])
              .append(new String(tailBytes, UTF_8));
{code} was providing a result as 
{code}
Container exited with a non-zero exit code 127. Error file(s): [DeprecatedRawLocalFileStatus{path=file:/opt/git/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/TestContainerLaunch-localLogDir/application_1450367986801_0001/container_1450367986801_0001_01_000001/error.log; isDirectory=false; length=60; replication=1; blocksize=33554432; modification_time=1450367987000; access_time=1450367987000; owner=; group=; permission=rw-rw-rw-; isSymlink=false}]
Last [B@78a7f540 bytes of DeprecatedRawLocalFileStatus{path=file:/opt/git/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/TestContainerLaunch-localLogDir/application_1450367986801_0001/container_1450367986801_0001_01_000001/error.log; isDirectory=false; length=60; replication=1; blocksize=33554432; modification_time=1450367987000; access_time=1450367987000; owner=; group=; permission=rw-rw-rw-; isSymlink=false} :
/bin/bash: /no/jvm/here/bin/java: No such file or directory
{code}
so have modified it to be 
{code}
          diagnosticInfo
              .append("Last ").append(tailSizeInBytes).append(" bytes of ")
              .append(listStatus[0].getPath().getName())
              .append(" :\n")
              .append(new String(tailBytes, UTF_8));
{code}
Hope its fine else can revert it as per your suggestion.

> Improve handling of container's stderr 
> ---------------------------------------
>
>                 Key: YARN-2934
>                 URL: https://issues.apache.org/jira/browse/YARN-2934
>             Project: Hadoop YARN
>          Issue Type: Improvement
>            Reporter: Gera Shegalov
>            Assignee: Naganarasimha G R
>            Priority: Critical
>         Attachments: YARN-2934.v1.001.patch, YARN-2934.v1.002.patch, YARN-2934.v1.003.patch, YARN-2934.v1.004.patch, YARN-2934.v1.005.patch, YARN-2934.v1.006.patch, YARN-2934.v1.007.patch, YARN-2934.v1.008.patch, YARN-2934.v2.001.patch
>
>
> Most YARN applications redirect stderr to some file. That's why when container launch fails with {{ExitCodeException}} the message is empty.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)