hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Naganarasimha G R (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-2934) Improve handling of container's stderr
Date Thu, 17 Dec 2015 16:35:46 GMT

     [ https://issues.apache.org/jira/browse/YARN-2934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Naganarasimha G R updated YARN-2934:
------------------------------------
    Attachment: YARN-2934.v2.001.patch

Thanks for the review comments and opinions [~jira.shegalov] & [~vvasudev],
bq.  Seeing as Gera Shegalov feels strongly about this, I'll defer to him unless you have
an example app log file that a glob can't catch but a regex would.
Well i don't have a strong case to back my hunch and neither could i get a better example
to show advantages of regex over glob. Most of the cases(MR, Distributed shell, Spark etc...)
are all using *stderr*, so i am attaching a patch with modifications incorporating  [~jira.shegalov]'s
suggestion along with fixes for other issues.
{code}
diagnosticInfo
              .append("Error files: ")
              .append(Arrays.toString(listStatus))
              .append("\n")
              .append("Last ").append(tailBytes).append(" bytes of ").append(listStatus[0])
              .append(new String(tailBytes, UTF_8));
{code} was providing a result as 
{code}
Container exited with a non-zero exit code 127. Error file(s): [DeprecatedRawLocalFileStatus{path=file:/opt/git/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/TestContainerLaunch-localLogDir/application_1450367986801_0001/container_1450367986801_0001_01_000001/error.log;
isDirectory=false; length=60; replication=1; blocksize=33554432; modification_time=1450367987000;
access_time=1450367987000; owner=; group=; permission=rw-rw-rw-; isSymlink=false}]
Last [B@78a7f540 bytes of DeprecatedRawLocalFileStatus{path=file:/opt/git/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/target/TestContainerLaunch-localLogDir/application_1450367986801_0001/container_1450367986801_0001_01_000001/error.log;
isDirectory=false; length=60; replication=1; blocksize=33554432; modification_time=1450367987000;
access_time=1450367987000; owner=; group=; permission=rw-rw-rw-; isSymlink=false} :
/bin/bash: /no/jvm/here/bin/java: No such file or directory
{code}
so have modified it to be 
{code}
          diagnosticInfo
              .append("Last ").append(tailSizeInBytes).append(" bytes of ")
              .append(listStatus[0].getPath().getName())
              .append(" :\n")
              .append(new String(tailBytes, UTF_8));
{code}
Hope its fine else can revert it as per your suggestion.

> Improve handling of container's stderr 
> ---------------------------------------
>
>                 Key: YARN-2934
>                 URL: https://issues.apache.org/jira/browse/YARN-2934
>             Project: Hadoop YARN
>          Issue Type: Improvement
>            Reporter: Gera Shegalov
>            Assignee: Naganarasimha G R
>            Priority: Critical
>         Attachments: YARN-2934.v1.001.patch, YARN-2934.v1.002.patch, YARN-2934.v1.003.patch,
YARN-2934.v1.004.patch, YARN-2934.v1.005.patch, YARN-2934.v1.006.patch, YARN-2934.v1.007.patch,
YARN-2934.v1.008.patch, YARN-2934.v2.001.patch
>
>
> Most YARN applications redirect stderr to some file. That's why when container launch
fails with {{ExitCodeException}} the message is empty.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message