hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Allen Wittenauer (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-9902) Shell script rewrite
Date Thu, 21 Aug 2014 20:39:14 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-9902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14105924#comment-14105924

Allen Wittenauer commented on HADOOP-9902:

bq. Found one more problem that RM and NM daemon logs are now inside *.out file, instead of
*.log file.

As mentioned in the release notes, YARN did a bunch of heinous stuff when it came to log4j
settings, counter to the rest of Hadoop, and to much frustration with ops teams.  This has
been made consistent, so there is good chance you were relying upon that behavior.  It could
be any number of things: NM and RM _OPT settings, dependence upon the nodemanger/log4j.settings
file or resourcemanager/log4j.settings file, yarn-env.sh settings, etc.  You can always do
'bash -x yarn --daemon start resourcemanager'. The out file should contain the java command

With the shipping *-env.sh files, you should see something similar to:
java -Dproc_resourcemanager -Djava.net.preferIPv4Stack=true -Djava.security.krb5.realm= -Djava.security.krb5.kdc=
-Djava.security.krb5.conf= -Dyarn.log.dir=/Users/aw/HADOOP/hadoop-3.0.0-SNAPSHOT/logs -Dyarn.log.file=hadoop-aw-resourcemanager-aw-mbp-work.local.log
-Dyarn.home.dir=/Users/aw/HADOOP/hadoop-3.0.0-SNAPSHOT -Dyarn.root.logger=INFO,RFA -Xmx1g
-Dhadoop.log.dir=/Users/aw/HADOOP/hadoop-3.0.0-SNAPSHOT/logs -Dhadoop.log.file=hadoop-aw-resourcemanager-aw-mbp-work.local.log
-Dhadoop.home.dir=/Users/aw/HADOOP/hadoop-3.0.0-SNAPSHOT -Dhadoop.id.str=aw -Dhadoop.root.logger=INFO,RFA
-Dhadoop.policy.file=hadoop-policy.xml -Dhadoop.security.logger=INFO,NullAppender org.apache.hadoop.yarn.server.resourcemanager.ResourceManager

bq. Also yarn daemon log file name earlier was starting with "yarn", now it's changed to "hadoop",
not sure if this is an intentional change.

Intentional.  This is to make YARN consistent with the rest of the system.  (Noticing a theme?)

> Shell script rewrite
> --------------------
>                 Key: HADOOP-9902
>                 URL: https://issues.apache.org/jira/browse/HADOOP-9902
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: scripts
>    Affects Versions: 3.0.0
>            Reporter: Allen Wittenauer
>            Assignee: Allen Wittenauer
>              Labels: releasenotes
>             Fix For: 3.0.0
>         Attachments: HADOOP-9902-10.patch, HADOOP-9902-11.patch, HADOOP-9902-12.patch,
HADOOP-9902-13-branch-2.patch, HADOOP-9902-13.patch, HADOOP-9902-14.patch, HADOOP-9902-15.patch,
HADOOP-9902-16.patch, HADOOP-9902-2.patch, HADOOP-9902-3.patch, HADOOP-9902-4.patch, HADOOP-9902-5.patch,
HADOOP-9902-6.patch, HADOOP-9902-7.patch, HADOOP-9902-8.patch, HADOOP-9902-9.patch, HADOOP-9902.patch,
HADOOP-9902.txt, hadoop-9902-1.patch, more-info.txt
> Umbrella JIRA for shell script rewrite.  See more-info.txt for more details.

This message was sent by Atlassian JIRA

View raw message