hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Siddharth Seth (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-812) Enabling app summary logs causes 'FileNotFound' errors
Date Fri, 14 Jun 2013 00:59:22 GMT

     [ https://issues.apache.org/jira/browse/YARN-812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Siddharth Seth updated YARN-812:
--------------------------------

    Attachment: YARN-812.txt

Patch changes the default appsummary logger to match hadoop.root.logger, which for clients
will be INFO, console. This matches the 1.x behaviour.

When starting the RM, the value would need to be set in hadoop-env.sh.

Log4j seems to initialize it's entire configuration up-front irrespective of which logger
is actually being used. That's what causes the exception - since users may not have access
to the path where the app-summary log is being written.
                
> Enabling app summary logs causes 'FileNotFound' errors
> ------------------------------------------------------
>
>                 Key: YARN-812
>                 URL: https://issues.apache.org/jira/browse/YARN-812
>             Project: Hadoop YARN
>          Issue Type: Bug
>    Affects Versions: 2.1.0-beta
>            Reporter: Ramya Sunil
>            Assignee: Siddharth Seth
>             Fix For: 2.1.0-beta
>
>         Attachments: YARN-812.txt
>
>
> RM app summary logs have been enabled as per the default config:
> {noformat}
> #
> # Yarn ResourceManager Application Summary Log 
> #
> # Set the ResourceManager summary log filename
> yarn.server.resourcemanager.appsummary.log.file=rm-appsummary.log
> # Set the ResourceManager summary log level and appender
> yarn.server.resourcemanager.appsummary.logger=INFO,RMSUMMARY
> # Appender for ResourceManager Application Summary Log
> # Requires the following properties to be set
> #    - hadoop.log.dir (Hadoop Log directory)
> #    - yarn.server.resourcemanager.appsummary.log.file (resource manager app summary
log filename)
> #    - yarn.server.resourcemanager.appsummary.logger (resource manager app summary log
level and appender)
> log4j.logger.org.apache.hadoop.yarn.server.resourcemanager.RMAppManager$ApplicationSummary=${yarn.server.resourcemanager.appsummary.logger}
> log4j.additivity.org.apache.hadoop.yarn.server.resourcemanager.RMAppManager$ApplicationSummary=false
> log4j.appender.RMSUMMARY=org.apache.log4j.RollingFileAppender
> log4j.appender.RMSUMMARY.File=${hadoop.log.dir}/${yarn.server.resourcemanager.appsummary.log.file}
> log4j.appender.RMSUMMARY.MaxFileSize=256MB
> log4j.appender.RMSUMMARY.MaxBackupIndex=20
> log4j.appender.RMSUMMARY.layout=org.apache.log4j.PatternLayout
> log4j.appender.RMSUMMARY.layout.ConversionPattern=%d{ISO8601} %p %c{2}: %m%n
> {noformat}
> This however, throws errors while running commands as non-superuser:
> {noformat}
> -bash-4.1$ hadoop dfs -ls /
> DEPRECATED: Use of this script to execute hdfs command is deprecated.
> Instead use the hdfs command for it.
> log4j:ERROR setFile(null,true) call failed.
> java.io.FileNotFoundException: /var/log/hadoop/hadoopqa/rm-appsummary.log (No such file
or directory)
>         at java.io.FileOutputStream.openAppend(Native Method)
>         at java.io.FileOutputStream.<init>(FileOutputStream.java:192)
>         at java.io.FileOutputStream.<init>(FileOutputStream.java:116)
>         at org.apache.log4j.FileAppender.setFile(FileAppender.java:294)
>         at org.apache.log4j.RollingFileAppender.setFile(RollingFileAppender.java:207)
>         at org.apache.log4j.FileAppender.activateOptions(FileAppender.java:165)
>         at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:307)
>         at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:172)
>         at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:104)
>         at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:842)
>         at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:768)
>         at org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:672)
>         at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:516)
>         at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:580)
>         at org.apache.log4j.helpers.OptionConverter.selectAndConfigure(OptionConverter.java:526)
>         at org.apache.log4j.LogManager.<clinit>(LogManager.java:127)
>         at org.apache.log4j.Logger.getLogger(Logger.java:104)
>         at org.apache.commons.logging.impl.Log4JLogger.getLogger(Log4JLogger.java:289)
>         at org.apache.commons.logging.impl.Log4JLogger.<init>(Log4JLogger.java:109)
>         at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
>         at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
>         at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
>         at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
>         at org.apache.commons.logging.impl.LogFactoryImpl.createLogFromClass(LogFactoryImpl.java:1116)
>         at org.apache.commons.logging.impl.LogFactoryImpl.discoverLogImplementation(LogFactoryImpl.java:858)
>         at org.apache.commons.logging.impl.LogFactoryImpl.newInstance(LogFactoryImpl.java:604)
>         at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:336)
>         at org.apache.commons.logging.impl.LogFactoryImpl.getInstance(LogFactoryImpl.java:310)
>         at org.apache.commons.logging.LogFactory.getLog(LogFactory.java:685)
>         at org.apache.hadoop.fs.FsShell.<clinit>(FsShell.java:41)
> Found 1 items
> drwxr-xr-x   - hadoop   hadoop            0 2013-06-12 21:28 /user
> {noformat}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message