hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daryn Sharp (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-10015) UserGroupInformation prints out excessive ERROR warnings
Date Wed, 09 Oct 2013 16:14:43 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-10015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13790559#comment-13790559

Daryn Sharp commented on HADOOP-10015:

I guess I'm surprised that distcp is operating within an explicit doAs.  Logging doesn't occur
when an exception generated under the login/current user, just an explicit doAs.

I do agree that the logging can be annoying at times, but it's a lifesaver at other times.
 Like code that loops w/o logging the exception that triggered a retry, code that swallows
an exception, code that catches and throws a generic "something failed" exception, etc.

If it's a debug level, then the logging becomes worthless - it's not possible to retroactively
enable debug logging to analyze race conditions or transient errors.  Debug is too voluminous
to enable for catching problems in production.

Perhaps a middle of the road solution is to create a new logger instance for doas, named with
a ".doAs" suffix to the ugi class.  That way those that want to disable the logging may do
so via log4j.properties.

> UserGroupInformation prints out excessive ERROR warnings
> --------------------------------------------------------
>                 Key: HADOOP-10015
>                 URL: https://issues.apache.org/jira/browse/HADOOP-10015
>             Project: Hadoop Common
>          Issue Type: Bug
>            Reporter: Haohui Mai
>            Assignee: Haohui Mai
>         Attachments: HADOOP-10015.000.patch, HADOOP-10015.001.patch, HADOOP-10015.002.patch
> In UserGroupInformation::doAs(), it prints out a log at ERROR level whenever it catches
an exception.
> However, it prints benign warnings in the following paradigm:
> {noformat}
>  try {
>         ugi.doAs(new PrivilegedExceptionAction<FileStatus>() {
>           @Override
>           public FileStatus run() throws Exception {
>             return fs.getFileStatus(nonExist);
>           }
>         });
>       } catch (FileNotFoundException e) {
>       }
> {noformat}
> For example, FileSystem#exists() follows this paradigm. Distcp uses this paradigm too.
The exception is expected therefore there should be no ERROR logs printed in the namenode
> Currently, the user quickly finds out that the namenode log is quickly filled by _benign_
ERROR logs when he or she runs distcp in secure set up. This behavior confuses the operators.
> This jira proposes to move the log to DEBUG level.

This message was sent by Atlassian JIRA

View raw message