Mailing-List: contact dev-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@flink.apache.org
Date: Tue, 29 Nov 2016 16:02:59 +0000 (UTC)
From: "Ufuk Celebi (JIRA)" <jira@apache.org>
To: dev@flink.apache.org
Message-ID: <JIRA.13024126.1480435338000.390407.1480435379113@Atlassian.JIRA>
In-Reply-To: <JIRA.13024126.1480435338000@Atlassian.JIRA>
References: <JIRA.13024126.1480435338000@Atlassian.JIRA> <JIRA.13024126.1480435338875@arcas>
Subject: [jira] [Created] (FLINK-5207) Decrease HadoopFileSystem logging
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Tue, 29 Nov 2016 16:03:01 -0000

Ufuk Celebi created FLINK-5207:
----------------------------------

             Summary: Decrease HadoopFileSystem logging
                 Key: FLINK-5207
                 URL: https://issues.apache.org/jira/browse/FLINK-5207
             Project: Flink
          Issue Type: Improvement
            Reporter: Ufuk Celebi


HadoopFileSystem logging is very noisy when trying to get the Hadoop configuration, which happens for every HadoopFileSystem. This happens for example when a checkpoint is written to HDFS in case of HA.

1) Two log messages relate to the user config:

{code}
final String hdfsDefaultPath = GlobalConfiguration.getString(ConfigConstants.HDFS_DEFAULT_CONFIG, null);
		if (hdfsDefaultPath != null) {
			retConf.addResource(new org.apache.hadoop.fs.Path(hdfsDefaultPath));
		} else {
			LOG.debug("Cannot find hdfs-default configuration file");
		}
{code}

Since this is reflected in the logged configuration properties, it is mostly unnecessary to log over and over again. Therefore I propose to go with TRACE.

(2) Two log messages relate to the default config files of Hadoop. There I would change the debug message to happen only when they are *not* found. If it is still too noisy, we could completely remove it or decrease to TRACE at another time.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)