hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-15278) log s3a at info
Date Wed, 07 Mar 2018 20:29:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-15278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16390165#comment-16390165
] 

Steve Loughran commented on HADOOP-15278:
-----------------------------------------

Actually, this is what you see in the default
{code}
2018-03-07 20:20:48,755 INFO beanutils.FluentPropertyBeanIntrospector: Error when creating
PropertyDescriptor for public final void org.apache.commons.configuration2.AbstractConfiguration.setProperty(java.lang.String,java.lang.Object)!
Ignoring this property.
2018-03-07 20:20:48,780 INFO impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
2018-03-07 20:20:48,844 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10
second(s).
2018-03-07 20:20:48,844 INFO impl.MetricsSystemImpl: s3a-file-system metrics system started
2018-03-07 20:20:50,108 INFO Configuration.deprecation: fs.s3a.server-side-encryption-key
is deprecated. Instead, use fs.s3a.server-side-encryption.key
Found 1 items
drwxrwxrwx   - stevel stevel          0 2018-03-07 20:20 s3a://hwdev-steve-london/Users
2018-03-07 20:20:50,274 INFO impl.MetricsSystemImpl: Stopping s3a-file-system metrics system...
2018-03-07 20:20:50,274 INFO impl.MetricsSystemImpl: s3a-file-system metrics system stopped.
2018-03-07 20:20:50,274 INFO impl.MetricsSystemImpl: s3a-file-system metrics system shutdown
complete.
{code}

Something about deprecation I've never got away, and metrics cruft. The fluent property one
is HADOOP-15277; think its actually metrics related.

for reference, azure
{code}
bin/hadoop fs -ls wasb://contract@contender.blob.core.windows.net
2018-03-07 20:26:16,848 INFO util.log: Logging initialized @1371ms
2018-03-07 20:26:16,968 INFO beanutils.FluentPropertyBeanIntrospector: Error when creating
PropertyDescriptor for public final void org.apache.commons.configuration2.AbstractConfiguration.setProperty(java.lang.String,java.lang.Object)!
Ignoring this property.
2018-03-07 20:26:17,003 INFO impl.MetricsConfig: loaded properties from hadoop-metrics2.properties
2018-03-07 20:26:17,075 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10
second(s).
2018-03-07 20:26:17,075 INFO impl.MetricsSystemImpl: azure-file-system metrics system started
Found 7 items
-rw-r--r--   1 stevel supergroup          0 2017-06-30 18:12 wasb://contract@contender.blob.core.windows.net/user/stevel/file
-rw-r--r--   1 stevel supergroup        513 2017-06-30 18:10 wasb://contract@contender.blob.core.windows.net/user/stevel/file.dat
-rw-r--r--   1 stevel supergroup          0 2017-06-30 18:13 wasb://contract@contender.blob.core.windows.net/user/stevel/fileToRename
drwxr-xr-x   - stevel supergroup          0 2017-06-30 18:11 wasb://contract@contender.blob.core.windows.net/user/stevel/parent
drwxr-xr-x   - stevel supergroup          0 2017-06-30 18:11 wasb://contract@contender.blob.core.windows.net/user/stevel/renamedFolder
drwxr-xr-x   - stevel supergroup          0 2016-10-20 22:00 wasb://contract@contender.blob.core.windows.net/user/stevel/streaming
drwxr-xr-x   - stevel supergroup          0 2017-06-30 18:11 wasb://contract@contender.blob.core.windows.net/user/stevel/testDestFolder
2018-03-07 20:26:17,756 INFO impl.MetricsSystemImpl: Stopping azure-file-system metrics system...
2018-03-07 20:26:17,756 INFO impl.MetricsSystemImpl: azure-file-system metrics system stopped.
2018-03-07 20:26:17,756 INFO impl.MetricsSystemImpl: azure-file-system metrics system shutdown
complete.
{code}

so: equivalent noisyness around metrics. Dangerous to turn that off though, as this is the
same log4j used in the servers, which also spin the metrics. wasb and s3a are just the two
clients to use the metrics classes

> log s3a at info
> ---------------
>
>                 Key: HADOOP-15278
>                 URL: https://issues.apache.org/jira/browse/HADOOP-15278
>             Project: Hadoop Common
>          Issue Type: Sub-task
>          Components: fs/s3
>    Affects Versions: 3.0.0
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>            Priority: Major
>
> since it was added, hadoop conf/log4j only logs s3a at ERROR, even though in our test/resources
it logs at info. We do actually log lots of stuff useful when debugging things
> Proposed: drop the log level to INFO here



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message