hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Elliott Clark (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-4709) Hadoop metrics2 setup in test MiniDFSClusters spewing JMX errors
Date Wed, 28 Nov 2012 17:54:58 GMT

    [ https://issues.apache.org/jira/browse/HBASE-4709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13505709#comment-13505709
] 

Elliott Clark commented on HBASE-4709:
--------------------------------------

So I had two thoughts.

# I'm not 100% sure that this kind of work around isn't needed in production.  Right now lots
of messages are logged by the metrics system that are really scary to users but not really
a bad thing (Specifically when starting/stopping the metrics system there are tons of errors
thrown).  So we could want this logging change for more than just one logger.
# If we decide that we shouldn't silence things in a real server there are already test utilities
that reach into Hadoop's guts that are in hbase-hadoop1-compat and hbase-hadoop2-compat. 
Seems like we should try and make the main jar have very few places where Hadoop internals
are touched just in case different versions diverge more in the future.
                
> Hadoop metrics2 setup in test MiniDFSClusters spewing JMX errors
> ----------------------------------------------------------------
>
>                 Key: HBASE-4709
>                 URL: https://issues.apache.org/jira/browse/HBASE-4709
>             Project: HBase
>          Issue Type: Bug
>          Components: test
>    Affects Versions: 0.92.0, 0.94.0, 0.94.1
>            Reporter: Gary Helmling
>            Priority: Minor
>             Fix For: 0.96.0
>
>         Attachments: 4709_workaround.v1.patch
>
>
> Since switching over HBase to build with Hadoop 0.20.205.0, we've been getting a lot
of metrics related errors in the log files for tests:
> {noformat}
> 2011-10-30 22:00:22,858 INFO  [main] log.Slf4jLog(67): jetty-6.1.26
> 2011-10-30 22:00:22,871 INFO  [main] log.Slf4jLog(67): Extract jar:file:/home/jenkins/.m2/repository/org/apache/hadoop/hadoop-core/0.20.205.0/hadoop-core-0.20.205.0.jar!/webapps/datanode
to /tmp/Jetty_localhost_55751_datanode____.kw16hy/webapp
> 2011-10-30 22:00:23,048 INFO  [main] log.Slf4jLog(67): Started SelectChannelConnector@localhost:55751
> Starting DataNode 1 with dfs.data.dir: /home/jenkins/jenkins-slave/workspace/HBase-TRUNK/trunk/target/test-data/7ba65a16-03ad-4624-b769-57405945ef58/dfscluster_3775fc23-1b51-4966-8133-205564bae762/dfs/data/data3,/home/jenkins/jenkins-slave/workspace/HBase-TRUNK/trunk/target/test-data/7ba65a16-03ad-4624-b769-57405945ef58/dfscluster_3775fc23-1b51-4966-8133-205564bae762/dfs/data/data4
> 2011-10-30 22:00:23,237 WARN  [main] impl.MetricsSystemImpl(137): Metrics system not
started: Cannot locate configuration: tried hadoop-metrics2-datanode.properties, hadoop-metrics2.properties
> 2011-10-30 22:00:23,237 WARN  [main] util.MBeans(59): Hadoop:service=DataNode,name=MetricsSystem,sub=Control
> javax.management.InstanceAlreadyExistsException: MXBean already registered with name
Hadoop:service=NameNode,name=MetricsSystem,sub=Control
> 	at com.sun.jmx.mbeanserver.MXBeanLookup.addReference(MXBeanLookup.java:120)
> 	at com.sun.jmx.mbeanserver.MXBeanSupport.register(MXBeanSupport.java:143)
> 	at com.sun.jmx.mbeanserver.MBeanSupport.preRegister2(MBeanSupport.java:183)
> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:941)
> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:917)
> 	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:312)
> 	at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:482)
> 	at org.apache.hadoop.metrics2.util.MBeans.register(MBeans.java:56)
> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.initSystemMBean(MetricsSystemImpl.java:500)
> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.init(MetricsSystemImpl.java:140)
> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.init(DefaultMetricsSystem.java:40)
> 	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.initialize(DefaultMetricsSystem.java:50)
> 	at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1483)
> 	at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1459)
> 	at org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:417)
> 	at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:280)
> 	at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniDFSCluster(HBaseTestingUtility.java:349)
> 	at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:518)
> 	at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:474)
> 	at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:461)
> {noformat}
> This seems to be due to errors initializing the new hadoop metrics2 code by default,
when running in a mini cluster.  The errors themselves seem to be harmless -- they're not
breaking any tests -- but we should figure out what configuration we need to eliminate them.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message