hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-19588) Additional jar dependencies needed for mapreduce PerformanceEvaluation
Date Mon, 01 Jan 2018 20:06:00 GMT

    [ https://issues.apache.org/jira/browse/HBASE-19588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16307537#comment-16307537
] 

stack commented on HBASE-19588:
-------------------------------

+1 thanks [~chu11]. Will commit....[~apurtell] FYI.

> Additional jar dependencies needed for mapreduce PerformanceEvaluation
> ----------------------------------------------------------------------
>
>                 Key: HBASE-19588
>                 URL: https://issues.apache.org/jira/browse/HBASE-19588
>             Project: HBase
>          Issue Type: Bug
>          Components: test
>    Affects Versions: 1.4.0
>            Reporter: Albert Chu
>            Priority: Minor
>         Attachments: HBASE-19588.branch-1.4.patch
>
>
> I have a unit test that runs a simple PerformanceEvaluation test to make sure things
are basically working
> {noformat}
> bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=50000 sequentialWrite
1
> {noformat}
> This test runs against Hadoop 2.7.0 and works against all past versions 0.99.0 and up.
 It broke with 1.4.0 with the following error.
> {noformat}
> 2017-12-21 13:49:40,974 INFO  [main] mapreduce.Job: Task Id : attempt_1513892752187_0002_m_000004_2,
Status : FAILED
> Error: java.io.IOException: java.lang.reflect.InvocationTargetException
> 	at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:240)
> 	at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:218)
> 	at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:119)
> 	at org.apache.hadoop.hbase.PerformanceEvaluation$EvaluationMapTask.map(PerformanceEvaluation.java:297)
> 	at org.apache.hadoop.hbase.PerformanceEvaluation$EvaluationMapTask.map(PerformanceEvaluation.java:250)
> 	at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
> 	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
> 	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
> 	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:415)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> 	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: java.lang.reflect.InvocationTargetException
> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> 	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> 	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> 	at org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238)
> 	... 12 more
> Caused by: java.lang.RuntimeException: Could not create  interface org.apache.hadoop.hbase.zookeeper.MetricsZooKeeperSource
Is the hadoop compatibility jar on the classpath?
> 	at org.apache.hadoop.hbase.CompatibilitySingletonFactory.getInstance(CompatibilitySingletonFactory.java:75)
> 	at org.apache.hadoop.hbase.zookeeper.MetricsZooKeeper.<init>(MetricsZooKeeper.java:38)
> 	at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.<init>(RecoverableZooKeeper.java:130)
> 	at org.apache.hadoop.hbase.zookeeper.ZKUtil.connect(ZKUtil.java:143)
> 	at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:181)
> 	at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:155)
> 	at org.apache.hadoop.hbase.client.ZooKeeperKeepAliveConnection.<init>(ZooKeeperKeepAliveConnection.java:43)
> 	at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.getKeepAliveZooKeeperWatcher(ConnectionManager.java:1737)
> 	at org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:104)
> 	at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:945)
> 	at org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.<init>(ConnectionManager.java:721)
> 	... 17 more
> Caused by: java.util.ServiceConfigurationError: org.apache.hadoop.hbase.zookeeper.MetricsZooKeeperSource:
Provider org.apache.hadoop.hbase.zookeeper.MetricsZooKeeperSourceImpl could not be instantiated
> 	at java.util.ServiceLoader.fail(ServiceLoader.java:224)
> 	at java.util.ServiceLoader.access$100(ServiceLoader.java:181)
> 	at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:377)
> 	at java.util.ServiceLoader$1.next(ServiceLoader.java:445)
> 	at org.apache.hadoop.hbase.CompatibilitySingletonFactory.getInstance(CompatibilitySingletonFactory.java:59)
> 	... 27 more
> Caused by: java.lang.NoClassDefFoundError: Lorg/apache/hadoop/hbase/metrics/MetricRegistry;
> 	at java.lang.Class.getDeclaredFields0(Native Method)
> 	at java.lang.Class.privateGetDeclaredFields(Class.java:2509)
> 	at java.lang.Class.getDeclaredFields(Class.java:1819)
> 	at org.apache.hadoop.util.ReflectionUtils.getDeclaredFieldsIncludingInherited(ReflectionUtils.java:323)
> 	at org.apache.hadoop.metrics2.lib.MetricsSourceBuilder.initRegistry(MetricsSourceBuilder.java:92)
> 	at org.apache.hadoop.metrics2.lib.MetricsSourceBuilder.<init>(MetricsSourceBuilder.java:56)
> 	at org.apache.hadoop.metrics2.lib.MetricsAnnotations.newSourceBuilder(MetricsAnnotations.java:42)
> 	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.register(MetricsSystemImpl.java:224)
> 	at org.apache.hadoop.hbase.metrics.BaseSourceImpl.<init>(BaseSourceImpl.java:115)
> 	at org.apache.hadoop.hbase.zookeeper.MetricsZooKeeperSourceImpl.<init>(MetricsZooKeeperSourceImpl.java:56)
> 	at org.apache.hadoop.hbase.zookeeper.MetricsZooKeeperSourceImpl.<init>(MetricsZooKeeperSourceImpl.java:51)
> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> 	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> 	at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
> 	at java.lang.Class.newInstance(Class.java:383)
> 	at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:373)
> 	... 29 more
> Caused by: java.lang.ClassNotFoundException: org.apache.hadoop.hbase.metrics.MetricRegistry
> 	at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> 	at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> 	at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> 	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:312)
> 	at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> 	... 46 more
> {noformat}
> The errors are coming from within the map tasks.  Suggesting that they are missing some
jar/class dependency.  I know there are several new jars created in Hbase 1.4.0.  Are they
missing/should be included in PerformanceEvaluation.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message