gora-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ferdy Galema (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (GORA-76) Upgrade to Hadoop 1.0.0
Date Wed, 07 Mar 2012 16:36:59 GMT

    [ https://issues.apache.org/jira/browse/GORA-76?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13224485#comment-13224485
] 

Ferdy Galema commented on GORA-76:
----------------------------------

Ok I got a bit more working now, but not yet fully done.

I solved the jackson issue by excluding it from hadoop (it include jackson 1.0.1) and including
it explicitely with specific version 1.4.2 because this what avro 1.3.3 requires. So that
is out of the way.

Next problem is this: When I run the gora-core tests the data directory cannot be initialized
it seems. This is partial output of org.apache.gora.avro.mapreduce.TestDataFileAvroStoreMapReduce
when it goes right:

Starting DataNode 0 with dfs.data.dir: /home/ferdy/workspace/gora/gora-core/target/test-data/dfs/data/data1,/home/ferdy/workspace/gora/gora-core/target/test-data/dfs/data/data2
12/03/07 17:26:40 INFO common.Storage: Storage directory /home/ferdy/workspace/gora/gora-core/target/test-data/dfs/data/data1
is not formatted.
12/03/07 17:26:40 INFO common.Storage: Formatting ...
12/03/07 17:26:40 INFO common.Storage: Storage directory /home/ferdy/workspace/gora/gora-core/target/test-data/dfs/data/data2
is not formatted.
12/03/07 17:26:40 INFO common.Storage: Formatting ...
12/03/07 17:26:40 INFO datanode.DataNode: Registered FSDatasetStatusMBean
12/03/07 17:26:40 INFO datanode.DataNode: Opened info server at 35222
12/03/07 17:26:40 INFO datanode.DataNode: Balancing bandwith is 1048576 bytes/s

Now output when it goes wrong (when I apply patch)

Starting DataNode 0 with dfs.data.dir: /home/ferdy/workspace/gora/gora-core/target/test-data/dfs/data/data1,/home/ferdy/workspace/gora/gora-core/target/test-data/dfs/data/data2
12/03/07 17:33:21 WARN impl.MetricsSystemImpl: Metrics system not started: Cannot locate configuration:
tried hadoop-metrics2-datanode.properties, hadoop-metrics2.properties
12/03/07 17:33:21 WARN util.MBeans: Hadoop:service=DataNode,name=MetricsSystem,sub=Control
javax.management.InstanceAlreadyExistsException: MXBean already registered with name Hadoop:service=NameNode,name=MetricsSystem,sub=Control
	at com.sun.jmx.mbeanserver.MXBeanLookup.addReference(MXBeanLookup.java:120)
	at com.sun.jmx.mbeanserver.MXBeanSupport.register(MXBeanSupport.java:143)
	at com.sun.jmx.mbeanserver.MBeanSupport.preRegister2(MBeanSupport.java:183)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:941)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:917)
	at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:312)
	at com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:482)
	at org.apache.hadoop.metrics2.util.MBeans.register(MBeans.java:56)
	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.initSystemMBean(MetricsSystemImpl.java:500)
	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.init(MetricsSystemImpl.java:140)
	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.init(DefaultMetricsSystem.java:40)
	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.initialize(DefaultMetricsSystem.java:50)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1491)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1467)
	at org.apache.hadoop.hdfs.MiniDFSCluster.startDataNodes(MiniDFSCluster.java:417)
	at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:280)
	at org.apache.hadoop.hdfs.MiniDFSCluster.<init>(MiniDFSCluster.java:124)
	at org.apache.hadoop.mapred.HadoopTestCase.setUp(HadoopTestCase.java:148)
	at org.apache.gora.mapreduce.DataStoreMapReduceTestBase.setUp(DataStoreMapReduceTestBase.java:54)
	at junit.framework.TestCase.runBare(TestCase.java:132)
	at junit.framework.TestResult$1.protect(TestResult.java:110)
	at junit.framework.TestResult.runProtected(TestResult.java:128)
	at junit.framework.TestResult.run(TestResult.java:113)
	at junit.framework.TestCase.run(TestCase.java:124)
	at junit.framework.TestSuite.runTest(TestSuite.java:243)
	at junit.framework.TestSuite.run(TestSuite.java:238)
	at org.junit.internal.runners.JUnit38ClassRunner.run(JUnit38ClassRunner.java:83)
	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:236)
	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:134)
	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:113)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.apache.maven.surefire.util.ReflectionUtils.invokeMethodWithArray(ReflectionUtils.java:189)
	at org.apache.maven.surefire.booter.ProviderFactory$ProviderProxy.invoke(ProviderFactory.java:165)
	at org.apache.maven.surefire.booter.ProviderFactory.invokeProvider(ProviderFactory.java:85)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:103)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:74)
12/03/07 17:33:21 WARN datanode.DataNode: Invalid directory in dfs.data.dir: Incorrect permission
for /home/ferdy/workspace/gora/gora-core/target/test-data/dfs/data/data1, expected: rwxr-xr-x,
while actual: rwxrwxr-x
12/03/07 17:33:21 WARN datanode.DataNode: Invalid directory in dfs.data.dir: Incorrect permission
for /home/ferdy/workspace/gora/gora-core/target/test-data/dfs/data/data2, expected: rwxr-xr-x,
while actual: rwxrwxr-x
12/03/07 17:33:21 ERROR datanode.DataNode: All directories in dfs.data.dir are invalid.

Not sure what exactly is the cause. Note that HBASE-4709 describes a similar problem. Any
help would be useful.

                
> Upgrade to Hadoop 1.0.0
> -----------------------
>
>                 Key: GORA-76
>                 URL: https://issues.apache.org/jira/browse/GORA-76
>             Project: Apache Gora
>          Issue Type: Improvement
>          Components: build process, storage
>    Affects Versions: 0.2
>            Reporter: Lewis John McGibbney
>             Fix For: 0.2
>
>         Attachments: GORA-76.patch
>
>
> This was driven entirely by Juliens recent issue over in Nutchland.
> Somewhere down the line this will be useful.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message