hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon" <edwardy...@apache.org>
Subject Re: Get struck after upgrading hadoop to 2.5.1
Date Mon, 22 Sep 2014 12:07:39 GMT
> May I file this one on JIRA?

Please do. It seems like a bug. Thanks!

On Mon, Sep 22, 2014 at 3:09 PM, Dongjin Lee <dongjin.lee.kr@gmail.com> wrote:
> Hello.
>
> I retried it with updating ${HAMA_HOME}/lib, but it also failed. I'm
> increasingly certain that this problem is caused by hadoop 2.5.x.
>
> OK, Let me explain in detail. I cleaned & rebuilded hama 0.6.4 with hadoop
> 2.5.1, using 'mvn clean package -Phadoop2 -Dhadoop.version=2.5.1'. After
> that, I substituted the whole installation to my new build, except conf
> directory.
>
> But when I run 'start-bspd.sh', it crashes with following error messages:
>
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/commons/collections/map/UnmodifiableMap
> at
> org.apache.hadoop.conf.Configuration$DeprecationContext.<init>(Configuration.java:394)
> at org.apache.hadoop.conf.Configuration.<clinit>(Configuration.java:434)
> at org.apache.hama.zookeeper.ZKServerTool.main(ZKServerTool.java:37)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.commons.collections.map.UnmodifiableMap
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
> ... 3 more
> starting bspmaster, logging to
> /home/dongjinleekr/opt/hama-0.6.4/bin/../logs/hama-dongjinleekr-bspmaster-dongjinleekr-unix.out
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/commons/collections/map/UnmodifiableMap
> at
> org.apache.hadoop.conf.Configuration$DeprecationContext.<init>(Configuration.java:394)
> at org.apache.hadoop.conf.Configuration.<clinit>(Configuration.java:434)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:62)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> at org.apache.hama.BSPMasterRunner.main(BSPMasterRunner.java:56)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.commons.collections.map.UnmodifiableMap
> at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> at java.security.AccessController.doPrivileged(Native Method)
> localhost: starting groom, logging to
> /home/dongjinleekr/opt/hama-0.6.4/bin/../logs/hama-dongjinleekr-groom-dongjinleekr-unix.out
> localhost: Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/commons/collections/map/UnmodifiableMap
> localhost: at
> org.apache.hadoop.conf.Configuration$DeprecationContext.<init>(Configuration.java:394)
> localhost: at
> org.apache.hadoop.conf.Configuration.<clinit>(Configuration.java:434)
> localhost: at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:62)
> localhost: at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> localhost: at
> org.apache.hama.GroomServerRunner.main(GroomServerRunner.java:59)
> localhost: Caused by: java.lang.ClassNotFoundException:
> org.apache.commons.collections.map.UnmodifiableMap
> localhost: at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
> localhost: at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
> localhost: at java.security.AccessController.doPrivileged(Native Method)
>
> The following is the contents of my hama-site.xml. As you can see, I run my
> hama distribution in pseudo-distributed mode, with separate zookeeper nodes.
>
> <?xml version="1.0"?>
> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
> <configuration>
> <property>
> <name>bsp.master.address</name>
> <value>localhost:40000</value>
> </property>
> <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:9000</value>
> </property>
> <property>
> <name>hama.zookeeper.property.clientPort</name>
> <value>2181</value>
> </property>
> </configuration>
>
> How do you think about this problem? May I file this one on JIRA? I am now
> inspecting ZKServerTool.main() and BSPMasterRunner.main(), seeking a
> solution.
>
> +1. What log do you exactly mean? Master branch's log?
>
> Thanks,
> Dongjin
>
> On Mon, Sep 22, 2014 at 8:58 AM, Edward J. Yoon <edwardyoon@apache.org>
> wrote:
>
>> Hi, have you looked at the Master's log?
>>
>> And, you also need to replace hadoop jar files in ${HAMA_HOME}/lib folder.
>>
>> On Mon, Sep 22, 2014 at 6:46 AM, Dongjin Lee <dongjin.lee.kr@gmail.com>
>> wrote:
>> > Hello All.
>> >
>> > I am a developer who have been run hama with hadoop 1.2.1. Last weekend,
>> I
>> > upgraded my hadoop distribution to latest one, version 2.5.1. Although
>> all
>> > other things run well, my hama distribution (0.6.4) does not run properly
>> > anymore.
>> >
>> > Let's explain a bit more in detail: I rebuilded hama with hadoop2 profile
>> > and updated all jar files. I never changed any hama config file. However,
>> > when I try to run hama job like following, it fails to connect to bsp
>> > master.
>> >
>> > dongjinleekr@dongjinleekr-unix:~$ hama jar hama-examples-0.6.4.jar gen
>> > fastgen 100 10 randomgraph 2
>> > 14/09/22 06:30:59 INFO ipc.Client: Retrying connect to server: localhost/
>> > 127.0.0.1:40000. Already tried 0 time(s); retry policy is
>> > RetryUpToMaximumCountWithFixedSleep(maxRetries=10, sleepTime=1 SECONDS)
>> > ...
>> > java.net.ConnectException: Call to localhost/127.0.0.1:40000 failed on
>> > connection exception: java.net.ConnectException: Connection refused
>> >
>> > I checked the processes with jps, but all the daemons were running well.
>> Is
>> > there any other person who experienced the same? I am now suspecting that
>> > some updates applied to hadoop 2.5.0 are causing this problem.
>> >
>> > Any guidance will be greatly helped.
>> >
>> > Thanks in Advance,
>> > Dongjin Lee
>> >
>> > --
>> > *Dongjin Lee*
>> >
>> >
>> > *Oracle Server Technologies Group.So interested in massive-scale machine
>> > learning.facebook: www.facebook.com/dongjin.lee.kr
>> > <http://www.facebook.com/dongjin.lee.kr>linkedin:
>> > kr.linkedin.com/in/dongjinleekr
>> > <http://kr.linkedin.com/in/dongjinleekr>github:
>> > <http://goog_969573159>github.com/dongjinleekr
>> > <http://github.com/dongjinleekr>twitter: www.twitter.com/dongjinleekr
>> > <http://www.twitter.com/dongjinleekr>*
>>
>>
>>
>> --
>> Best Regards, Edward J. Yoon
>> CEO at DataSayer Co., Ltd.
>>
>
>
>
> --
> *Dongjin Lee*
>
>
> *Oracle Server Technologies Group.So interested in massive-scale machine
> learning.facebook: www.facebook.com/dongjin.lee.kr
> <http://www.facebook.com/dongjin.lee.kr>linkedin:
> kr.linkedin.com/in/dongjinleekr
> <http://kr.linkedin.com/in/dongjinleekr>github:
> <http://goog_969573159>github.com/dongjinleekr
> <http://github.com/dongjinleekr>twitter: www.twitter.com/dongjinleekr
> <http://www.twitter.com/dongjinleekr>*



-- 
Best Regards, Edward J. Yoon
CEO at DataSayer Co., Ltd.

Mime
View raw message