incubator-hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon" <edwardy...@apache.org>
Subject Re: Debugging hama in eclipse
Date Wed, 22 Jun 2011 22:00:19 GMT
Oh... sorry, you're right.

It looks like a bug. We have to fix this.

[INFO] Total time: 4:36.098s
[INFO] Finished at: Thu Jun 23 06:58:28 KST 2011
[INFO] Final Memory: 20M/51M
[INFO] ------------------------------------------------------------------------
edward@test ~/workspace/hama-trunk $ ls lib/
commons-logging-1.0.4.jar  hadoop-test-0.20.2.jar
jetty-util-6.1H.22.jar  log4j-1.2.15.jar
hadoop-core-0.20.2.jar     jetty-6.1H.22.jar       junit-3.8.1.jar
    zookeeper-3.3.1.jar
edward@tweetple ~/workspace/hama-trunk $ bin/hama zookeeper
-bash: bin/hama: Permission denied
edward@test ~/workspace/hama-trunk $ chmod 777 bin/**
edward@test ~/workspace/hama-trunk $ bin/hama zookeeper
Exception in thread "main" java.lang.NoClassDefFoundError:
org/apache/commons/cli/ParseException
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:59)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
	at org.apache.hama.ZooKeeperRunner.main(ZooKeeperRunner.java:41)
Caused by: java.lang.ClassNotFoundException:
org.apache.commons.cli.ParseException
	at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
	at java.security.AccessController.doPrivileged(Native Method)
	at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:303)
	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
	at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
	at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:316)
	... 3 more



On Thu, Jun 23, 2011 at 6:45 AM, Edward J. Yoon <edwardyoon@apache.org> wrote:
> That's so weird.
>
> It works well on my test cluster.
>
> edward@slave:~/workspace/hama-trunk$ ls lib/
> commons-logging-1.0.4.jar  hadoop-test-0.20.2.jar
> jetty-util-6.1H.22.jar  log4j-1.2.15.jar
> hadoop-core-0.20.2.jar     jetty-6.1H.22.jar       junit-3.8.1.jar
>    zookeeper-3.3.1.jar
> edward@slave:~/workspace/hama-trunk$ bin/hama jar
> hama-examples-0.3.0-incubating-SNAPSHOT.jar pi
> 11/06/23 06:41:55 DEBUG bsp.BSPJobClient: BSPJobClient.submitJobDir:
> hdfs://slave.udanax.org:9000/tmp/hadoop-edward/bsp/system/submit_ebr5do
> 11/06/23 06:41:57 INFO bsp.BSPJobClient: Running job: job_201106230640_0001
> 11/06/23 06:42:00 INFO bsp.BSPJobClient: Current supersteps number: 0
> 11/06/23 06:42:03 INFO bsp.BSPJobClient: Current supersteps number: 1
> 11/06/23 06:42:03 INFO bsp.BSPJobClient: The total number of supersteps: 1
> Estimated value of PI is 3.1534
> Job Finished in 7.886 seconds
> edward@slave:~/workspace/hama-trunk$
>
> On Thu, Jun 23, 2011 at 4:21 AM, Joseph Boyd
> <joseph.boyd@cbsinteractive.com> wrote:
>> Replying to my own post ...
>>
>> On Wed, Jun 22, 2011 at 11:05 AM, Joseph Boyd
>> <joseph.boyd@cbsinteractive.com> wrote:
>>> On Thu, Jun 16, 2011 at 4:37 AM, Ashish Agarwal <ashish.a77777@gmail.com>
wrote:
>>>>
>>>> When I try to run the hama daemons using the command -
>>>> bin/hama zookeeper
>>>>
>>>> I get this error -
>>
>> It looks like the problem is actually this line in pom.xml (line 205)  :
>>                    <excludeTransitive>true</excludeTransitive>
>>
>> This prevents commons-cli (and other transitive dependencies) from
>> being copied to the lib dir like they should be.
>>
>> Removing line 205 fixes the NoClassDefFoundError for me.  I can open a
>> JIRA for this after lunch, the command line is fairly broken with this
>> line in, especially after several dependencies were (correctly)
>> removed in HAMA-402.
>>
>>
>> ...joe
>>
>>
>>> I'm getting the same NoClassDefFoundError using the svn code.  The
>>> commons-cli dependency was removed recently as part of HAMA-402, and
>>> re-adding it fixes the NoClassDefFoundError.
>>>
>>> I believe commons-cli should have been pulled as part of the
>>> hadoop-core dependency, and wasn't.  (though my maven skills are
>>> rusty, and I'm no longer sure just how transitive dependencies are
>>> pulled)
>>>
>>> While a quick work-around is to re-add the dependency here, the proper
>>> fix is probably to fix the hadoop-core pom, and properly mark that it
>>> depends on commons-cli.
>>>
>>>
>>> ...joe
>>>
>>>
>>>> Exception in thread "main" java.lang.NoClassDefFoundError:
>>>> org/apache/commons/cli/ParseException
>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:59)
>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>>>> at org.apache.hama.ZooKeeperRunner.main(ZooKeeperRunner.java:41)
>>>> Caused by: java.lang.ClassNotFoundException:
>>>> org.apache.commons.cli.ParseException
>>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>>>> at java.security.AccessController.doPrivileged(Native Method)
>>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>>>> ... 3 more
>>>>
>>>>
>>>> I tried installing it a few times, but can't think of a solution to the
>>>> problem. Can you please help ?
>>>>
>>>> Thank you
>>>>
>>>> Sincerely,
>>>> Ashish
>>>>
>>>>
>>>> On Wed, Jun 15, 2011 at 3:18 AM, Edward J. Yoon <edwardyoon@apache.org>wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I would suggest following steps to debug:
>>>>>
>>>>> 1. Install the Hadoop 0.20.x on your laptop.
>>>>> 2. Run the Hama deamons with 3 terminals.
>>>>>
>>>>> % bin/hama zookeeper
>>>>> % bin/hama bspmaster
>>>>> % bin/hama groom
>>>>>
>>>>> 3. Use the logging API to log debug.
>>>>> For example, LOG.debug("like this");
>>>>>
>>>>> On Wed, Jun 15, 2011 at 6:48 PM, Ashish Agarwal <ashish.a77777@gmail.com>
>>>>> wrote:
>>>>> > Hi,
>>>>> >
>>>>> > I was wondering if anyone can give me instructions on how to debug
code
>>>>> in
>>>>> > distributed or pseudo-distributed mode ? It would really help me.
>>>>> >
>>>>> > Thank you
>>>>> >
>>>>> > Sincerely,
>>>>> > Ashish
>>>>> >
>>>>> >
>>>>> >
>>>>> > On Sat, Jun 11, 2011 at 3:41 AM, Ashish Agarwal <ashish.a77777@gmail.com
>>>>> >wrote:
>>>>> >
>>>>> >>
>>>>> >> Hi,
>>>>> >>
>>>>> >> I have written some code for my GSoC task (Hama-367) and am
now trying
>>>>> to
>>>>> >> debug.
>>>>> >>
>>>>> >> I am trying to set up the environment into pseudo-distributed
mode so I
>>>>> can
>>>>> >> debug. I think the value of bsp.master.address in hama-site.xml
has to
>>>>> be
>>>>> >> local as this means LocalBSPRunner (as localhost would call
127.0.0.1
>>>>> >> server), but where do I specify that in the code in eclipse
?
>>>>> >>
>>>>> >> My next question is - what is the difference between MiniBSPCluster
and
>>>>> >> LocalBSPRunner  and which file should I be using to be able
to step
>>>>> through
>>>>> >> the code ?
>>>>> >>
>>>>> >> These are the processes currently running on my machine -
>>>>> >>
>>>>> >> 6112 NameNode
>>>>> >> 3319 ZooKeeperRunner
>>>>> >> 3018
>>>>> >> 6449 SecondaryNameNode
>>>>> >> 5709 HMaster
>>>>> >> 5643 HQuorumPeer
>>>>> >> 6684 TaskTracker
>>>>> >> 6526 JobTracker
>>>>> >> 8074 Jps
>>>>> >> 6277 DataNode
>>>>> >>
>>>>> >> Thank you,
>>>>> >> Ashish
>>>>> >>
>>>>> >
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Best Regards, Edward J. Yoon
>>>>> @eddieyoon
>>>>>
>>>>
>>>
>>
>
>
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon
>



-- 
Best Regards, Edward J. Yoon
@eddieyoon

Mime
View raw message