hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon" <edwardy...@apache.org>
Subject Re: Debugging hama in eclipse
Date Wed, 22 Jun 2011 21:45:25 GMT
That's so weird.

It works well on my test cluster.

edward@slave:~/workspace/hama-trunk$ ls lib/
commons-logging-1.0.4.jar  hadoop-test-0.20.2.jar
jetty-util-6.1H.22.jar  log4j-1.2.15.jar
hadoop-core-0.20.2.jar     jetty-6.1H.22.jar       junit-3.8.1.jar
    zookeeper-3.3.1.jar
edward@slave:~/workspace/hama-trunk$ bin/hama jar
hama-examples-0.3.0-incubating-SNAPSHOT.jar pi
11/06/23 06:41:55 DEBUG bsp.BSPJobClient: BSPJobClient.submitJobDir:
hdfs://slave.udanax.org:9000/tmp/hadoop-edward/bsp/system/submit_ebr5do
11/06/23 06:41:57 INFO bsp.BSPJobClient: Running job: job_201106230640_0001
11/06/23 06:42:00 INFO bsp.BSPJobClient: Current supersteps number: 0
11/06/23 06:42:03 INFO bsp.BSPJobClient: Current supersteps number: 1
11/06/23 06:42:03 INFO bsp.BSPJobClient: The total number of supersteps: 1
Estimated value of PI is 3.1534
Job Finished in 7.886 seconds
edward@slave:~/workspace/hama-trunk$

On Thu, Jun 23, 2011 at 4:21 AM, Joseph Boyd
<joseph.boyd@cbsinteractive.com> wrote:
> Replying to my own post ...
>
> On Wed, Jun 22, 2011 at 11:05 AM, Joseph Boyd
> <joseph.boyd@cbsinteractive.com> wrote:
>> On Thu, Jun 16, 2011 at 4:37 AM, Ashish Agarwal <ashish.a77777@gmail.com> wrote:
>>>
>>> When I try to run the hama daemons using the command -
>>> bin/hama zookeeper
>>>
>>> I get this error -
>
> It looks like the problem is actually this line in pom.xml (line 205)  :
>                    <excludeTransitive>true</excludeTransitive>
>
> This prevents commons-cli (and other transitive dependencies) from
> being copied to the lib dir like they should be.
>
> Removing line 205 fixes the NoClassDefFoundError for me.  I can open a
> JIRA for this after lunch, the command line is fairly broken with this
> line in, especially after several dependencies were (correctly)
> removed in HAMA-402.
>
>
> ...joe
>
>
>> I'm getting the same NoClassDefFoundError using the svn code.  The
>> commons-cli dependency was removed recently as part of HAMA-402, and
>> re-adding it fixes the NoClassDefFoundError.
>>
>> I believe commons-cli should have been pulled as part of the
>> hadoop-core dependency, and wasn't.  (though my maven skills are
>> rusty, and I'm no longer sure just how transitive dependencies are
>> pulled)
>>
>> While a quick work-around is to re-add the dependency here, the proper
>> fix is probably to fix the hadoop-core pom, and properly mark that it
>> depends on commons-cli.
>>
>>
>> ...joe
>>
>>
>>> Exception in thread "main" java.lang.NoClassDefFoundError:
>>> org/apache/commons/cli/ParseException
>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:59)
>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>>> at org.apache.hama.ZooKeeperRunner.main(ZooKeeperRunner.java:41)
>>> Caused by: java.lang.ClassNotFoundException:
>>> org.apache.commons.cli.ParseException
>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>>> at java.security.AccessController.doPrivileged(Native Method)
>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>>> ... 3 more
>>>
>>>
>>> I tried installing it a few times, but can't think of a solution to the
>>> problem. Can you please help ?
>>>
>>> Thank you
>>>
>>> Sincerely,
>>> Ashish
>>>
>>>
>>> On Wed, Jun 15, 2011 at 3:18 AM, Edward J. Yoon <edwardyoon@apache.org>wrote:
>>>
>>>> Hi,
>>>>
>>>> I would suggest following steps to debug:
>>>>
>>>> 1. Install the Hadoop 0.20.x on your laptop.
>>>> 2. Run the Hama deamons with 3 terminals.
>>>>
>>>> % bin/hama zookeeper
>>>> % bin/hama bspmaster
>>>> % bin/hama groom
>>>>
>>>> 3. Use the logging API to log debug.
>>>> For example, LOG.debug("like this");
>>>>
>>>> On Wed, Jun 15, 2011 at 6:48 PM, Ashish Agarwal <ashish.a77777@gmail.com>
>>>> wrote:
>>>> > Hi,
>>>> >
>>>> > I was wondering if anyone can give me instructions on how to debug code
>>>> in
>>>> > distributed or pseudo-distributed mode ? It would really help me.
>>>> >
>>>> > Thank you
>>>> >
>>>> > Sincerely,
>>>> > Ashish
>>>> >
>>>> >
>>>> >
>>>> > On Sat, Jun 11, 2011 at 3:41 AM, Ashish Agarwal <ashish.a77777@gmail.com
>>>> >wrote:
>>>> >
>>>> >>
>>>> >> Hi,
>>>> >>
>>>> >> I have written some code for my GSoC task (Hama-367) and am now
trying
>>>> to
>>>> >> debug.
>>>> >>
>>>> >> I am trying to set up the environment into pseudo-distributed mode
so I
>>>> can
>>>> >> debug. I think the value of bsp.master.address in hama-site.xml
has to
>>>> be
>>>> >> local as this means LocalBSPRunner (as localhost would call 127.0.0.1
>>>> >> server), but where do I specify that in the code in eclipse ?
>>>> >>
>>>> >> My next question is - what is the difference between MiniBSPCluster
and
>>>> >> LocalBSPRunner  and which file should I be using to be able to
step
>>>> through
>>>> >> the code ?
>>>> >>
>>>> >> These are the processes currently running on my machine -
>>>> >>
>>>> >> 6112 NameNode
>>>> >> 3319 ZooKeeperRunner
>>>> >> 3018
>>>> >> 6449 SecondaryNameNode
>>>> >> 5709 HMaster
>>>> >> 5643 HQuorumPeer
>>>> >> 6684 TaskTracker
>>>> >> 6526 JobTracker
>>>> >> 8074 Jps
>>>> >> 6277 DataNode
>>>> >>
>>>> >> Thank you,
>>>> >> Ashish
>>>> >>
>>>> >
>>>>
>>>>
>>>>
>>>> --
>>>> Best Regards, Edward J. Yoon
>>>> @eddieyoon
>>>>
>>>
>>
>



-- 
Best Regards, Edward J. Yoon
@eddieyoon

Mime
View raw message