hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joseph Boyd <joseph.b...@cbsinteractive.com>
Subject Re: Debugging hama in eclipse
Date Wed, 22 Jun 2011 19:21:34 GMT
Replying to my own post ...

On Wed, Jun 22, 2011 at 11:05 AM, Joseph Boyd
<joseph.boyd@cbsinteractive.com> wrote:
> On Thu, Jun 16, 2011 at 4:37 AM, Ashish Agarwal <ashish.a77777@gmail.com> wrote:
>>
>> When I try to run the hama daemons using the command -
>> bin/hama zookeeper
>>
>> I get this error -

It looks like the problem is actually this line in pom.xml (line 205)  :
                    <excludeTransitive>true</excludeTransitive>

This prevents commons-cli (and other transitive dependencies) from
being copied to the lib dir like they should be.

Removing line 205 fixes the NoClassDefFoundError for me.  I can open a
JIRA for this after lunch, the command line is fairly broken with this
line in, especially after several dependencies were (correctly)
removed in HAMA-402.


...joe


> I'm getting the same NoClassDefFoundError using the svn code.  The
> commons-cli dependency was removed recently as part of HAMA-402, and
> re-adding it fixes the NoClassDefFoundError.
>
> I believe commons-cli should have been pulled as part of the
> hadoop-core dependency, and wasn't.  (though my maven skills are
> rusty, and I'm no longer sure just how transitive dependencies are
> pulled)
>
> While a quick work-around is to re-add the dependency here, the proper
> fix is probably to fix the hadoop-core pom, and properly mark that it
> depends on commons-cli.
>
>
> ...joe
>
>
>> Exception in thread "main" java.lang.NoClassDefFoundError:
>> org/apache/commons/cli/ParseException
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:59)
>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>> at org.apache.hama.ZooKeeperRunner.main(ZooKeeperRunner.java:41)
>> Caused by: java.lang.ClassNotFoundException:
>> org.apache.commons.cli.ParseException
>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>> at java.security.AccessController.doPrivileged(Native Method)
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>> ... 3 more
>>
>>
>> I tried installing it a few times, but can't think of a solution to the
>> problem. Can you please help ?
>>
>> Thank you
>>
>> Sincerely,
>> Ashish
>>
>>
>> On Wed, Jun 15, 2011 at 3:18 AM, Edward J. Yoon <edwardyoon@apache.org>wrote:
>>
>>> Hi,
>>>
>>> I would suggest following steps to debug:
>>>
>>> 1. Install the Hadoop 0.20.x on your laptop.
>>> 2. Run the Hama deamons with 3 terminals.
>>>
>>> % bin/hama zookeeper
>>> % bin/hama bspmaster
>>> % bin/hama groom
>>>
>>> 3. Use the logging API to log debug.
>>> For example, LOG.debug("like this");
>>>
>>> On Wed, Jun 15, 2011 at 6:48 PM, Ashish Agarwal <ashish.a77777@gmail.com>
>>> wrote:
>>> > Hi,
>>> >
>>> > I was wondering if anyone can give me instructions on how to debug code
>>> in
>>> > distributed or pseudo-distributed mode ? It would really help me.
>>> >
>>> > Thank you
>>> >
>>> > Sincerely,
>>> > Ashish
>>> >
>>> >
>>> >
>>> > On Sat, Jun 11, 2011 at 3:41 AM, Ashish Agarwal <ashish.a77777@gmail.com
>>> >wrote:
>>> >
>>> >>
>>> >> Hi,
>>> >>
>>> >> I have written some code for my GSoC task (Hama-367) and am now trying
>>> to
>>> >> debug.
>>> >>
>>> >> I am trying to set up the environment into pseudo-distributed mode so
I
>>> can
>>> >> debug. I think the value of bsp.master.address in hama-site.xml has
to
>>> be
>>> >> local as this means LocalBSPRunner (as localhost would call 127.0.0.1
>>> >> server), but where do I specify that in the code in eclipse ?
>>> >>
>>> >> My next question is - what is the difference between MiniBSPCluster
and
>>> >> LocalBSPRunner  and which file should I be using to be able to step
>>> through
>>> >> the code ?
>>> >>
>>> >> These are the processes currently running on my machine -
>>> >>
>>> >> 6112 NameNode
>>> >> 3319 ZooKeeperRunner
>>> >> 3018
>>> >> 6449 SecondaryNameNode
>>> >> 5709 HMaster
>>> >> 5643 HQuorumPeer
>>> >> 6684 TaskTracker
>>> >> 6526 JobTracker
>>> >> 8074 Jps
>>> >> 6277 DataNode
>>> >>
>>> >> Thank you,
>>> >> Ashish
>>> >>
>>> >
>>>
>>>
>>>
>>> --
>>> Best Regards, Edward J. Yoon
>>> @eddieyoon
>>>
>>
>

Mime
View raw message