incubator-hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon" <edwardy...@apache.org>
Subject Re: Debugging hama in eclipse
Date Wed, 22 Jun 2011 22:15:03 GMT
I found another problem about ClassNotFoundException.

java.lang.ClassNotFoundException: org.mortbay.jetty.annotations.Configuration

> It looks like the problem is actually this line in pom.xml (line 205)  :
>                   <excludeTransitive>true</excludeTransitive>

Instead removing 206 line, why don't we add dependencies as below?

    <dependency>
      <groupId>org.mortbay.jetty</groupId>
      <artifactId>jetty-annotations</artifactId>
      <version>6.1H.22</version>
    </dependency>
    <dependency>
      <groupId>org.slf4j</groupId>
      <artifactId>slf4j-log4j12</artifactId>
      <version>1.5.2</version>
    </dependency>
    <dependency>
      <groupId>commons-cli</groupId>
      <artifactId>commons-cli</artifactId>
      <version>1.2</version>
    </dependency>

Thanks.

On Thu, Jun 23, 2011 at 7:00 AM, Edward J. Yoon <edwardyoon@apache.org> wrote:
> Oh... sorry, you're right.
>
> It looks like a bug. We have to fix this.
>
> [INFO] Total time: 4:36.098s
> [INFO] Finished at: Thu Jun 23 06:58:28 KST 2011
> [INFO] Final Memory: 20M/51M
> [INFO] ------------------------------------------------------------------------
> edward@test ~/workspace/hama-trunk $ ls lib/
> commons-logging-1.0.4.jar  hadoop-test-0.20.2.jar
> jetty-util-6.1H.22.jar  log4j-1.2.15.jar
> hadoop-core-0.20.2.jar     jetty-6.1H.22.jar       junit-3.8.1.jar
>    zookeeper-3.3.1.jar
> edward@tweetple ~/workspace/hama-trunk $ bin/hama zookeeper
> -bash: bin/hama: Permission denied
> edward@test ~/workspace/hama-trunk $ chmod 777 bin/**
> edward@test ~/workspace/hama-trunk $ bin/hama zookeeper
> Exception in thread "main" java.lang.NoClassDefFoundError:
> org/apache/commons/cli/ParseException
>        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:59)
>        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>        at org.apache.hama.ZooKeeperRunner.main(ZooKeeperRunner.java:41)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.commons.cli.ParseException
>        at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
>        at java.security.AccessController.doPrivileged(Native Method)
>        at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
>        at java.lang.ClassLoader.loadClass(ClassLoader.java:303)
>        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>        at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>        at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:316)
>        ... 3 more
>
>
>
> On Thu, Jun 23, 2011 at 6:45 AM, Edward J. Yoon <edwardyoon@apache.org> wrote:
>> That's so weird.
>>
>> It works well on my test cluster.
>>
>> edward@slave:~/workspace/hama-trunk$ ls lib/
>> commons-logging-1.0.4.jar  hadoop-test-0.20.2.jar
>> jetty-util-6.1H.22.jar  log4j-1.2.15.jar
>> hadoop-core-0.20.2.jar     jetty-6.1H.22.jar       junit-3.8.1.jar
>>    zookeeper-3.3.1.jar
>> edward@slave:~/workspace/hama-trunk$ bin/hama jar
>> hama-examples-0.3.0-incubating-SNAPSHOT.jar pi
>> 11/06/23 06:41:55 DEBUG bsp.BSPJobClient: BSPJobClient.submitJobDir:
>> hdfs://slave.udanax.org:9000/tmp/hadoop-edward/bsp/system/submit_ebr5do
>> 11/06/23 06:41:57 INFO bsp.BSPJobClient: Running job: job_201106230640_0001
>> 11/06/23 06:42:00 INFO bsp.BSPJobClient: Current supersteps number: 0
>> 11/06/23 06:42:03 INFO bsp.BSPJobClient: Current supersteps number: 1
>> 11/06/23 06:42:03 INFO bsp.BSPJobClient: The total number of supersteps: 1
>> Estimated value of PI is 3.1534
>> Job Finished in 7.886 seconds
>> edward@slave:~/workspace/hama-trunk$
>>
>> On Thu, Jun 23, 2011 at 4:21 AM, Joseph Boyd
>> <joseph.boyd@cbsinteractive.com> wrote:
>>> Replying to my own post ...
>>>
>>> On Wed, Jun 22, 2011 at 11:05 AM, Joseph Boyd
>>> <joseph.boyd@cbsinteractive.com> wrote:
>>>> On Thu, Jun 16, 2011 at 4:37 AM, Ashish Agarwal <ashish.a77777@gmail.com>
wrote:
>>>>>
>>>>> When I try to run the hama daemons using the command -
>>>>> bin/hama zookeeper
>>>>>
>>>>> I get this error -
>>>
>>> It looks like the problem is actually this line in pom.xml (line 205)  :
>>>                    <excludeTransitive>true</excludeTransitive>
>>>
>>> This prevents commons-cli (and other transitive dependencies) from
>>> being copied to the lib dir like they should be.
>>>
>>> Removing line 205 fixes the NoClassDefFoundError for me.  I can open a
>>> JIRA for this after lunch, the command line is fairly broken with this
>>> line in, especially after several dependencies were (correctly)
>>> removed in HAMA-402.
>>>
>>>
>>> ...joe
>>>
>>>
>>>> I'm getting the same NoClassDefFoundError using the svn code.  The
>>>> commons-cli dependency was removed recently as part of HAMA-402, and
>>>> re-adding it fixes the NoClassDefFoundError.
>>>>
>>>> I believe commons-cli should have been pulled as part of the
>>>> hadoop-core dependency, and wasn't.  (though my maven skills are
>>>> rusty, and I'm no longer sure just how transitive dependencies are
>>>> pulled)
>>>>
>>>> While a quick work-around is to re-add the dependency here, the proper
>>>> fix is probably to fix the hadoop-core pom, and properly mark that it
>>>> depends on commons-cli.
>>>>
>>>>
>>>> ...joe
>>>>
>>>>
>>>>> Exception in thread "main" java.lang.NoClassDefFoundError:
>>>>> org/apache/commons/cli/ParseException
>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:59)
>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>>>>> at org.apache.hama.ZooKeeperRunner.main(ZooKeeperRunner.java:41)
>>>>> Caused by: java.lang.ClassNotFoundException:
>>>>> org.apache.commons.cli.ParseException
>>>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>>>>> at java.security.AccessController.doPrivileged(Native Method)
>>>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>>>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>>>>> ... 3 more
>>>>>
>>>>>
>>>>> I tried installing it a few times, but can't think of a solution to the
>>>>> problem. Can you please help ?
>>>>>
>>>>> Thank you
>>>>>
>>>>> Sincerely,
>>>>> Ashish
>>>>>
>>>>>
>>>>> On Wed, Jun 15, 2011 at 3:18 AM, Edward J. Yoon <edwardyoon@apache.org>wrote:
>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I would suggest following steps to debug:
>>>>>>
>>>>>> 1. Install the Hadoop 0.20.x on your laptop.
>>>>>> 2. Run the Hama deamons with 3 terminals.
>>>>>>
>>>>>> % bin/hama zookeeper
>>>>>> % bin/hama bspmaster
>>>>>> % bin/hama groom
>>>>>>
>>>>>> 3. Use the logging API to log debug.
>>>>>> For example, LOG.debug("like this");
>>>>>>
>>>>>> On Wed, Jun 15, 2011 at 6:48 PM, Ashish Agarwal <ashish.a77777@gmail.com>
>>>>>> wrote:
>>>>>> > Hi,
>>>>>> >
>>>>>> > I was wondering if anyone can give me instructions on how to
debug code
>>>>>> in
>>>>>> > distributed or pseudo-distributed mode ? It would really help
me.
>>>>>> >
>>>>>> > Thank you
>>>>>> >
>>>>>> > Sincerely,
>>>>>> > Ashish
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> > On Sat, Jun 11, 2011 at 3:41 AM, Ashish Agarwal <ashish.a77777@gmail.com
>>>>>> >wrote:
>>>>>> >
>>>>>> >>
>>>>>> >> Hi,
>>>>>> >>
>>>>>> >> I have written some code for my GSoC task (Hama-367) and
am now trying
>>>>>> to
>>>>>> >> debug.
>>>>>> >>
>>>>>> >> I am trying to set up the environment into pseudo-distributed
mode so I
>>>>>> can
>>>>>> >> debug. I think the value of bsp.master.address in hama-site.xml
has to
>>>>>> be
>>>>>> >> local as this means LocalBSPRunner (as localhost would call
127.0.0.1
>>>>>> >> server), but where do I specify that in the code in eclipse
?
>>>>>> >>
>>>>>> >> My next question is - what is the difference between MiniBSPCluster
and
>>>>>> >> LocalBSPRunner  and which file should I be using to be
able to step
>>>>>> through
>>>>>> >> the code ?
>>>>>> >>
>>>>>> >> These are the processes currently running on my machine
-
>>>>>> >>
>>>>>> >> 6112 NameNode
>>>>>> >> 3319 ZooKeeperRunner
>>>>>> >> 3018
>>>>>> >> 6449 SecondaryNameNode
>>>>>> >> 5709 HMaster
>>>>>> >> 5643 HQuorumPeer
>>>>>> >> 6684 TaskTracker
>>>>>> >> 6526 JobTracker
>>>>>> >> 8074 Jps
>>>>>> >> 6277 DataNode
>>>>>> >>
>>>>>> >> Thank you,
>>>>>> >> Ashish
>>>>>> >>
>>>>>> >
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Best Regards, Edward J. Yoon
>>>>>> @eddieyoon
>>>>>>
>>>>>
>>>>
>>>
>>
>>
>>
>> --
>> Best Regards, Edward J. Yoon
>> @eddieyoon
>>
>
>
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon
>



-- 
Best Regards, Edward J. Yoon
@eddieyoon

Mime
View raw message