incubator-hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ashish Agarwal <ashish.a77...@gmail.com>
Subject Re: Debugging hama in eclipse
Date Mon, 27 Jun 2011 04:18:19 GMT
Hi,

I think the error might be because I have to start the BSP Master
server before trying to run the BSP Master node.

But, when I run the command bin/start-bspd.sh, hama tries to use
openjdk for some reason. I have set the default version to sun JDK and
my JAVA_HOME points to java-6-sun.

Any idea why I might be getting this error ?

ashish@ubuntu:~/Desktop/Project/hama-trunk$ bin/start-bspd.sh
localhost: starting zookeeper, logging to
/home/ashish/Desktop/Project/hama-trunk/bin/../logs/hama-ashish-zookeeper-ubuntu.out
localhost: /home/ashish/Desktop/Project/hama-trunk/bin/../bin/hama:
line 186: /home/ashish/Desktop/Project/hama-trunk/”/usr/lib/jvm/java-6-openjdk”/bin/java:
No such file or directory
starting bspmaster, logging to
/home/ashish/Desktop/Project/hama-trunk/bin/../logs/hama-ashish-bspmaster-ubuntu.out
localhost: starting groom, logging to
/home/ashish/Desktop/Project/hama-trunk/bin/../logs/hama-ashish-groom-ubuntu.out
localhost: /home/ashish/Desktop/Project/hama-trunk/bin/../bin/hama:
line 186: /home/ashish/Desktop/Project/hama-trunk/”/usr/lib/jvm/java-6-openjdk”/bin/java:
No such file or directory
ashish@ubuntu:~/Desktop/Project/hama-trunk$


Thanks
Ashish


On Thu, Jun 23, 2011 at 2:49 AM, Edward J. Yoon <edwardyoon@apache.org> wrote:
> Hey,
>
> Let's see the error message.
>
>> java.net.UnknownHostException: Invalid hostname for server: local
>
> and
>
>> at org.apache.hama.bsp.BSPMaster.<init>(BSPMaster.java:248)
>
> line 248:   host = getAddress(conf).getHostName();
>
> I think, you have to try to fix/debug your problem yourself before calling.
>
> On Thu, Jun 23, 2011 at 6:33 PM, Ashish Agarwal <ashish.a77777@gmail.com> wrote:
>> It didn't work.
>> I have attached my  hama-site.xml.template file.
>>
>>
>> On Thu, Jun 23, 2011 at 2:16 AM, Edward J. Yoon <edwardyoon@apache.org> wrote:
>>> Have you tried to run with <value>ubuntu</value>?
>>>
>>> On Thu, Jun 23, 2011 at 6:04 PM, Ashish Agarwal <ashish.a77777@gmail.com>
wrote:
>>>> <property>
>>>>   <name>fs.default.name</name>
>>>>    <value>localhost:9000</value>
>>>>  </property>
>>>>
>>>> This is the same in both hadoop-site.xml and the hama-site.xml
>>>>
>>>> The /etc/hosts file looks like this -
>>>>
>>>> 69.90.54.2      ubuntu  # Added by NetworkManager
>>>> 127.0.0.1       localhost.localdomain   localhost
>>>> ::1     ubuntu  localhost6.localdomain6 localhost6
>>>> 127.0.1.1       ubuntu.ubuntu-domain    ubuntu
>>>>
>>>> # The following lines are desirable for IPv6 capable hosts
>>>> ::1     localhost ip6-localhost ip6-loopback
>>>> fe00::0 ip6-localnet
>>>> ff00::0 ip6-mcastprefix
>>>> ff02::1 ip6-allnodes
>>>> ff02::2 ip6-allrouters
>>>> ff02::3 ip6-allhosts
>>>>
>>>> Thanks
>>>> Ashish
>>>>
>>>>
>>>> On Thu, Jun 23, 2011 at 1:40 AM, Edward J. Yoon <edwardyoon@apache.org>
wrote:
>>>>> What is your 'fs.default.name' of hadoop?
>>>>>
>>>>> What is your hostname?
>>>>>
>>>>> Have you seen the /etc/hosts file?
>>>>>
>>>>> On Thu, Jun 23, 2011 at 5:22 PM, Ashish Agarwal <ashish.a77777@gmail.com>
wrote:
>>>>>> I already have made that change, but I still have the same error
>>>>>>
>>>>>> I have tried
>>>>>> <value>localhost:40000</value>
>>>>>> and also
>>>>>> <value>local</value>
>>>>>>
>>>>>>
>>>>>> Thanks
>>>>>> Ashish
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Thu, Jun 23, 2011 at 1:12 AM, Edward J. Yoon <edwardyoon@apache.org>
wrote:
>>>>>>>
>>>>>>> You need to set the 'bsp.master.address' to localhost or your
hostname.
>>>>>>>
>>>>>>> conf/hama-site.xml:
>>>>>>>
>>>>>>>  <property>
>>>>>>>    <name>bsp.master.address</name>
>>>>>>>    <value>localhost:40000</value>
>>>>>>>  </property>
>>>>>>>
>>>>>>> On Thu, Jun 23, 2011 at 5:01 PM, Ashish Agarwal <ashish.a77777@gmail.com>
wrote:
>>>>>>> > Now, I am having a problem starting the bspmaster :(
>>>>>>> > bin/hama bspmaster
>>>>>>> >
>>>>>>> >
>>>>>>> > hadoop@ubuntu:~/Desktop/SecondWorkspace/hama-trunk$ bin/hama
bspmaster
>>>>>>> > 11/06/23 01:00:13 INFO hama.BSPMasterRunner: STARTUP_MSG:
>>>>>>> > /************************************************************
>>>>>>> > STARTUP_MSG: Starting BSPMaster
>>>>>>> > STARTUP_MSG:   host = ubuntu/127.0.1.1
>>>>>>> > STARTUP_MSG:   args = []
>>>>>>> > STARTUP_MSG:   version = 0.20.2
>>>>>>> > STARTUP_MSG:   build =  -r ; compiled by 'chrisdo' on
Wed Mar 10
>>>>>>> > 13:22:54 PST 2010
>>>>>>> > ************************************************************/
>>>>>>> > 11/06/23 01:00:13 INFO bsp.BSPMaster: RPC BSPMaster: host
local port 40000
>>>>>>> > 11/06/23 01:00:14 FATAL hama.BSPMasterRunner:
>>>>>>> > java.net.UnknownHostException: Invalid hostname for server:
local
>>>>>>> >        at org.apache.hadoop.ipc.Server.bind(Server.java:198)
>>>>>>> >        at org.apache.hadoop.ipc.Server$Listener.<init>(Server.java:253)
>>>>>>> >        at org.apache.hadoop.ipc.Server.<init>(Server.java:1026)
>>>>>>> >        at org.apache.hadoop.ipc.RPC$Server.<init>(RPC.java:488)
>>>>>>> >        at org.apache.hadoop.ipc.RPC.getServer(RPC.java:450)
>>>>>>> >        at org.apache.hadoop.ipc.RPC.getServer(RPC.java:441)
>>>>>>> >        at org.apache.hama.bsp.BSPMaster.<init>(BSPMaster.java:248)
>>>>>>> >        at org.apache.hama.bsp.BSPMaster.startMaster(BSPMaster.java:419)
>>>>>>> >        at org.apache.hama.bsp.BSPMaster.startMaster(BSPMaster.java:413)
>>>>>>> >        at org.apache.hama.BSPMasterRunner.run(BSPMasterRunner.java:46)
>>>>>>> >        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
>>>>>>> >        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>>>>>>> >        at org.apache.hama.BSPMasterRunner.main(BSPMasterRunner.java:56)
>>>>>>> >
>>>>>>> > 11/06/23 01:00:14 INFO hama.BSPMasterRunner: SHUTDOWN_MSG:
>>>>>>> > /************************************************************
>>>>>>> > SHUTDOWN_MSG: Shutting down BSPMaster at ubuntu/127.0.1.1
>>>>>>> > ************************************************************/
>>>>>>> > hadoop@ubuntu:~/Desktop/SecondWorkspace/hama-trunk$
>>>>>>> >
>>>>>>> >
>>>>>>> > Thanks
>>>>>>> > Ashish
>>>>>>> >
>>>>>>> > On Thu, Jun 23, 2011 at 12:49 AM, Edward J. Yoon <edwardyoon@apache.org>
wrote:
>>>>>>> >> If you want to execute a command in the background without
watching
>>>>>>> >> logs, you can do the following:
>>>>>>> >>
>>>>>>> >> % bin/hama zookeeper &
>>>>>>> >>
>>>>>>> >> On Thu, Jun 23, 2011 at 4:43 PM, Ashish Agarwal <ashish.a77777@gmail.com>
wrote:
>>>>>>> >>> How long does it take to execute the command
>>>>>>> >>> bin/hama zookeeper
>>>>>>> >>>
>>>>>>> >>> I had to stop the execution as it stopped doing
anything after some time.
>>>>>>> >>>
>>>>>>> >>> hadoop@ubuntu:~/Desktop/SecondWorkspace/hama-trunk$
bin/hama zookeeper
>>>>>>> >>> 11/06/23 00:27:41 INFO server.ZooKeeperServerMain:
Starting server
>>>>>>> >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server
>>>>>>> >>> environment:zookeeper.version=3.3.1-942149, built
on 05/07/2010 17:14
>>>>>>> >>> GMT
>>>>>>> >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server
>>>>>>> >>> environment:host.name=ubuntu.ubuntu-domain
>>>>>>> >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server
>>>>>>> >>> environment:java.version=1.6.0_24
>>>>>>> >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server
>>>>>>> >>> environment:java.vendor=Sun Microsystems Inc.
>>>>>>> >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server
>>>>>>> >>> environment:java.home=/usr/lib/jvm/java-6-sun-1.6.0.24/jre
>>>>>>> >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server
>>>>>>> >>> environment:java.class.path=/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../conf:/usr/lib/jvm/java-6-sun/lib/tools.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../target/classes:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../hama-**.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../lib/commons-cli-1.2.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../lib/commons-logging-1.0.4.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../lib/hadoop-core-0.20.2.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../lib/hadoop-test-0.20.2.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../lib/jetty-6.1H.22.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../lib/jetty-annotations-6.1H.22.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../lib/jetty-util-6.1H.22.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../lib/junit-3.8.1.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../lib/log4j-1.2.15.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../lib/slf4j-log4j12-1.5.2.jar:/home/hadoop/Desktop/SecondWorkspace/hama-trunk/bin/../lib/zookeeper-3.3.1.jar
>>>>>>> >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server
>>>>>>> >>> environment:java.library.path=/usr/lib/jvm/java-6-sun-1.6.0.24/jre/lib/i386/server:/usr/lib/jvm/java-6-sun-1.6.0.24/jre/lib/i386:/usr/lib/jvm/java-6-sun-1.6.0.24/jre/../lib/i386:/usr/java/packages/lib/i386:/lib:/usr/lib
>>>>>>> >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server
>>>>>>> >>> environment:java.io.tmpdir=/tmp
>>>>>>> >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server
>>>>>>> >>> environment:java.compiler=<NA>
>>>>>>> >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server
environment:os.name=Linux
>>>>>>> >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server
environment:os.arch=i386
>>>>>>> >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server
>>>>>>> >>> environment:os.version=2.6.35-28-generic
>>>>>>> >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server
>>>>>>> >>> environment:user.name=hadoop
>>>>>>> >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server
>>>>>>> >>> environment:user.home=/home/hadoop
>>>>>>> >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: Server
>>>>>>> >>> environment:user.dir=/home/hadoop/Desktop/SecondWorkspace/hama-trunk
>>>>>>> >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: tickTime
set to 3000
>>>>>>> >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: minSessionTimeout
set to -1
>>>>>>> >>> 11/06/23 00:27:41 INFO server.ZooKeeperServer: maxSessionTimeout
set to 1200000
>>>>>>> >>> 11/06/23 00:27:41 INFO server.NIOServerCnxn: binding
to port
>>>>>>> >>> 0.0.0.0/0.0.0.0:21810
>>>>>>> >>> 11/06/23 00:27:41 INFO persistence.FileSnap: Reading
snapshot
>>>>>>> >>> /tmp/hama-hadoop/zookeeper/version-2/snapshot.0
>>>>>>> >>> 11/06/23 00:27:41 INFO persistence.FileSnap: Reading
snapshot
>>>>>>> >>> /tmp/hama-hadoop/zookeeper/version-2/snapshot.0
>>>>>>> >>> 11/06/23 00:27:41 INFO persistence.FileTxnSnapLog:
Snapshotting: 0
>>>>>>> >>> ^Chadoop@ubuntu:~/Desktop/SecondWorkspace/hama-trunk$
>>>>>>> >>>
>>>>>>> >>>
>>>>>>> >>> Thanks
>>>>>>> >>> Ashish
>>>>>>> >>>
>>>>>>> >>>
>>>>>>> >>>
>>>>>>> >>> On Wed, Jun 22, 2011 at 3:31 PM, Joseph Boyd
>>>>>>> >>> <joseph.boyd@cbsinteractive.com> wrote:
>>>>>>> >>>>
>>>>>>> >>>> On Wed, Jun 22, 2011 at 3:15 PM, Edward J. Yoon
<edwardyoon@apache.org> wrote:
>>>>>>> >>>> > Instead removing 206 line, why don't we
add dependencies as below?
>>>>>>> >>>>
>>>>>>> >>>> I think removing line 206 is the right thing
to do.  Hama doesn't
>>>>>>> >>>> depend on commons-cli, hadoop-core does, and
maven will properly pick
>>>>>>> >>>> that up and include the jar if allowed.
>>>>>>> >>>>
>>>>>>> >>>> The fact that Hama doesn't depend on the several
items below was
>>>>>>> >>>> correctly detected by maven, in the report description
of HAMA-402.
>>>>>>> >>>>
>>>>>>> >>>> You don't mention a particular problem with
including the transitive
>>>>>>> >>>> dependencies, but if your problem is that it
includes too many items,
>>>>>>> >>>> maven does have options to exclude a particular
transitive dependency
>>>>>>> >>>> on a case-by-case basis that might be more appropriate
to use.
>>>>>>> >>>>
>>>>>>> >>>> Including the dependencies below as you suggest
puts Hama in the place
>>>>>>> >>>> of maintaining a transitive dependency like
commons-cli in lock-step
>>>>>>> >>>> hadoop-core, where the actual dependency comes
from.
>>>>>>> >>>>
>>>>>>> >>>>
>>>>>>> >>>> This is all just my $.02 worth, I'm open to
arguments in favor of
>>>>>>> >>>> <excludeTransitive/>, and there is often
more than one way to skin a
>>>>>>> >>>> cat with respect to dependency management.
>>>>>>> >>>>
>>>>>>> >>>>
>>>>>>> >>>> ...joe
>>>>>>> >>>>
>>>>>>> >>>>
>>>>>>> >>>>
>>>>>>> >>>> >    <dependency>
>>>>>>> >>>> >      <groupId>org.mortbay.jetty</groupId>
>>>>>>> >>>> >      <artifactId>jetty-annotations</artifactId>
>>>>>>> >>>> >      <version>6.1H.22</version>
>>>>>>> >>>> >    </dependency>
>>>>>>> >>>> >    <dependency>
>>>>>>> >>>> >      <groupId>org.slf4j</groupId>
>>>>>>> >>>> >      <artifactId>slf4j-log4j12</artifactId>
>>>>>>> >>>> >      <version>1.5.2</version>
>>>>>>> >>>> >    </dependency>
>>>>>>> >>>> >    <dependency>
>>>>>>> >>>> >      <groupId>commons-cli</groupId>
>>>>>>> >>>> >      <artifactId>commons-cli</artifactId>
>>>>>>> >>>> >      <version>1.2</version>
>>>>>>> >>>> >    </dependency>
>>>>>>> >>>> >
>>>>>>> >>>> > Thanks.
>>>>>>> >>>> >
>>>>>>> >>>> > On Thu, Jun 23, 2011 at 7:00 AM, Edward
J. Yoon <edwardyoon@apache.org> wrote:
>>>>>>> >>>> >> Oh... sorry, you're right.
>>>>>>> >>>> >>
>>>>>>> >>>> >> It looks like a bug. We have to fix
this.
>>>>>>> >>>> >>
>>>>>>> >>>> >> [INFO] Total time: 4:36.098s
>>>>>>> >>>> >> [INFO] Finished at: Thu Jun 23 06:58:28
KST 2011
>>>>>>> >>>> >> [INFO] Final Memory: 20M/51M
>>>>>>> >>>> >> [INFO] ------------------------------------------------------------------------
>>>>>>> >>>> >> edward@test ~/workspace/hama-trunk
$ ls lib/
>>>>>>> >>>> >> commons-logging-1.0.4.jar  hadoop-test-0.20.2.jar
>>>>>>> >>>> >> jetty-util-6.1H.22.jar  log4j-1.2.15.jar
>>>>>>> >>>> >> hadoop-core-0.20.2.jar     jetty-6.1H.22.jar
      junit-3.8.1.jar
>>>>>>> >>>> >>    zookeeper-3.3.1.jar
>>>>>>> >>>> >> edward@tweetple ~/workspace/hama-trunk
$ bin/hama zookeeper
>>>>>>> >>>> >> -bash: bin/hama: Permission denied
>>>>>>> >>>> >> edward@test ~/workspace/hama-trunk
$ chmod 777 bin/**
>>>>>>> >>>> >> edward@test ~/workspace/hama-trunk
$ bin/hama zookeeper
>>>>>>> >>>> >> Exception in thread "main" java.lang.NoClassDefFoundError:
>>>>>>> >>>> >> org/apache/commons/cli/ParseException
>>>>>>> >>>> >>        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:59)
>>>>>>> >>>> >>        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>>>>>>> >>>> >>        at org.apache.hama.ZooKeeperRunner.main(ZooKeeperRunner.java:41)
>>>>>>> >>>> >> Caused by: java.lang.ClassNotFoundException:
>>>>>>> >>>> >> org.apache.commons.cli.ParseException
>>>>>>> >>>> >>        at java.net.URLClassLoader$1.run(URLClassLoader.java:200)
>>>>>>> >>>> >>        at java.security.AccessController.doPrivileged(Native
Method)
>>>>>>> >>>> >>        at java.net.URLClassLoader.findClass(URLClassLoader.java:188)
>>>>>>> >>>> >>        at java.lang.ClassLoader.loadClass(ClassLoader.java:303)
>>>>>>> >>>> >>        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>>>>>>> >>>> >>        at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>>>>>>> >>>> >>        at java.lang.ClassLoader.loadClassInternal(ClassLoader.java:316)
>>>>>>> >>>> >>        ... 3 more
>>>>>>> >>>> >>
>>>>>>> >>>> >>
>>>>>>> >>>> >>
>>>>>>> >>>> >> On Thu, Jun 23, 2011 at 6:45 AM, Edward
J. Yoon <edwardyoon@apache.org> wrote:
>>>>>>> >>>> >>> That's so weird.
>>>>>>> >>>> >>>
>>>>>>> >>>> >>> It works well on my test cluster.
>>>>>>> >>>> >>>
>>>>>>> >>>> >>> edward@slave:~/workspace/hama-trunk$
ls lib/
>>>>>>> >>>> >>> commons-logging-1.0.4.jar  hadoop-test-0.20.2.jar
>>>>>>> >>>> >>> jetty-util-6.1H.22.jar  log4j-1.2.15.jar
>>>>>>> >>>> >>> hadoop-core-0.20.2.jar     jetty-6.1H.22.jar
      junit-3.8.1.jar
>>>>>>> >>>> >>>    zookeeper-3.3.1.jar
>>>>>>> >>>> >>> edward@slave:~/workspace/hama-trunk$
bin/hama jar
>>>>>>> >>>> >>> hama-examples-0.3.0-incubating-SNAPSHOT.jar
pi
>>>>>>> >>>> >>> 11/06/23 06:41:55 DEBUG bsp.BSPJobClient:
BSPJobClient.submitJobDir:
>>>>>>> >>>> >>> hdfs://slave.udanax.org:9000/tmp/hadoop-edward/bsp/system/submit_ebr5do
>>>>>>> >>>> >>> 11/06/23 06:41:57 INFO bsp.BSPJobClient:
Running job: job_201106230640_0001
>>>>>>> >>>> >>> 11/06/23 06:42:00 INFO bsp.BSPJobClient:
Current supersteps number: 0
>>>>>>> >>>> >>> 11/06/23 06:42:03 INFO bsp.BSPJobClient:
Current supersteps number: 1
>>>>>>> >>>> >>> 11/06/23 06:42:03 INFO bsp.BSPJobClient:
The total number of supersteps: 1
>>>>>>> >>>> >>> Estimated value of PI is 3.1534
>>>>>>> >>>> >>> Job Finished in 7.886 seconds
>>>>>>> >>>> >>> edward@slave:~/workspace/hama-trunk$
>>>>>>> >>>> >>>
>>>>>>> >>>> >>> On Thu, Jun 23, 2011 at 4:21 AM,
Joseph Boyd
>>>>>>> >>>> >>> <joseph.boyd@cbsinteractive.com>
wrote:
>>>>>>> >>>> >>>> Replying to my own post ...
>>>>>>> >>>> >>>>
>>>>>>> >>>> >>>> On Wed, Jun 22, 2011 at 11:05
AM, Joseph Boyd
>>>>>>> >>>> >>>> <joseph.boyd@cbsinteractive.com>
wrote:
>>>>>>> >>>> >>>>> On Thu, Jun 16, 2011 at
4:37 AM, Ashish Agarwal <ashish.a77777@gmail.com> wrote:
>>>>>>> >>>> >>>>>>
>>>>>>> >>>> >>>>>> When I try to run the
hama daemons using the command -
>>>>>>> >>>> >>>>>> bin/hama zookeeper
>>>>>>> >>>> >>>>>>
>>>>>>> >>>> >>>>>> I get this error -
>>>>>>> >>>> >>>>
>>>>>>> >>>> >>>> It looks like the problem is
actually this line in pom.xml (line 205)  :
>>>>>>> >>>> >>>>                  
 <excludeTransitive>true</excludeTransitive>
>>>>>>> >>>> >>>>
>>>>>>> >>>> >>>> This prevents commons-cli (and
other transitive dependencies) from
>>>>>>> >>>> >>>> being copied to the lib dir
like they should be.
>>>>>>> >>>> >>>>
>>>>>>> >>>> >>>> Removing line 205 fixes the
NoClassDefFoundError for me.  I can open a
>>>>>>> >>>> >>>> JIRA for this after lunch,
the command line is fairly broken with this
>>>>>>> >>>> >>>> line in, especially after several
dependencies were (correctly)
>>>>>>> >>>> >>>> removed in HAMA-402.
>>>>>>> >>>> >>>>
>>>>>>> >>>> >>>>
>>>>>>> >>>> >>>> ...joe
>>>>>>> >>>> >>>>
>>>>>>> >>>> >>>>
>>>>>>> >>>> >>>>> I'm getting the same NoClassDefFoundError
using the svn code.  The
>>>>>>> >>>> >>>>> commons-cli dependency
was removed recently as part of HAMA-402, and
>>>>>>> >>>> >>>>> re-adding it fixes the
NoClassDefFoundError.
>>>>>>> >>>> >>>>>
>>>>>>> >>>> >>>>> I believe commons-cli should
have been pulled as part of the
>>>>>>> >>>> >>>>> hadoop-core dependency,
and wasn't.  (though my maven skills are
>>>>>>> >>>> >>>>> rusty, and I'm no longer
sure just how transitive dependencies are
>>>>>>> >>>> >>>>> pulled)
>>>>>>> >>>> >>>>>
>>>>>>> >>>> >>>>> While a quick work-around
is to re-add the dependency here, the proper
>>>>>>> >>>> >>>>> fix is probably to fix
the hadoop-core pom, and properly mark that it
>>>>>>> >>>> >>>>> depends on commons-cli.
>>>>>>> >>>> >>>>>
>>>>>>> >>>> >>>>>
>>>>>>> >>>> >>>>> ...joe
>>>>>>> >>>> >>>>>
>>>>>>> >>>> >>>>>
>>>>>>> >>>> >>>>>> Exception in thread
"main" java.lang.NoClassDefFoundError:
>>>>>>> >>>> >>>>>> org/apache/commons/cli/ParseException
>>>>>>> >>>> >>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:59)
>>>>>>> >>>> >>>>>> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>>>>>>> >>>> >>>>>> at org.apache.hama.ZooKeeperRunner.main(ZooKeeperRunner.java:41)
>>>>>>> >>>> >>>>>> Caused by: java.lang.ClassNotFoundException:
>>>>>>> >>>> >>>>>> org.apache.commons.cli.ParseException
>>>>>>> >>>> >>>>>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>>>>>>> >>>> >>>>>> at java.security.AccessController.doPrivileged(Native
Method)
>>>>>>> >>>> >>>>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>>>>>>> >>>> >>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
>>>>>>> >>>> >>>>>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>>>>>>> >>>> >>>>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>>>>>>> >>>> >>>>>> ... 3 more
>>>>>>> >>>> >>>>>>
>>>>>>> >>>> >>>>>>
>>>>>>> >>>> >>>>>> I tried installing
it a few times, but can't think of a solution to the
>>>>>>> >>>> >>>>>> problem. Can you please
help ?
>>>>>>> >>>> >>>>>>
>>>>>>> >>>> >>>>>> Thank you
>>>>>>> >>>> >>>>>>
>>>>>>> >>>> >>>>>> Sincerely,
>>>>>>> >>>> >>>>>> Ashish
>>>>>>> >>>> >>>>>>
>>>>>>> >>>> >>>>>>
>>>>>>> >>>> >>>>>> On Wed, Jun 15, 2011
at 3:18 AM, Edward J. Yoon <edwardyoon@apache.org>wrote:
>>>>>>> >>>> >>>>>>
>>>>>>> >>>> >>>>>>> Hi,
>>>>>>> >>>> >>>>>>>
>>>>>>> >>>> >>>>>>> I would suggest
following steps to debug:
>>>>>>> >>>> >>>>>>>
>>>>>>> >>>> >>>>>>> 1. Install the
Hadoop 0.20.x on your laptop.
>>>>>>> >>>> >>>>>>> 2. Run the Hama
deamons with 3 terminals.
>>>>>>> >>>> >>>>>>>
>>>>>>> >>>> >>>>>>> % bin/hama zookeeper
>>>>>>> >>>> >>>>>>> % bin/hama bspmaster
>>>>>>> >>>> >>>>>>> % bin/hama groom
>>>>>>> >>>> >>>>>>>
>>>>>>> >>>> >>>>>>> 3. Use the logging
API to log debug.
>>>>>>> >>>> >>>>>>> For example, LOG.debug("like
this");
>>>>>>> >>>> >>>>>>>
>>>>>>> >>>> >>>>>>> On Wed, Jun 15,
2011 at 6:48 PM, Ashish Agarwal <ashish.a77777@gmail.com>
>>>>>>> >>>> >>>>>>> wrote:
>>>>>>> >>>> >>>>>>> > Hi,
>>>>>>> >>>> >>>>>>> >
>>>>>>> >>>> >>>>>>> > I was wondering
if anyone can give me instructions on how to debug code
>>>>>>> >>>> >>>>>>> in
>>>>>>> >>>> >>>>>>> > distributed
or pseudo-distributed mode ? It would really help me.
>>>>>>> >>>> >>>>>>> >
>>>>>>> >>>> >>>>>>> > Thank you
>>>>>>> >>>> >>>>>>> >
>>>>>>> >>>> >>>>>>> > Sincerely,
>>>>>>> >>>> >>>>>>> > Ashish
>>>>>>> >>>> >>>>>>> >
>>>>>>> >>>> >>>>>>> >
>>>>>>> >>>> >>>>>>> >
>>>>>>> >>>> >>>>>>> > On Sat, Jun
11, 2011 at 3:41 AM, Ashish Agarwal <ashish.a77777@gmail.com
>>>>>>> >>>> >>>>>>> >wrote:
>>>>>>> >>>> >>>>>>> >
>>>>>>> >>>> >>>>>>> >>
>>>>>>> >>>> >>>>>>> >> Hi,
>>>>>>> >>>> >>>>>>> >>
>>>>>>> >>>> >>>>>>> >> I have
written some code for my GSoC task (Hama-367) and am now trying
>>>>>>> >>>> >>>>>>> to
>>>>>>> >>>> >>>>>>> >> debug.
>>>>>>> >>>> >>>>>>> >>
>>>>>>> >>>> >>>>>>> >> I am trying
to set up the environment into pseudo-distributed mode so I
>>>>>>> >>>> >>>>>>> can
>>>>>>> >>>> >>>>>>> >> debug.
I think the value of bsp.master.address in hama-site.xml has to
>>>>>>> >>>> >>>>>>> be
>>>>>>> >>>> >>>>>>> >> local
as this means LocalBSPRunner (as localhost would call 127.0.0.1
>>>>>>> >>>> >>>>>>> >> server),
but where do I specify that in the code in eclipse ?
>>>>>>> >>>> >>>>>>> >>
>>>>>>> >>>> >>>>>>> >> My next
question is - what is the difference between MiniBSPCluster and
>>>>>>> >>>> >>>>>>> >> LocalBSPRunner
 and which file should I be using to be able to step
>>>>>>> >>>> >>>>>>> through
>>>>>>> >>>> >>>>>>> >> the code
?
>>>>>>> >>>> >>>>>>> >>
>>>>>>> >>>> >>>>>>> >> These
are the processes currently running on my machine -
>>>>>>> >>>> >>>>>>> >>
>>>>>>> >>>> >>>>>>> >> 6112 NameNode
>>>>>>> >>>> >>>>>>> >> 3319 ZooKeeperRunner
>>>>>>> >>>> >>>>>>> >> 3018
>>>>>>> >>>> >>>>>>> >> 6449 SecondaryNameNode
>>>>>>> >>>> >>>>>>> >> 5709 HMaster
>>>>>>> >>>> >>>>>>> >> 5643 HQuorumPeer
>>>>>>> >>>> >>>>>>> >> 6684 TaskTracker
>>>>>>> >>>> >>>>>>> >> 6526 JobTracker
>>>>>>> >>>> >>>>>>> >> 8074 Jps
>>>>>>> >>>> >>>>>>> >> 6277 DataNode
>>>>>>> >>>> >>>>>>> >>
>>>>>>> >>>> >>>>>>> >> Thank
you,
>>>>>>> >>>> >>>>>>> >> Ashish
>>>>>>> >>>> >>>>>>> >>
>>>>>>> >>>> >>>>>>> >
>>>>>>> >>>> >>>>>>>
>>>>>>> >>>> >>>>>>>
>>>>>>> >>>> >>>>>>>
>>>>>>> >>>> >>>>>>> --
>>>>>>> >>>> >>>>>>> Best Regards, Edward
J. Yoon
>>>>>>> >>>> >>>>>>> @eddieyoon
>>>>>>> >>>> >>>>>>>
>>>>>>> >>>> >>>>>>
>>>>>>> >>>> >>>>>
>>>>>>> >>>> >>>>
>>>>>>> >>>> >>>
>>>>>>> >>>> >>>
>>>>>>> >>>> >>>
>>>>>>> >>>> >>> --
>>>>>>> >>>> >>> Best Regards, Edward J. Yoon
>>>>>>> >>>> >>> @eddieyoon
>>>>>>> >>>> >>>
>>>>>>> >>>> >>
>>>>>>> >>>> >>
>>>>>>> >>>> >>
>>>>>>> >>>> >> --
>>>>>>> >>>> >> Best Regards, Edward J. Yoon
>>>>>>> >>>> >> @eddieyoon
>>>>>>> >>>> >>
>>>>>>> >>>> >
>>>>>>> >>>> >
>>>>>>> >>>> >
>>>>>>> >>>> > --
>>>>>>> >>>> > Best Regards, Edward J. Yoon
>>>>>>> >>>> > @eddieyoon
>>>>>>> >>>> >
>>>>>>> >>>
>>>>>>> >>
>>>>>>> >>
>>>>>>> >>
>>>>>>> >> --
>>>>>>> >> Best Regards, Edward J. Yoon
>>>>>>> >> @eddieyoon
>>>>>>> >>
>>>>>>> >
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Best Regards, Edward J. Yoon
>>>>>>> @eddieyoon
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Best Regards, Edward J. Yoon
>>>>> @eddieyoon
>>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Best Regards, Edward J. Yoon
>>> @eddieyoon
>>>
>>
>
>
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon
>

Mime
View raw message