Mailing-List: contact hama-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: hama-dev@incubator.apache.org
MIME-Version: 1.0
In-Reply-To: <1302778971.21992.chl501@nuk.edu.tw>
References: <1302767283.3491.chl501@nuk.edu.tw>
	<BANLkTi=8o8AmLzWS0fg_4VmmmgmN6MK=Pw@mail.gmail.com>
	<1302774338.17293.chl501@nuk.edu.tw>
	<1302778971.21992.chl501@nuk.edu.tw>
Date: Fri, 15 Apr 2011 15:49:52 +0900
Message-ID: <BANLkTimH0RPJbaONKRtAuisM3S_2_naSGA@mail.gmail.com>
Subject: Re: Eclipse test failure for HAMA-376
From: "Edward J. Yoon" <edwardyoon@apache.org>
To: hama-dev@incubator.apache.org, chl501@nuk.edu.tw
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

I think, many things should be fixed.

1. Basically the value of "bsp.master.address" is setted to
"localhost" In hama-default.xml:

    <name>bsp.master.address</name>
    <value>localhost</value>
    <description>The address of the bsp master server. Either the
    literal string "local" or a host[:port] (where host is a name or

If we add a LocalBSPJobRunner to Hama 0.3, I think, we should use
"local" as a default value. ("local" =E2=89=A0 "localhost"). Description
strings should be fixed, too.

2. In eclipse, conf.get("bsp.master.address") returns null.

    String mode =3D conf.get("bsp.master.address");
    LOG.info(">>>>>> " + mode);

It means that the config files are not loaded. If we run with advanced
configurations in eclipse, it'll be work but, it's really annoying.

I think, we can set the basic configurations for JUnit tests by
hard-coding in MiniClusterTest constructor.

On Thu, Apr 14, 2011 at 8:02 PM, chl501 <chl501@nuk.edu.tw> wrote:
> My question should be `Can't we simply use the default value, which sets =
bsp.master.address to localhost, in hama-default.xml'?
>
> Currently when using the default value (localhost), the bspMasterAddr wil=
l has the value referenced to master address/port.
>
> Resetting bsp.master.address to local will cause bspMasterAddr to null.
>
> Why do we need to reset bsp.master.address to local?
>
> Sorry for my poor English.
>
> -----Original message-----
> From:chl501 <chl501@nuk.edu.tw>
> To:hama-dev <hama-dev@incubator.apache.org>
> Date:Thu, 14 Apr 2011 17:45:38 +0800 (CST)
> Subject:Re: Eclipse test failure for HAMA-376
>
> Looks like the problem is because BSPMaster.getAddress(conf) in GroomServ=
er is trying to create socket again after BSPMaster successfully occupies t=
he ip/port (GroomServers need to wait until BSPMaster is up).
>
> in the end of NetUtil.createSocketAddr() it shows
>
> public static InetSocketAddress createSocketAddr(String target,
> =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=
=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =C2=A0 =
=C2=A0 =C2=A0 =C2=A0 =C2=A0 int defaultPort) {
> =C2=A0 =C2=A0...
> =C2=A0 =C2=A0return new InetSocketAddress(hostname, port);
> }
>
> Should we change not to use bspMasterAddr because GroomServer can directl=
y obtain bsp.master.address (ip/port) from conf/hama-site.xml?
>
> -----Original message-----
> From:Edward J. Yoon <edwardyoon@apache.org>
> To:hama-dev@incubator.apache.org,chl501@nuk.edu.tw
> Date:Thu, 14 Apr 2011 17:32:01 +0900
> Subject:Re: Eclipse test failure for HAMA-376
>
> Nope, "Class not found error" and "setJar()" problems are unconnected
> things from my report.
>
> I think, the point of miniBSPCluster problem with eclipse and maven
> build, is that the value of "bsp.master.address" is always null:
>
>>11/04/12 20:49:50 INFO bsp.GroomServer: groom start
>>>11/04/12 20:49:51 INFO hama.MiniBSPCluster: Waitin for GroomServer up.
>>>11/04/12 20:49:52 INFO hama.MiniBSPCluster: Waitin for GroomServer up.
>>>11/04/12 20:49:53 INFO hama.MiniBSPCluster: Waitin for GroomServer up.
>>>11/04/12 20:49:54 INFO hama.MiniBSPCluster: Waitin for GroomServer up.
>>>11/04/12 20:49:55 INFO hama.MiniBSPCluster: Waitin for GroomServer up.
>
> Try to set the value of "mode" to "local" as below:
>
> =C2=A0public GroomServer(Configuration conf) throws IOException {
> =C2=A0 =C2=A0....
> =C2=A0 =C2=A0mode =3D "local";
> =C2=A0 =C2=A0....
>
> 2011/4/14 chl501 <chl501@nuk.edu.tw>:
>> It is the same issue that the jar file is missing. If it is required to =
ask use explicitly setJar, is there any place that we can reference to the =
build version? For instance, the BSPJob.setJar() requires to set jar path e=
.g. /path/to/eclipse/hama-1090935/build/hama-0.3.0-test.jar where build ver=
sion 0.3.0 is changed according to ant's property ${version}. Otherwise, we=
 may need to parse ant file for explicitly adding this info.
>>
>> 11/04/14 14:50:13 WARN bsp.BSPJobClient: No job jar file set. =C2=A0User=
 classes may not be found. See BSPJob#setJar(String) or check Your jar file=
.
>> 11/04/14 14:50:13 INFO bsp.GroomServer: jobConf.getJar():null
>> 11/04/14 14:50:13 WARN bsp.GroomServer: Error initializing attempt_20110=
4141450_0001_000001_0:
>> java.lang.IllegalArgumentException: Can not create a Path from a null st=
ring
>> =C2=A0 =C2=A0 =C2=A0 =C2=A0at org.apache.hadoop.fs.Path.checkPathArg(Pat=
h.java:78)
>> =C2=A0 =C2=A0 =C2=A0 =C2=A0at org.apache.hadoop.fs.Path.<init>(Path.java=
:90)
>> =C2=A0 =C2=A0 =C2=A0 =C2=A0at org.apache.hama.bsp.GroomServer.localizeJo=
b(GroomServer.java:519)
>> =C2=A0 =C2=A0 =C2=A0 =C2=A0at org.apache.hama.bsp.GroomServer.startNewTa=
sk(GroomServer.java:451)
>> =C2=A0 =C2=A0 =C2=A0 =C2=A0at org.apache.hama.bsp.GroomServer.access$100=
(GroomServer.java:70)
>> =C2=A0 =C2=A0 =C2=A0 =C2=A0at org.apache.hama.bsp.GroomServer$DispatchTa=
sksHandler.handle(GroomServer.java:144)
>> =C2=A0 =C2=A0 =C2=A0 =C2=A0at org.apache.hama.bsp.GroomServer$Instructor=
.run(GroomServer.java:192)
>>
>>
>>>Basically, your miniBSPCluster doesn't work with eclipse JUnit test and =
maven build as below:
>>
>>>11/04/12 20:49:49 INFO bsp.BSPMaster: Cleaning up the system directory
>>>11/04/12 20:49:49 INFO bsp.BSPMaster: file:/tmp/hadoop/bsp/system
>>>11/04/12 20:49:49 INFO bsp.BSPMaster: java.lang.NullPointerException
>>>11/04/12 20:49:49 INFO ipc.Server: IPC Server Responder: starting
>>>11/04/12 20:49:49 INFO ipc.Server: IPC Server listener on 40000: startin=
g
>>>11/04/12 20:49:49 INFO ipc.Server: IPC Server handler 0 on 40000: starti=
ng
>>>11/04/12 20:49:49 INFO bsp.BSPMaster: Starting RUNNING
>>>11/04/12 20:49:50 INFO hama.MiniBSPCluster: Waitin for GroomServer up.
>>>11/04/12 20:49:50 INFO bsp.GroomServer: groom start
>>>11/04/12 20:49:51 INFO hama.MiniBSPCluster: Waitin for GroomServer up.
>>>11/04/12 20:49:52 INFO hama.MiniBSPCluster: Waitin for GroomServer up.
>>>11/04/12 20:49:53 INFO hama.MiniBSPCluster: Waitin for GroomServer up.
>>>11/04/12 20:49:54 INFO hama.MiniBSPCluster: Waitin for GroomServer up.
>>>11/04/12 20:49:55 INFO hama.MiniBSPCluster: Waitin for GroomServer up.
>>>11/04/12 20:49:56 INFO hama.MiniBSPCluster: Waitin for GroomServer up.
>>>^CDestroying 1 processes
>>>Destroying process..
>>>Destroyed 1 processes
>>>Can you find out the reason why?
>>
>>
>>
>>
>>
>> ChiaHung Lin
>> Department of Information Management
>> National University of Kaohsiung
>> Taiwan
>>
>
>
>
> --
> Best Regards, Edward J. Yoon
> http://blog.udanax.org
> http://twitter.com/eddieyoon
>
>
> ChiaHung Lin
> Department of Information Management
> National University of Kaohsiung
> Taiwan
>
>
> ChiaHung Lin
> Department of Information Management
> National University of Kaohsiung
> Taiwan
>


--=20
Best Regards, Edward J. Yoon
http://blog.udanax.org
http://twitter.com/eddieyoon