hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward J. Yoon" <edwardy...@apache.org>
Subject Re: Eclipse test failure for HAMA-376
Date Fri, 15 Apr 2011 06:49:52 GMT
I think, many things should be fixed.

1. Basically the value of "bsp.master.address" is setted to
"localhost" In hama-default.xml:

    <name>bsp.master.address</name>
    <value>localhost</value>
    <description>The address of the bsp master server. Either the
    literal string "local" or a host[:port] (where host is a name or

If we add a LocalBSPJobRunner to Hama 0.3, I think, we should use
"local" as a default value. ("local" ≠ "localhost"). Description
strings should be fixed, too.

2. In eclipse, conf.get("bsp.master.address") returns null.

    String mode = conf.get("bsp.master.address");
    LOG.info(">>>>>> " + mode);

It means that the config files are not loaded. If we run with advanced
configurations in eclipse, it'll be work but, it's really annoying.

I think, we can set the basic configurations for JUnit tests by
hard-coding in MiniClusterTest constructor.

On Thu, Apr 14, 2011 at 8:02 PM, chl501 <chl501@nuk.edu.tw> wrote:
> My question should be `Can't we simply use the default value, which sets bsp.master.address
to localhost, in hama-default.xml'?
>
> Currently when using the default value (localhost), the bspMasterAddr will has the value
referenced to master address/port.
>
> Resetting bsp.master.address to local will cause bspMasterAddr to null.
>
> Why do we need to reset bsp.master.address to local?
>
> Sorry for my poor English.
>
> -----Original message-----
> From:chl501 <chl501@nuk.edu.tw>
> To:hama-dev <hama-dev@incubator.apache.org>
> Date:Thu, 14 Apr 2011 17:45:38 +0800 (CST)
> Subject:Re: Eclipse test failure for HAMA-376
>
> Looks like the problem is because BSPMaster.getAddress(conf) in GroomServer is trying
to create socket again after BSPMaster successfully occupies the ip/port (GroomServers need
to wait until BSPMaster is up).
>
> in the end of NetUtil.createSocketAddr() it shows
>
> public static InetSocketAddress createSocketAddr(String target,
>                                                   int defaultPort)
{
>    ...
>    return new InetSocketAddress(hostname, port);
> }
>
> Should we change not to use bspMasterAddr because GroomServer can directly obtain bsp.master.address
(ip/port) from conf/hama-site.xml?
>
> -----Original message-----
> From:Edward J. Yoon <edwardyoon@apache.org>
> To:hama-dev@incubator.apache.org,chl501@nuk.edu.tw
> Date:Thu, 14 Apr 2011 17:32:01 +0900
> Subject:Re: Eclipse test failure for HAMA-376
>
> Nope, "Class not found error" and "setJar()" problems are unconnected
> things from my report.
>
> I think, the point of miniBSPCluster problem with eclipse and maven
> build, is that the value of "bsp.master.address" is always null:
>
>>11/04/12 20:49:50 INFO bsp.GroomServer: groom start
>>>11/04/12 20:49:51 INFO hama.MiniBSPCluster: Waitin for GroomServer up.
>>>11/04/12 20:49:52 INFO hama.MiniBSPCluster: Waitin for GroomServer up.
>>>11/04/12 20:49:53 INFO hama.MiniBSPCluster: Waitin for GroomServer up.
>>>11/04/12 20:49:54 INFO hama.MiniBSPCluster: Waitin for GroomServer up.
>>>11/04/12 20:49:55 INFO hama.MiniBSPCluster: Waitin for GroomServer up.
>
> Try to set the value of "mode" to "local" as below:
>
>  public GroomServer(Configuration conf) throws IOException {
>    ....
>    mode = "local";
>    ....
>
> 2011/4/14 chl501 <chl501@nuk.edu.tw>:
>> It is the same issue that the jar file is missing. If it is required to ask use explicitly
setJar, is there any place that we can reference to the build version? For instance, the BSPJob.setJar()
requires to set jar path e.g. /path/to/eclipse/hama-1090935/build/hama-0.3.0-test.jar where
build version 0.3.0 is changed according to ant's property ${version}. Otherwise, we may need
to parse ant file for explicitly adding this info.
>>
>> 11/04/14 14:50:13 WARN bsp.BSPJobClient: No job jar file set.  User classes may
not be found. See BSPJob#setJar(String) or check Your jar file.
>> 11/04/14 14:50:13 INFO bsp.GroomServer: jobConf.getJar():null
>> 11/04/14 14:50:13 WARN bsp.GroomServer: Error initializing attempt_201104141450_0001_000001_0:
>> java.lang.IllegalArgumentException: Can not create a Path from a null string
>>        at org.apache.hadoop.fs.Path.checkPathArg(Path.java:78)
>>        at org.apache.hadoop.fs.Path.<init>(Path.java:90)
>>        at org.apache.hama.bsp.GroomServer.localizeJob(GroomServer.java:519)
>>        at org.apache.hama.bsp.GroomServer.startNewTask(GroomServer.java:451)
>>        at org.apache.hama.bsp.GroomServer.access$100(GroomServer.java:70)
>>        at org.apache.hama.bsp.GroomServer$DispatchTasksHandler.handle(GroomServer.java:144)
>>        at org.apache.hama.bsp.GroomServer$Instructor.run(GroomServer.java:192)
>>
>>
>>>Basically, your miniBSPCluster doesn't work with eclipse JUnit test and maven
build as below:
>>
>>>11/04/12 20:49:49 INFO bsp.BSPMaster: Cleaning up the system directory
>>>11/04/12 20:49:49 INFO bsp.BSPMaster: file:/tmp/hadoop/bsp/system
>>>11/04/12 20:49:49 INFO bsp.BSPMaster: java.lang.NullPointerException
>>>11/04/12 20:49:49 INFO ipc.Server: IPC Server Responder: starting
>>>11/04/12 20:49:49 INFO ipc.Server: IPC Server listener on 40000: starting
>>>11/04/12 20:49:49 INFO ipc.Server: IPC Server handler 0 on 40000: starting
>>>11/04/12 20:49:49 INFO bsp.BSPMaster: Starting RUNNING
>>>11/04/12 20:49:50 INFO hama.MiniBSPCluster: Waitin for GroomServer up.
>>>11/04/12 20:49:50 INFO bsp.GroomServer: groom start
>>>11/04/12 20:49:51 INFO hama.MiniBSPCluster: Waitin for GroomServer up.
>>>11/04/12 20:49:52 INFO hama.MiniBSPCluster: Waitin for GroomServer up.
>>>11/04/12 20:49:53 INFO hama.MiniBSPCluster: Waitin for GroomServer up.
>>>11/04/12 20:49:54 INFO hama.MiniBSPCluster: Waitin for GroomServer up.
>>>11/04/12 20:49:55 INFO hama.MiniBSPCluster: Waitin for GroomServer up.
>>>11/04/12 20:49:56 INFO hama.MiniBSPCluster: Waitin for GroomServer up.
>>>^CDestroying 1 processes
>>>Destroying process..
>>>Destroyed 1 processes
>>>Can you find out the reason why?
>>
>>
>>
>>
>>
>> ChiaHung Lin
>> Department of Information Management
>> National University of Kaohsiung
>> Taiwan
>>
>
>
>
> --
> Best Regards, Edward J. Yoon
> http://blog.udanax.org
> http://twitter.com/eddieyoon
>
>
> ChiaHung Lin
> Department of Information Management
> National University of Kaohsiung
> Taiwan
>
>
> ChiaHung Lin
> Department of Information Management
> National University of Kaohsiung
> Taiwan
>



-- 
Best Regards, Edward J. Yoon
http://blog.udanax.org
http://twitter.com/eddieyoon

Mime
View raw message