hama-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "步青云" <mailliup...@qq.com>
Subject Re:RE: RE: Is This a Bug of Hama?
Date Fri, 29 Apr 2016 07:51:30 GMT
I had set "hama.graph.self.ref" to true as follow:
public static void main(String[] args){
       ......
       HamaConfiguration conf = new HamaConfiguration();
       conf.getBoolean("hama.graph.self.ref", true);
       GraphJob pageJob = createJob(args, opts, conf);
       ......
}


 public static GraphJob createJob(String[] args, HamaConfiguration conf,
	      Options opts) throws IOException, ParseException {

       ......
       pageJob.setBoolean("hama.graph.self.ref", true);
       ......
}


But I still get the error. While I restart hama and then run the job again, sometime the job
would run successfully. 
If there is any fault with my configuration, please tell me. 


By the way, my input graph is synthetic with 1 million nodes and 8 million edges generated
by Hama.
And I have debug my program, the error occured after the first time sendMessageToNeighbor.
So I also doubt that maybe there is error in my input graph. I'll try to solve it.
Thanks a lot.




       




------------------ Original ------------------
From:  "Edward J. Yoon";<edward.yoon@samsung.com>;
Date:  Apr 28, 2016
To:  "user"<user@hama.apache.org>; 

Subject:  RE: RE: Is This a Bug of Hama?



I doubt you have an orphaned or dangling links. For example, vertex a has 
edges b, c, d. The vertex b and c are exist in your input graph but c doesn't. 
Then, the framework will try to find and compute c vertex with received 
messages but it's null.

Please try to set "hama.graph.self.ref" to true in your job configuration.

conf.getBoolean("hama.graph.self.ref", true);

--
Best Regards, Edward J. Yoon

-----Original Message-----
From: 步?云 [mailto:mailliuping@qq.com]
Sent: Thursday, April 28, 2016 12:15 PM
To: user
Subject: Re:RE: Is This a Bug of Hama?

Hi,
    Thank you for your reply. TriTextPair is wirtable and the code is as 
follow.
    I don't know whether there is problem in TriTextPair. I re-write TextPair 
in hama examples to TriTextPair.
    I'm really confused wht sometime the job run successfully and other time 
failed. I need help.
   Thanks a lot.


public final class TriTextPair implements Writable {
	Text first;
	Text second;
	Text third;


	public TriTextPair() {
		first = new Text("");
		second = new Text("");
		third = new Text("");
		fourth = new Text("");
	}

	public TriTextPair(Text first, Text second, Text third, Text fourth) {
		this.first = first;
		this.second = second;
		this.third = third;
		this.fourth = fourth;
	}

	public Text getFirst() {
		return first;
	}


	public void setFirst(Text first) {
		this.first = first;
	}


	public Text getSecond() {
		return second;
	}


	public void setSecond(Text second) {
		this.second = second;
	}


	public Text getThird() {
		return third;
	}


	public void setThird(Text third) {
		this.third = third;
	}


	@Override
	public void write(DataOutput out) throws IOException {
		first.write(out);
		second.write(out);
		third.write(out);
	}


	@Override
	public void readFields(DataInput in) throws IOException {


		first.readFields(in);
		second.readFields(in);
		third.readFields(in);
	}}



------------------ Original ------------------
From:  "Edward J. Yoon";<edward.yoon@samsung.com>;
Date:  Apr 28, 2016
To:  "user"<user@hama.apache.org>;

Subject:  RE: Is This a Bug of Hama?



Hello,

Please check whether TriTextPair is writable. Please attach that code too if
possible.

--
Best Regards, Edward J. Yoon


-----Original Message-----
From: 꼍행暾 [mailto:mailliuping@qq.com]
Sent: Wednesday, April 27, 2016 8:11 PM
To: user
Subject: Is This a Bug of Hama?

Hello,
     I'm using hama-0.7.0 to do my experiment. While I had some problems.
For the same program and the same inputs, sometimes I could run the program
successfully. But sometimes some problems occured, and then the job failed.
     I would show my code before error message. My code is as follow:


public static class ProbMatchVertex extends Vertex<Text, NullWritable,
TriTextPair> {
		private String parents = new String("");

		@Override
		public void compute(Iterable<TriTextPair> messages) throws
IOException {

			if (getSuperstepCount() == 0) {
				sendMessageToNeighbors(new
TriTextPair(getVertexID(), getVertexLabel(), new Text(""), new Text(""), new
Text()));
			} else if (getSuperstepCount() == 1) {
				for (TriTextPair msg : messages) {
					parents += msg.toString() + "|";
					sendMessage(msg.getFirst(), new
TriTextPair(getVertexID(), getVertexLabel(), new Text(""), new Text(""), new
Text()));
				}
			} else if (getSuperstepCount() == 2) {
				List<TriTextPair> parentList =
str2TriList(parents);
				for (int i=0;i<parentList.size();i++) {

sendMessage(parentList.get(i).getFirst(),
							new
TriTextPair(getVertexID(), new Text("frC"), new Text(matchMapStr), new
Text(loseMapStr), getVertexID()));
				}
			} else if (getSuperstepCount() > 2) {
				......
			}
		}



                 @Override
		public void writeState(DataOutput out) throws IOException {
		    out.writeUTF(parents);
		  }

		@Override
		  public void readState(DataInput in) throws IOException {
		    parents = in.readUTF();
		  }
}
         And the error message is as follow:
INFO message.HamaMessageManagerImpl: BSPPeer address:slave2.novalocal
port:61001
attempt_201604270849_0010_000014_0: 16/04/27 10:53:52 INFO
sync.ZKSyncClient: Initializing ZK Sync Client
attempt_201604270849_0010_000014_0: 16/04/27 10:53:52 INFO
sync.ZooKeeperSyncClientImpl: Start connecting to Zookeeper! At
slave2.novalocal/10.0.2.98:61001
attempt_201604270849_0010_000014_0: 16/04/27 10:53:52 INFO
sync.ZooKeeperSyncClientImpl: TaskAttemptID :
attempt_201604270849_0010_000014_0
attempt_201604270849_0010_000014_0: 16/04/27 10:54:13 INFO
graph.GraphJobRunner: 55884 vertices are loaded into slave2.novalocal:61001
attempt_201604270849_0010_000014_0: 16/04/27 10:54:13 INFO
graph.GraphJobRunner: Total time spent for loading vertices: 8860 ms
attempt_201604270849_0010_000014_0: 16/04/27 10:54:14 INFO
graph.GraphJobRunner: Total time spent for broadcasting global vertex count:
349 ms
attempt_201604270849_0010_000014_0: 16/04/27 10:54:21 INFO
graph.GraphJobRunner: Total time spent for initial superstep: 6937 ms
attempt_201604270849_0010_000014_0: 16/04/27 10:54:33 INFO
graph.GraphJobRunner: Total time spent for broadcasting aggregation values:
276 ms
attempt_201604270849_0010_000014_0: 16/04/27 10:54:33 ERROR bsp.BSPTask:
Error running bsp setup and bsp function.
attempt_201604270849_0010_000014_0: java.lang.NullPointerException
attempt_201604270849_0010_000014_0:     at
org.apache.hama.util.UnsafeByteArrayInputStream.<init>(UnsafeByteArrayInputS
tream.java:63)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.util.WritableUtils.unsafeDeserialize(WritableUtils.java:63)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.graph.MapVerticesInfo.get(MapVerticesInfo.java:101)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.graph.GraphJobRunner$ComputeRunnable.<init>(GraphJobRunner.j
ava:359)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.graph.GraphJobRunner.doSuperstep(GraphJobRunner.java:270)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:182)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:171)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.bsp.BSPTask.run(BSPTask.java:144)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1255)
attempt_201604270849_0010_000014_0: 16/04/27 10:54:34 INFO ipc.Server:
Stopping server on 61001
attempt_201604270849_0010_000014_0: 16/04/27 10:54:34 INFO ipc.Server: IPC
Server handler 0 on 61001: exiting
attempt_201604270849_0010_000014_0: 16/04/27 10:54:34 INFO ipc.Server: IPC
Server handler 1 on 61001: exiting
attempt_201604270849_0010_000014_0: 16/04/27 10:54:34 INFO ipc.Server:
Stopping IPC Server listener on 61001
attempt_201604270849_0010_000014_0: 16/04/27 10:54:34 INFO ipc.Server: IPC
Server handler 4 on 61001: exiting
attempt_201604270849_0010_000014_0: 16/04/27 10:54:34 INFO ipc.Server: IPC
Server handler 3 on 61001: exiting
attempt_201604270849_0010_000014_0: 16/04/27 10:54:34 INFO
Configuration.deprecation: mapred.cache.localFiles is deprecated. Instead,
use mapreduce.job.cache.local.files
attempt_201604270849_0010_000014_0: 16/04/27 10:54:34 INFO ipc.Server: IPC
Server handler 2 on 61001: exiting
attempt_201604270849_0010_000014_0: 16/04/27 10:54:34 ERROR bsp.BSPTask:
Shutting down ping service.
attempt_201604270849_0010_000014_0: 16/04/27 10:54:34 INFO ipc.Server:
Stopping IPC Server Responder
attempt_201604270849_0010_000014_0: 16/04/27 10:54:34 FATAL bsp.GroomServer:
Error running child
attempt_201604270849_0010_000014_0: java.lang.NullPointerException
attempt_201604270849_0010_000014_0:     at
org.apache.hama.util.UnsafeByteArrayInputStream.<init>(UnsafeByteArrayInputS
tream.java:63)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.util.WritableUtils.unsafeDeserialize(WritableUtils.java:63)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.graph.MapVerticesInfo.get(MapVerticesInfo.java:101)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.graph.GraphJobRunner$ComputeRunnable.<init>(GraphJobRunner.j
ava:359)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.graph.GraphJobRunner.doSuperstep(GraphJobRunner.java:270)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:182)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:171)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.bsp.BSPTask.run(BSPTask.java:144)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1255)
attempt_201604270849_0010_000014_0: java.lang.NullPointerException
attempt_201604270849_0010_000014_0:     at
org.apache.hama.util.UnsafeByteArrayInputStream.<init>(UnsafeByteArrayInputS
tream.java:63)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.util.WritableUtils.unsafeDeserialize(WritableUtils.java:63)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.graph.MapVerticesInfo.get(MapVerticesInfo.java:101)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.graph.GraphJobRunner$ComputeRunnable.<init>(GraphJobRunner.j
ava:359)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.graph.GraphJobRunner.doSuperstep(GraphJobRunner.java:270)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.graph.GraphJobRunner.bsp(GraphJobRunner.java:182)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.bsp.BSPTask.runBSP(BSPTask.java:171)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.bsp.BSPTask.run(BSPTask.java:144)
attempt_201604270849_0010_000014_0:     at
org.apache.hama.bsp.GroomServer$BSPPeerChild.main(GroomServer.java:1255)
16/04/27 10:55:39 INFO bsp.BSPJobClient: Job failed.



         I really don't know how this problem occured. Who can help me?
Thanks a lot.


Ping Liu
Sincerely
Mime
  • Unnamed multipart/alternative (inline, 8-Bit, 0 bytes)
View raw message