giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Claudio Martella <claudio.marte...@gmail.com>
Subject Re: Giraph only works with LongWritables (workers >1)
Date Tue, 08 Apr 2014 21:41:17 GMT
At a very quick glance, you might have to implement a consistent hashCode()
funciton for your type.
Also, I'm not sure about your compute method. it's missing the vertex
parameter, and the getId().get_x call looks spooky.


On Tue, Apr 8, 2014 at 10:09 AM, chadi jaber <chadijaber986@hotmail.com>wrote:

> Hello!!
>
> i have realized a project with Giraph and i wanted to do a feedback about
> some problems i encountred.
> In one of my giraph Jobs, i wanted to construct a graph before making the
> processing on it.
> this graph is composed of vertices with an id composed of two integers
>
>
> import java.io.DataInput;
> import java.io.DataOutput;
> import java.io.IOException;
>
> import org.apache.hadoop.io.WritableComparable;
>
>
> /**
>  * This Class will identify a unique Node in the Graph. It represents an
> alignement
>  * configuration (one point from each signal)
>  * @author cjaber
>  */
> public class NodeKey implements WritableComparable<NodeKey>{
>
> private int _x;
>  private int _y;
>
>  public int get_x() {
> return _x;
>  }
>
> public int get_y() {
> return _y;
> }
>
>
> public NodeKey() {
> this._x = 0;
> this._y = 0;
>  }
>
> public NodeKey(int _x, int _y) {
> this._x = _x;
>  this._y = _y;
> }
>  public NodeKey(NodeKey key) {
>  this._x = key.get_x();
> this._y = key.get_y();
> }
>
>
> @Override
> public void readFields(DataInput input) throws IOException {
> _x=input.readInt();
>  _y=input.readInt();
> }
>
> @Override
> public void write(DataOutput output) throws IOException {
>  output.writeInt(_x);
> output.writeInt(_y);
>
> }
>
> @Override
> public int compareTo(NodeKey otherkey) {
>
> int ret=this._x-otherkey._x;
>  if(ret == 0){
> return this._y-otherkey._y;
> }else{
> return ret;
>  }
>
> }
>
> @Override
> public String toString() {
>  return "("+_x+"|"+_y+")";
> }
>
> public boolean isSource() {
>  // TODO Auto-generated method stub
> return (_x==0) && (_y==0);
> }
>
> @Override
> public boolean equals(Object obj) {
> if (!(obj instanceof NodeKey))return false;
>  NodeKey other = (NodeKey)obj;
>
> return (this._x==other._x) && (this._y==other._y);
> }
>
>
>
> }
> *my compute function doing changes to the graph:*
>
> @Override
> public void compute(Iterable<NodeMessage> messages) throws IOException {
>
> int currentId[]= {getId().get_x,getId().get_y};
>
> System.out.println("("+currentId[0]+"|"+currentId[1]+")"+": I'm in the
> superstep "+getSuperstep());
>
>  if(getSuperstep() == currentId[0] && getSuperstep() < 10){
> addVertexRequest(new NodeKey(currentId[0]+1,currentId[1]+1),new
> NodeValue());
>  }else{
> voteToHalt();
> }
> }
>
>
> but when launching this with more than one worker it throws null pointer
> exception:
>
> java.lang.IllegalStateException: run: Caught an unrecoverable exception
> null
>  at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:102)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
>  at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
>  at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
>  at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
> at org.apache.hadoop.mapred.Child.main(Child.java:249)
> Caused by: java.lang.NullPointerException
> at
> org.apache.giraph.comm.netty.NettyWorkerServer.resolveMutations(NettyWorkerServer.java:201)
> at
> org.apache.giraph.comm.netty.NettyWorkerServer.prepareSuperstep(NettyWorkerServer.java:152)
>  at
> org.apache.giraph.worker.BspServiceWorker.startSuperstep(BspServiceWorker.java:677)
> at
> org.apache.giraph.graph.GraphTaskManager.execute(GraphTaskManager.java:249)
>  at org.apache.giraph.graph.GraphMapper.run(GraphMapper.java:92)
> ... 7 more
>
>
> i had to use a longwritable to replace NodeKey combining  x and y of the
> NodeKey class
>
> So does giraph works with writables as vertex id  in giraph ? if yes does
> i missed something?
>
> Chadi
>
>
>
>


-- 
   Claudio Martella

Mime
View raw message