giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From chadi jaber <chadijaber...@hotmail.com>
Subject RE: Problem with Giraph (please help me)
Date Thu, 09 Jan 2014 10:08:07 GMT
Hello LukasI have enclosed in my previous emails the exception. It seems to be a serialization
issue (This occurs only when workers > 1)
...2013-12-31 16:27:33,494 INFO org.apache.giraph.comm.netty.NettyClient: connectAllAddresses:
Successfully added 4 connections, (4 total connected) 0 failed, 0 failures total.2013-12-31
16:27:33,501 INFO org.apache.giraph.worker.BspServiceWorker: loadInputSplits: Using 1 thread(s),
originally 1 threads(s) for 1 total splits.2013-12-31 16:27:33,508 INFO org.apache.giraph.comm.SendPartitionCache:
SendPartitionCache: maxVerticesPerTransfer = 100002013-12-31 16:27:33,508 INFO org.apache.giraph.comm.SendPartitionCache:
SendPartitionCache: maxEdgesPerTransfer = 800002013-12-31 16:27:33,524 INFO org.apache.giraph.worker.InputSplitsCallable:
call: Loaded 0 input splits in 0.020270009 secs, (v=0, e=0) 0.0 vertices/sec, 0.0 edges/sec2013-12-31
16:27:33,527 INFO org.apache.giraph.comm.netty.NettyClient: waitAllRequests: Finished all
requests. MBytes/sec sent = 0, MBytes/sec received = 0, MBytesSent = 0, MBytesReceived = 0,
ave sent req MBytes = 0, ave received req MBytes = 0, secs waited = 0.6562013-12-31 16:27:33,527
INFO org.apache.giraph.worker.BspServiceWorker: setup: Finally loaded a total of (v=0, e=0)2013-12-31
16:27:33,598 INFO org.apache.giraph.comm.netty.handler.RequestDecoder: decode: Server window
metrics MBytes/sec sent = 0, MBytes/sec received = 0, MBytesSent = 0, MBytesReceived = 0,
ave sent req MBytes = 0, ave received req MBytes = 0, secs waited = 0.8162013-12-31 16:27:33,605
WARN org.apache.giraph.comm.netty.handler.RequestServerHandler: exceptionCaught: Channel failed
with remote address /172.16.45.53:59257java.io.EOFException	at org.jboss.netty.buffer.ChannelBufferInputStream.checkAvailable(ChannelBufferInputStream.java:231)
at org.jboss.netty.buffer.ChannelBufferInputStream.readInt(ChannelBufferInputStream.java:174)
at org.apache.giraph.edge.ByteArrayEdges.readFields(ByteArrayEdges.java:172)	at org.apache.giraph.utils.WritableUtils.reinitializeVertexFromDataInput(WritableUtils.java:480)
at org.apache.giraph.utils.WritableUtils.readVertexFromDataInput(WritableUtils.java:511)	at
org.apache.giraph.partition.SimplePartition.readFields(SimplePartition.java:126)	at org.apache.giraph.comm.requests.SendVertexRequest.readFieldsRequest(SendVertexRequest.java:66)
at org.apache.giraph.comm.requests.WritableRequest.readFields(WritableRequest.java:120)	at
org.apache.giraph.comm.netty.handler.RequestDecoder.decode(RequestDecoder.java:92)	at org.jboss.netty.handler.codec.oneone.OneToOneDecoder.handleUpstream(OneToOneDecoder.java:72)
at org.jboss.netty.handler.execution.ChannelEventRunnable.run(ChannelEventRunnable.java:69)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)the code for my vertex compute function :public class
MergeVertex extendsVertex<LongWritable,DoubleWritable, DoubleWritable, NodeMessage>
{.../***	 * Convert a Vertex Id from its LongWritable format to Point format (2 Element Array
Format)	 * @param lng LongWritable Format of the VertexId	 * @return Alignment point Array
 */	public static int[] cvtLongToPoint(LongWritable lng){		int[] point={0,0};		point[0]=(int)
(lng.get()/1000);		point[1]=(int) (lng.get()% 1000);		return point;	}	@Override	public void
compute(Iterable<NodeMessage> messages) throws IOException {		int currentId[]= cvtLongToPoint(getId());
	if (getSuperstep()==0) {			//NodeValue nv=new NodeValue();			setValue(new DoubleWritable(0d));
	}		_signallength=getContext().getConfiguration().getInt("SignalLength",0);		if((getSuperstep()
< _signallength && getId().get()!=0L) || (getSuperstep()== 0 && getId().get()==0L)){
		LongWritable dstId=new LongWritable();			//Nodes which are on Graph "Spine" //Remaining
Edges Construction			if(currentId[0]== currentId[1]){				//right Side				for (int i=currentId[1]+1;i<_signallength;i++){
				dstId=cvtPointToLong(currentId[0]+1,i);					addVertexRequest(dstId,new DoubleWritable(Double.MAX_VALUE));
				addEdgeRequest(getId(),EdgeFactory.create(dstId, new DoubleWritable(computeCost(getId(),dstId))));
			}				//Left Side				for (int i=currentId[0]+2;i<_signallength;i++){					dstId=cvtPointToLong(i,currentId[1]+1);
				addVertexRequest(dstId,new DoubleWritable(Double.MAX_VALUE));					addEdgeRequest(getId(),EdgeFactory.create(dstId,
new DoubleWritable(computeCost(getId(),dstId))));				}				//Nodes which are not on Graph "Spine"
//Remaining Edges Construction			}else{				//right Side				if(currentId[0]+1<_signallength){
				for (int i=currentId[1]+1;i<_signallength;i++){						dstId=cvtPointToLong(currentId[0]+1,i);
					addEdgeRequest(getId(),EdgeFactory.create(dstId, new DoubleWritable(computeCost(getId(),dstId))));
				}				}				//Left Side				if(currentId[1]+1<_signallength){					for (int i=currentId[0]+2;i<_signallength;i++){
					dstId=cvtPointToLong(i,currentId[1]+1);						addEdgeRequest(getId(),EdgeFactory.create(dstId,
new DoubleWritable(computeCost(getId(),dstId))));					}				}			}			//No need to other vertex
than source to be active			if(getId().get() != 0L){				voteToHalt();			}		}else if (getSuperstep()
>= _signallength && getSuperstep() < 2*_signallength){			double minDist;			long
minSource=0L;			if(getId().get() == 0L){				minDist=0;			}else{				minDist=Double.MAX_VALUE;
		}			for(NodeMessage message : messages){				if(minDist > message.get()){					minDist=message.get();
				minSource=message.getSourceID();				}			}			if (minDist < getValue().get()){				setValue(new
DoubleWritable(minDist));								for (Edge<LongWritable, DoubleWritable> edge : getEdges())
{					double distance = minDist + edge.getValue().get();					sendMessage(edge.getTargetVertexId(),
						new NodeMessage(distance,getId().get()));				}			}			//Only last Node is active			if(currentId[0]
!= _signallength-1 || currentId[1] != _signallength-1){				voteToHalt();			}		}else if(getSuperstep()
>= 2*_signallength){			voteToHalt();		}	} If you need more details please don't hesitate.Thanks
in advance,Chadi
Date: Thu, 9 Jan 2014 10:49:54 +0100
From: lukas.nalezenec@firma.seznam.cz
To: chadijaber986@hotmail.com
Subject: Re: Problem with Giraph (please help me)


  
    
  
  
    Hi, 

      Find the mapper running on the remote address and check what
      happened. Maybe there will be exception.

      Lukas

      

      On 9.1.2014 09:38, chadi jaber wrote:

    
    exceptionCaught: Channel
          failed with remote address /172.16.45.53:59257
    
 		 	   		  
Mime
View raw message