giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Simon McGloin <simonmcgl...@gmail.com>
Subject Re: Error when using ArrayListWritable<Text> as message
Date Thu, 17 Oct 2013 14:04:18 GMT
Thanks for the advice Manuel. I created a TextArrayListMessage object to
use as a message between supersteps. I had to include a line to wipe the
ArrayList during the readFields method or else during the next superstep
the array would have unwanted Text objects in it. It seemed like the
TextArrayListMessage gets reused and the readFields just keeps adding to
the textArrayList.  e.g.

@Override
public void readFields(DataInput in) throws IOException {
int numFields = in.readInt();
textArrayList.clear(); // Have to clear the list or get unexpected results
for(int i = 0; i < numFields; i++) {
Text t = new Text(WritableUtils.readCompressedByteArray(in));
textArrayList.add(t);
}
}


On Wed, Oct 16, 2013 at 7:21 PM, Manuel Lagang <manuellagang@gmail.com>wrote:

> I think you need to have your message value class as TextArrayListMessage
> instead of ArrayListWritable<Text>. That might require you to move
> TextArrayListMessage outside of ArrayListTextBug.
>
>
> On Wed, Oct 16, 2013 at 10:01 AM, Simon McGloin <simonmcgloin@gmail.com>wrote:
>
>> Hey Guys,
>>
>> I've only been using Giraph a few days so am very new to it. I'm currently using
Giraph 1.0.0. I'm getting the error below when I try to send an ArrayListWritable<Text>
message. The error happens between supersteps. If you run the sample code I've included "Superstep
1" never gets printed as the job fails after Superstep 0. Is this a bug or am I doing something
wrong. In my full code I need to be able to send a list of Text based vertex ids between supersteps.
Should I not be using org.apache.hadoop.io.Text and implement my own writable object?
>>
>> Any help is appreciated.
>>
>> Regards,
>>
>> Simon
>>
>>
>> Caused by: java.util.concurrent.ExecutionException: java.lang.IllegalArgumentException:
createMessageValue: Failed to instantiate
>> 	at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:232)
>> 	at java.util.concurrent.FutureTask.get(FutureTask.java:91)
>> 	at org.apache.giraph.utils.ProgressableUtils$FutureWaitable.waitFor(ProgressableUtils.java:271)
>> 	at org.apache.giraph.utils.ProgressableUtils.waitFor(ProgressableUtils.java:143)
>> 	... 13 more
>> Caused by: java.lang.IllegalArgumentException: createMessageValue: Failed to instantiate
>> 	at org.apache.giraph.conf.ImmutableClassesGiraphConfiguration.createMessageValue(ImmutableClassesGiraphConfiguration.java:581)
>> 	at org.apache.giraph.utils.ByteArrayVertexIdMessages.createData(ByteArrayVertexIdMessages.java:66)
>> 	at org.apache.giraph.utils.ByteArrayVertexIdMessages.createData(ByteArrayVertexIdMessages.java:34)
>> 	at org.apache.giraph.utils.ByteArrayVertexIdData$VertexIdDataIterator.next(ByteArrayVertexIdData.java:205)
>> 	at org.apache.giraph.comm.messages.ByteArrayMessagesPerVertexStore.addPartitionMessages(ByteArrayMessagesPerVertexStore.java:116)
>> 	at org.apache.giraph.comm.requests.SendWorkerMessagesRequest.doRequest(SendWorkerMessagesRequest.java:72)
>> 	at org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.doRequest(NettyWorkerClientRequestProcessor.java:470)
>> 	at org.apache.giraph.comm.netty.NettyWorkerClientRequestProcessor.flush(NettyWorkerClientRequestProcessor.java:419)
>> 	at org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:193)
>> 	at org.apache.giraph.graph.ComputeCallable.call(ComputeCallable.java:70)
>> 	at org.apache.giraph.utils.LogStacktraceCallable.call(LogStacktraceCallable.java:51)
>> 	at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
>> 	at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>> 	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>> 	at java.lang.Thread.run(Thread.java:662)
>>
>>
>>
>>
>> package com.adaptivemobile.tarantula.batchlayer.giraph.run;
>>
>> import java.io.IOException;
>>
>> import org.apache.giraph.graph.Vertex;
>> import org.apache.giraph.utils.ArrayListWritable;
>> import org.apache.hadoop.io.NullWritable;
>> import org.apache.hadoop.io.Text;
>>
>> public class ArrayListTextBug extends Vertex<Text, NullWritable, NullWritable,
ArrayListWritable<Text>>{
>>
>> 	@Override
>> 	public void compute(Iterable<ArrayListWritable<Text>> messages) throws
IOException {
>> 		
>> 		if (getSuperstep() == 0) {
>> 			System.out.println("\nSUPERSTEP 0 - " + getId() + "\n------------------------------");
>> 			TextArrayListMessage initialMessage = new TextArrayListMessage();
>> 			initialMessage.add(getId());
>> 			this.sendMessageToAllEdges(initialMessage);	
>> 		    System.out.println("Vertex " + getId() + " sends TextArrayListMessage to "
+ getNumEdges() + " edges");
>> 	    }
>> 		if (getSuperstep() == 1) {
>> 			System.out.println("\nSUPERSTEP 1 - " + getId() + "\n------------------------------");
>> 		}
>> 	}
>> 		
>> 	public class TextArrayListMessage extends ArrayListWritable<Text>{
>> 			
>> 		/**
>> 		 *
>> 		 */
>> 		private static final long serialVersionUID = 1L;
>>
>> 		public TextArrayListMessage() {
>> 			super();
>> 		}
>>
>> 		@Override
>> 		public void setClass() {
>> 			this.setClass(Text.class);
>> 		}
>>
>> 	}
>>
>> }
>>
>>
>>
>

Mime
View raw message