There is a bug in ArrayListWritable readFileds should clear the list before reading any data to it. We are reusing message objects and every time we call readFields data is just going to be appended to the previous message. It's very easy to fix, Kyle please feel free to open an issue and submit a patch for it.

From: Kyle Orlando <kyle.r.orlando@gmail.com>
Reply-To: "user@giraph.apache.org" <user@giraph.apache.org>
Date: Tuesday, August 13, 2013 3:30 PM
To: "user@giraph.apache.org" <user@giraph.apache.org>
Subject: Re: Problems with sending/receiving messages of type IntArrayListWritable

Tried it, got the same exact results.  Do you think it's a problem with ArrayListWritable, or with sendMessage()?  I don't have a great understanding of how sendMessage() utilizes the Writable interface exactly... I'm guessing that at some point it uses readFields() and write() because those obviously need to be implemented for a class to be considered "Writable".  Looking at those two methods for ArrayListWritable, nothing jumps out at me as being a bug, though I can see where the minimum number of values in the ArrayList could be set incorrectly in readFields():

    int numValues = in.readInt();            // read number of values
    ensureCapacity(numValues);

What do you think?


On Tue, Aug 13, 2013 at 5:50 PM, Claudio Martella <claudio.martella@gmail.com> wrote:
could you try using:

-D giraph.oneToAllMsgSending=true when you run it?


On Tue, Aug 13, 2013 at 11:41 PM, Kyle Orlando <kyle.r.orlando@gmail.com> wrote:
Yep, and I've pulled recently as well.


On Tue, Aug 13, 2013 at 3:10 PM, Claudio Martella <claudio.martella@gmail.com> wrote:
That's strange. This looks like a bug to me. Are you using trunk?


On Mon, Aug 12, 2013 at 4:32 PM, Kyle Orlando <kyle.r.orlando@gmail.com> wrote:
Also, I realize that I just sent an uncommented version... sorry guys,
but the algorithm consists of just three supersteps:


Superstep 0.) Source sends list of source's neighbors to each neighbor

Superstep 1.) Neighbor receives list of source's neighbors from
source, counts number of mutual neighbors, sends this value (in a
1-element list) back to source

Superstep 2.) Source receives messages from neighbors that contain
number of mutual neighbors. These are summed up and divided by the
number of possible connections between neighbors to get the local
clustering coefficient (between 0 and 1). This is set as the source
vertex's value.

On Mon, Aug 12, 2013 at 7:28 AM, Kyle Orlando <kyle.r.orlando@gmail.com> wrote:
> Alrighty, here's my code:
>
>           public void compute(
>                       Vertex<IntWritable, DoubleWritable, NullWritable> vertex,
>                       Iterable<IntArrayListWritable> messages) throws IOException {
>
>                   if (isSource(vertex)) {
>                           if (getSuperstep() == 0) {
>                                   System.out.println("\nSUPERSTEP 0\n------------------------------");
>                                   IntArrayListWritable sourceNeighbors = new IntArrayListWritable();
>
>                                   vertex.setValue(new DoubleWritable(0));
>
>                                   for (Edge<IntWritable, NullWritable> edge : vertex.getEdges()) {
>                                           IntWritable targetVertex = new
> IntWritable(edge.getTargetVertexId().get());
>                                           sourceNeighbors.add(targetVertex);
>                                   }
>
>                                   for (IntWritable neighbor : sourceNeighbors) {
>                                           sendMessage(neighbor, sourceNeighbors);
>                                           System.out.println("Vertex " + vertex.getId() + " sends " +
> sourceNeighbors + " to " + neighbor);
>                                   }
>
>                           } else if (getSuperstep() == 2) {
>                                   int total = vertex.getNumEdges();
>                                   int possibleCombos = total * (total - 1);
>                                   int mutualNeighbors = 0;
>                                   double coefficient = 0;
>                                   int count = 1;
>
>                                   System.out.println("\nSUPERSTEP 2\n------------------------------");
>
>                                   for (IntArrayListWritable message : messages) {
>                                           System.out.println("Message " + count + " contains: " + message);
>                                           mutualNeighbors += message.get(0).get();
>                                           count++;
>                                   }
>
>                                   if (possibleCombos > 0) {
>                                           coefficient = (double)mutualNeighbors / possibleCombos;
>                                           vertex.setValue(new DoubleWritable(coefficient));
>                                   }
>
>                           }
>                   } else {
>                           vertex.setValue(new DoubleWritable(0));
>
>                           if (getSuperstep() == 1) {
>                                   int mutualNeighbors = 0;
>
>                                   System.out.println("\nSUPERSTEP 1\n------------------------------");
>                                   IntArrayListWritable sourceNeighbors = messages.iterator().next();
>                                   System.out.println("Vertex " + vertex.getId() + " receives " +
> sourceNeighbors);
>
>                                   for (Edge<IntWritable, NullWritable> edge : vertex.getEdges()) {
>                                           IntWritable targetVertex = new
> IntWritable(edge.getTargetVertexId().get());
>                                           if (sourceNeighbors.contains(targetVertex)) mutualNeighbors++;
>                                   }
>
>                                   IntArrayListWritable neighborCount = new IntArrayListWritable();
>                                   neighborCount.add(new IntWritable(mutualNeighbors));
>
>                                   sendMessage(new IntWritable(SOURCE_ID.get(getConf())), neighborCount);
>                                   System.out.println("Vertex " + vertex.getId() + " sends " +
> neighborCount + " to Vertex " + SOURCE_ID.get(getConf()));
>
>                           }
>                   }
>
>                 vertex.voteToHalt();
>           }
>
>
> On Fri, Aug 9, 2013 at 10:15 PM, Claudio Martella
> <claudio.martella@gmail.com> wrote:
>> you'd have to show us the code in the compute method. your problem might be
>> caused by object reusal.
>>
>>
>> On Fri, Aug 9, 2013 at 9:05 PM, Kyle Orlando <kyle.r.orlando@gmail.com>
>> wrote:
>>>
>>> Hello,
>>>
>>> I am trying to write code to compute the local clustering coefficient
>>> of a vertex/some vertices, and to do this I send a message that
>>> contains a list of the source's neighbors to each of its neighbors.
>>> This is in, of course, an IntArrayListWritable.  I check the list that
>>> I am sending before invoking sendMessage(), and it appears to be
>>> correct. However, when I use message.iterator.next() or something
>>> similar, the IntArrayListWritable objects seem to repeat or something.
>>> I did some logging, and here was the output:
>>>
>>> SUPERSTEP 0
>>> ------------------------------
>>> Vertex 1 sends [2, 3, 4] to Vertex 2
>>> Vertex 1 sends [2, 3, 4] to Vertex 3
>>> Vertex 1 sends [2, 3, 4] to Vertex 4
>>>
>>> SUPERSTEP 1
>>> ------------------------------
>>> Vertex 2 receives [2, 3, 4]
>>> Vertex 2 sends [1] to Vertex 1
>>>
>>> SUPERSTEP 1
>>> ------------------------------
>>> Vertex 3 receives [2, 3, 4, 2, 3, 4]
>>> Vertex 3 sends [1] to Vertex 1
>>>
>>> SUPERSTEP 1
>>> ------------------------------
>>> Vertex 4 receives [2, 3, 4, 2, 3, 4, 2, 3, 4]
>>> Vertex 4 sends [0] to Vertex 1
>>>
>>> SUPERSTEP 2
>>> ------------------------------
>>> Message 1 contains: [1]
>>> Message 2 contains: [1, 1, 1]
>>> Message 3 contains: [1, 1, 1, 1, 1, 0]
>>>
>>> What is happening?
>>>
>>> --
>>> Kyle Orlando
>>> Computer Engineering Major
>>> University of Maryland
>>
>>
>>
>>
>> --
>>    Claudio Martella
>>    claudio.martella@gmail.com
>
>
>
> --
> Kyle Orlando
> Computer Engineering Major
> University of Maryland



--
Kyle Orlando
Computer Engineering Major
University of Maryland



--
   Claudio Martella
   claudio.martella@gmail.com   



--
Kyle Orlando
Computer Engineering Major
University of Maryland



--
   Claudio Martella
   claudio.martella@gmail.com   



--
Kyle Orlando
Computer Engineering Major
University of Maryland