giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Claudio Martella <claudio.marte...@gmail.com>
Subject Re: Problems with sending/receiving messages of type IntArrayListWritable
Date Wed, 14 Aug 2013 00:08:59 GMT
It looks like there's a bug in ArrayListWritable. Try adding a
super.clear() at the beginning of readFields(). That should clear the list
before it's reused.


On Wed, Aug 14, 2013 at 12:30 AM, Kyle Orlando <kyle.r.orlando@gmail.com>wrote:

> Tried it, got the same exact results.  Do you think it's a problem with
> ArrayListWritable, or with sendMessage()?  I don't have a great
> understanding of how sendMessage() utilizes the Writable interface
> exactly... I'm guessing that at some point it uses readFields() and write()
> because those obviously need to be implemented for a class to be considered
> "Writable".  Looking at those two methods for ArrayListWritable, nothing
> jumps out at me as being a bug, though I can see where the minimum number
> of values in the ArrayList could be set incorrectly in readFields():
>
>     int numValues = in.readInt();            // read number of values
>     ensureCapacity(numValues);
>
> What do you think?
>
>
> On Tue, Aug 13, 2013 at 5:50 PM, Claudio Martella <
> claudio.martella@gmail.com> wrote:
>
>> could you try using:
>>
>> -D giraph.oneToAllMsgSending=true when you run it?
>>
>>
>> On Tue, Aug 13, 2013 at 11:41 PM, Kyle Orlando <kyle.r.orlando@gmail.com>wrote:
>>
>>> Yep, and I've pulled recently as well.
>>>
>>>
>>> On Tue, Aug 13, 2013 at 3:10 PM, Claudio Martella <
>>> claudio.martella@gmail.com> wrote:
>>>
>>>> That's strange. This looks like a bug to me. Are you using trunk?
>>>>
>>>>
>>>> On Mon, Aug 12, 2013 at 4:32 PM, Kyle Orlando <kyle.r.orlando@gmail.com
>>>> > wrote:
>>>>
>>>>> Also, I realize that I just sent an uncommented version... sorry guys,
>>>>> but the algorithm consists of just three supersteps:
>>>>>
>>>>>
>>>>> Superstep 0.) Source sends list of source's neighbors to each neighbor
>>>>>
>>>>> Superstep 1.) Neighbor receives list of source's neighbors from
>>>>> source, counts number of mutual neighbors, sends this value (in a
>>>>> 1-element list) back to source
>>>>>
>>>>> Superstep 2.) Source receives messages from neighbors that contain
>>>>> number of mutual neighbors. These are summed up and divided by the
>>>>> number of possible connections between neighbors to get the local
>>>>> clustering coefficient (between 0 and 1). This is set as the source
>>>>> vertex's value.
>>>>>
>>>>> On Mon, Aug 12, 2013 at 7:28 AM, Kyle Orlando <
>>>>> kyle.r.orlando@gmail.com> wrote:
>>>>> > Alrighty, here's my code:
>>>>> >
>>>>> >           public void compute(
>>>>> >                       Vertex<IntWritable, DoubleWritable,
>>>>> NullWritable> vertex,
>>>>> >                       Iterable<IntArrayListWritable> messages)
>>>>> throws IOException {
>>>>> >
>>>>> >                   if (isSource(vertex)) {
>>>>> >                           if (getSuperstep() == 0) {
>>>>> >                                   System.out.println("\nSUPERSTEP
>>>>> 0\n------------------------------");
>>>>> >                                   IntArrayListWritable
>>>>> sourceNeighbors = new IntArrayListWritable();
>>>>> >
>>>>> >                                   vertex.setValue(new
>>>>> DoubleWritable(0));
>>>>> >
>>>>> >                                   for (Edge<IntWritable,
>>>>> NullWritable> edge : vertex.getEdges()) {
>>>>> >                                           IntWritable targetVertex
=
>>>>> new
>>>>> > IntWritable(edge.getTargetVertexId().get());
>>>>> >
>>>>> sourceNeighbors.add(targetVertex);
>>>>> >                                   }
>>>>> >
>>>>> >                                   for (IntWritable neighbor :
>>>>> sourceNeighbors) {
>>>>> >                                           sendMessage(neighbor,
>>>>> sourceNeighbors);
>>>>> >                                           System.out.println("Vertex
>>>>> " + vertex.getId() + " sends " +
>>>>> > sourceNeighbors + " to " + neighbor);
>>>>> >                                   }
>>>>> >
>>>>> >                           } else if (getSuperstep() == 2) {
>>>>> >                                   int total = vertex.getNumEdges();
>>>>> >                                   int possibleCombos = total *
>>>>> (total - 1);
>>>>> >                                   int mutualNeighbors = 0;
>>>>> >                                   double coefficient = 0;
>>>>> >                                   int count = 1;
>>>>> >
>>>>> >                                   System.out.println("\nSUPERSTEP
>>>>> 2\n------------------------------");
>>>>> >
>>>>> >                                   for (IntArrayListWritable message
>>>>> : messages) {
>>>>> >
>>>>> System.out.println("Message " + count + " contains: " + message);
>>>>> >                                           mutualNeighbors +=
>>>>> message.get(0).get();
>>>>> >                                           count++;
>>>>> >                                   }
>>>>> >
>>>>> >                                   if (possibleCombos > 0) {
>>>>> >                                           coefficient =
>>>>> (double)mutualNeighbors / possibleCombos;
>>>>> >                                           vertex.setValue(new
>>>>> DoubleWritable(coefficient));
>>>>> >                                   }
>>>>> >
>>>>> >                           }
>>>>> >                   } else {
>>>>> >                           vertex.setValue(new DoubleWritable(0));
>>>>> >
>>>>> >                           if (getSuperstep() == 1) {
>>>>> >                                   int mutualNeighbors = 0;
>>>>> >
>>>>> >                                   System.out.println("\nSUPERSTEP
>>>>> 1\n------------------------------");
>>>>> >                                   IntArrayListWritable
>>>>> sourceNeighbors = messages.iterator().next();
>>>>> >                                   System.out.println("Vertex " +
>>>>> vertex.getId() + " receives " +
>>>>> > sourceNeighbors);
>>>>> >
>>>>> >                                   for (Edge<IntWritable,
>>>>> NullWritable> edge : vertex.getEdges()) {
>>>>> >                                           IntWritable targetVertex
=
>>>>> new
>>>>> > IntWritable(edge.getTargetVertexId().get());
>>>>> >                                           if
>>>>> (sourceNeighbors.contains(targetVertex)) mutualNeighbors++;
>>>>> >                                   }
>>>>> >
>>>>> >                                   IntArrayListWritable neighborCount
>>>>> = new IntArrayListWritable();
>>>>> >                                   neighborCount.add(new
>>>>> IntWritable(mutualNeighbors));
>>>>> >
>>>>> >                                   sendMessage(new
>>>>> IntWritable(SOURCE_ID.get(getConf())), neighborCount);
>>>>> >                                   System.out.println("Vertex " +
>>>>> vertex.getId() + " sends " +
>>>>> > neighborCount + " to Vertex " + SOURCE_ID.get(getConf()));
>>>>> >
>>>>> >                           }
>>>>> >                   }
>>>>> >
>>>>> >                 vertex.voteToHalt();
>>>>> >           }
>>>>> >
>>>>> >
>>>>> > On Fri, Aug 9, 2013 at 10:15 PM, Claudio Martella
>>>>> > <claudio.martella@gmail.com> wrote:
>>>>> >> you'd have to show us the code in the compute method. your problem
>>>>> might be
>>>>> >> caused by object reusal.
>>>>> >>
>>>>> >>
>>>>> >> On Fri, Aug 9, 2013 at 9:05 PM, Kyle Orlando <
>>>>> kyle.r.orlando@gmail.com>
>>>>> >> wrote:
>>>>> >>>
>>>>> >>> Hello,
>>>>> >>>
>>>>> >>> I am trying to write code to compute the local clustering
>>>>> coefficient
>>>>> >>> of a vertex/some vertices, and to do this I send a message
that
>>>>> >>> contains a list of the source's neighbors to each of its
neighbors.
>>>>> >>> This is in, of course, an IntArrayListWritable.  I check
the list
>>>>> that
>>>>> >>> I am sending before invoking sendMessage(), and it appears
to be
>>>>> >>> correct. However, when I use message.iterator.next() or
something
>>>>> >>> similar, the IntArrayListWritable objects seem to repeat
or
>>>>> something.
>>>>> >>> I did some logging, and here was the output:
>>>>> >>>
>>>>> >>> SUPERSTEP 0
>>>>> >>> ------------------------------
>>>>> >>> Vertex 1 sends [2, 3, 4] to Vertex 2
>>>>> >>> Vertex 1 sends [2, 3, 4] to Vertex 3
>>>>> >>> Vertex 1 sends [2, 3, 4] to Vertex 4
>>>>> >>>
>>>>> >>> SUPERSTEP 1
>>>>> >>> ------------------------------
>>>>> >>> Vertex 2 receives [2, 3, 4]
>>>>> >>> Vertex 2 sends [1] to Vertex 1
>>>>> >>>
>>>>> >>> SUPERSTEP 1
>>>>> >>> ------------------------------
>>>>> >>> Vertex 3 receives [2, 3, 4, 2, 3, 4]
>>>>> >>> Vertex 3 sends [1] to Vertex 1
>>>>> >>>
>>>>> >>> SUPERSTEP 1
>>>>> >>> ------------------------------
>>>>> >>> Vertex 4 receives [2, 3, 4, 2, 3, 4, 2, 3, 4]
>>>>> >>> Vertex 4 sends [0] to Vertex 1
>>>>> >>>
>>>>> >>> SUPERSTEP 2
>>>>> >>> ------------------------------
>>>>> >>> Message 1 contains: [1]
>>>>> >>> Message 2 contains: [1, 1, 1]
>>>>> >>> Message 3 contains: [1, 1, 1, 1, 1, 0]
>>>>> >>>
>>>>> >>> What is happening?
>>>>> >>>
>>>>> >>> --
>>>>> >>> Kyle Orlando
>>>>> >>> Computer Engineering Major
>>>>> >>> University of Maryland
>>>>> >>
>>>>> >>
>>>>> >>
>>>>> >>
>>>>> >> --
>>>>> >>    Claudio Martella
>>>>> >>    claudio.martella@gmail.com
>>>>> >
>>>>> >
>>>>> >
>>>>> > --
>>>>> > Kyle Orlando
>>>>> > Computer Engineering Major
>>>>> > University of Maryland
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Kyle Orlando
>>>>> Computer Engineering Major
>>>>> University of Maryland
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>>    Claudio Martella
>>>>    claudio.martella@gmail.com
>>>>
>>>
>>>
>>>
>>> --
>>> Kyle Orlando
>>> Computer Engineering Major
>>> University of Maryland
>>>
>>
>>
>>
>> --
>>    Claudio Martella
>>    claudio.martella@gmail.com
>>
>
>
>
> --
> Kyle Orlando
> Computer Engineering Major
> University of Maryland
>



-- 
   Claudio Martella
   claudio.martella@gmail.com

Mime
View raw message