giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Claudio Martella <claudio.marte...@gmail.com>
Subject Re: Problems with sending/receiving messages of type IntArrayListWritable
Date Tue, 13 Aug 2013 21:50:48 GMT
could you try using:

-D giraph.oneToAllMsgSending=true when you run it?


On Tue, Aug 13, 2013 at 11:41 PM, Kyle Orlando <kyle.r.orlando@gmail.com>wrote:

> Yep, and I've pulled recently as well.
>
>
> On Tue, Aug 13, 2013 at 3:10 PM, Claudio Martella <
> claudio.martella@gmail.com> wrote:
>
>> That's strange. This looks like a bug to me. Are you using trunk?
>>
>>
>> On Mon, Aug 12, 2013 at 4:32 PM, Kyle Orlando <kyle.r.orlando@gmail.com>wrote:
>>
>>> Also, I realize that I just sent an uncommented version... sorry guys,
>>> but the algorithm consists of just three supersteps:
>>>
>>>
>>> Superstep 0.) Source sends list of source's neighbors to each neighbor
>>>
>>> Superstep 1.) Neighbor receives list of source's neighbors from
>>> source, counts number of mutual neighbors, sends this value (in a
>>> 1-element list) back to source
>>>
>>> Superstep 2.) Source receives messages from neighbors that contain
>>> number of mutual neighbors. These are summed up and divided by the
>>> number of possible connections between neighbors to get the local
>>> clustering coefficient (between 0 and 1). This is set as the source
>>> vertex's value.
>>>
>>> On Mon, Aug 12, 2013 at 7:28 AM, Kyle Orlando <kyle.r.orlando@gmail.com>
>>> wrote:
>>> > Alrighty, here's my code:
>>> >
>>> >           public void compute(
>>> >                       Vertex<IntWritable, DoubleWritable,
>>> NullWritable> vertex,
>>> >                       Iterable<IntArrayListWritable> messages) throws
>>> IOException {
>>> >
>>> >                   if (isSource(vertex)) {
>>> >                           if (getSuperstep() == 0) {
>>> >                                   System.out.println("\nSUPERSTEP
>>> 0\n------------------------------");
>>> >                                   IntArrayListWritable sourceNeighbors
>>> = new IntArrayListWritable();
>>> >
>>> >                                   vertex.setValue(new
>>> DoubleWritable(0));
>>> >
>>> >                                   for (Edge<IntWritable, NullWritable>
>>> edge : vertex.getEdges()) {
>>> >                                           IntWritable targetVertex =
>>> new
>>> > IntWritable(edge.getTargetVertexId().get());
>>> >
>>> sourceNeighbors.add(targetVertex);
>>> >                                   }
>>> >
>>> >                                   for (IntWritable neighbor :
>>> sourceNeighbors) {
>>> >                                           sendMessage(neighbor,
>>> sourceNeighbors);
>>> >                                           System.out.println("Vertex "
>>> + vertex.getId() + " sends " +
>>> > sourceNeighbors + " to " + neighbor);
>>> >                                   }
>>> >
>>> >                           } else if (getSuperstep() == 2) {
>>> >                                   int total = vertex.getNumEdges();
>>> >                                   int possibleCombos = total * (total
>>> - 1);
>>> >                                   int mutualNeighbors = 0;
>>> >                                   double coefficient = 0;
>>> >                                   int count = 1;
>>> >
>>> >                                   System.out.println("\nSUPERSTEP
>>> 2\n------------------------------");
>>> >
>>> >                                   for (IntArrayListWritable message :
>>> messages) {
>>> >                                           System.out.println("Message
>>> " + count + " contains: " + message);
>>> >                                           mutualNeighbors +=
>>> message.get(0).get();
>>> >                                           count++;
>>> >                                   }
>>> >
>>> >                                   if (possibleCombos > 0) {
>>> >                                           coefficient =
>>> (double)mutualNeighbors / possibleCombos;
>>> >                                           vertex.setValue(new
>>> DoubleWritable(coefficient));
>>> >                                   }
>>> >
>>> >                           }
>>> >                   } else {
>>> >                           vertex.setValue(new DoubleWritable(0));
>>> >
>>> >                           if (getSuperstep() == 1) {
>>> >                                   int mutualNeighbors = 0;
>>> >
>>> >                                   System.out.println("\nSUPERSTEP
>>> 1\n------------------------------");
>>> >                                   IntArrayListWritable sourceNeighbors
>>> = messages.iterator().next();
>>> >                                   System.out.println("Vertex " +
>>> vertex.getId() + " receives " +
>>> > sourceNeighbors);
>>> >
>>> >                                   for (Edge<IntWritable, NullWritable>
>>> edge : vertex.getEdges()) {
>>> >                                           IntWritable targetVertex =
>>> new
>>> > IntWritable(edge.getTargetVertexId().get());
>>> >                                           if
>>> (sourceNeighbors.contains(targetVertex)) mutualNeighbors++;
>>> >                                   }
>>> >
>>> >                                   IntArrayListWritable neighborCount =
>>> new IntArrayListWritable();
>>> >                                   neighborCount.add(new
>>> IntWritable(mutualNeighbors));
>>> >
>>> >                                   sendMessage(new
>>> IntWritable(SOURCE_ID.get(getConf())), neighborCount);
>>> >                                   System.out.println("Vertex " +
>>> vertex.getId() + " sends " +
>>> > neighborCount + " to Vertex " + SOURCE_ID.get(getConf()));
>>> >
>>> >                           }
>>> >                   }
>>> >
>>> >                 vertex.voteToHalt();
>>> >           }
>>> >
>>> >
>>> > On Fri, Aug 9, 2013 at 10:15 PM, Claudio Martella
>>> > <claudio.martella@gmail.com> wrote:
>>> >> you'd have to show us the code in the compute method. your problem
>>> might be
>>> >> caused by object reusal.
>>> >>
>>> >>
>>> >> On Fri, Aug 9, 2013 at 9:05 PM, Kyle Orlando <
>>> kyle.r.orlando@gmail.com>
>>> >> wrote:
>>> >>>
>>> >>> Hello,
>>> >>>
>>> >>> I am trying to write code to compute the local clustering coefficient
>>> >>> of a vertex/some vertices, and to do this I send a message that
>>> >>> contains a list of the source's neighbors to each of its neighbors.
>>> >>> This is in, of course, an IntArrayListWritable.  I check the list
>>> that
>>> >>> I am sending before invoking sendMessage(), and it appears to be
>>> >>> correct. However, when I use message.iterator.next() or something
>>> >>> similar, the IntArrayListWritable objects seem to repeat or
>>> something.
>>> >>> I did some logging, and here was the output:
>>> >>>
>>> >>> SUPERSTEP 0
>>> >>> ------------------------------
>>> >>> Vertex 1 sends [2, 3, 4] to Vertex 2
>>> >>> Vertex 1 sends [2, 3, 4] to Vertex 3
>>> >>> Vertex 1 sends [2, 3, 4] to Vertex 4
>>> >>>
>>> >>> SUPERSTEP 1
>>> >>> ------------------------------
>>> >>> Vertex 2 receives [2, 3, 4]
>>> >>> Vertex 2 sends [1] to Vertex 1
>>> >>>
>>> >>> SUPERSTEP 1
>>> >>> ------------------------------
>>> >>> Vertex 3 receives [2, 3, 4, 2, 3, 4]
>>> >>> Vertex 3 sends [1] to Vertex 1
>>> >>>
>>> >>> SUPERSTEP 1
>>> >>> ------------------------------
>>> >>> Vertex 4 receives [2, 3, 4, 2, 3, 4, 2, 3, 4]
>>> >>> Vertex 4 sends [0] to Vertex 1
>>> >>>
>>> >>> SUPERSTEP 2
>>> >>> ------------------------------
>>> >>> Message 1 contains: [1]
>>> >>> Message 2 contains: [1, 1, 1]
>>> >>> Message 3 contains: [1, 1, 1, 1, 1, 0]
>>> >>>
>>> >>> What is happening?
>>> >>>
>>> >>> --
>>> >>> Kyle Orlando
>>> >>> Computer Engineering Major
>>> >>> University of Maryland
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> --
>>> >>    Claudio Martella
>>> >>    claudio.martella@gmail.com
>>> >
>>> >
>>> >
>>> > --
>>> > Kyle Orlando
>>> > Computer Engineering Major
>>> > University of Maryland
>>>
>>>
>>>
>>> --
>>> Kyle Orlando
>>> Computer Engineering Major
>>> University of Maryland
>>>
>>
>>
>>
>> --
>>    Claudio Martella
>>    claudio.martella@gmail.com
>>
>
>
>
> --
> Kyle Orlando
> Computer Engineering Major
> University of Maryland
>



-- 
   Claudio Martella
   claudio.martella@gmail.com

Mime
View raw message