giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kyle Orlando <kyle.r.orla...@gmail.com>
Subject Re: Problems with sending/receiving messages of type IntArrayListWritable
Date Tue, 13 Aug 2013 22:30:05 GMT
Tried it, got the same exact results.  Do you think it's a problem with
ArrayListWritable, or with sendMessage()?  I don't have a great
understanding of how sendMessage() utilizes the Writable interface
exactly... I'm guessing that at some point it uses readFields() and write()
because those obviously need to be implemented for a class to be considered
"Writable".  Looking at those two methods for ArrayListWritable, nothing
jumps out at me as being a bug, though I can see where the minimum number
of values in the ArrayList could be set incorrectly in readFields():

    int numValues = in.readInt();            // read number of values
    ensureCapacity(numValues);

What do you think?


On Tue, Aug 13, 2013 at 5:50 PM, Claudio Martella <
claudio.martella@gmail.com> wrote:

> could you try using:
>
> -D giraph.oneToAllMsgSending=true when you run it?
>
>
> On Tue, Aug 13, 2013 at 11:41 PM, Kyle Orlando <kyle.r.orlando@gmail.com>wrote:
>
>> Yep, and I've pulled recently as well.
>>
>>
>> On Tue, Aug 13, 2013 at 3:10 PM, Claudio Martella <
>> claudio.martella@gmail.com> wrote:
>>
>>> That's strange. This looks like a bug to me. Are you using trunk?
>>>
>>>
>>> On Mon, Aug 12, 2013 at 4:32 PM, Kyle Orlando <kyle.r.orlando@gmail.com>wrote:
>>>
>>>> Also, I realize that I just sent an uncommented version... sorry guys,
>>>> but the algorithm consists of just three supersteps:
>>>>
>>>>
>>>> Superstep 0.) Source sends list of source's neighbors to each neighbor
>>>>
>>>> Superstep 1.) Neighbor receives list of source's neighbors from
>>>> source, counts number of mutual neighbors, sends this value (in a
>>>> 1-element list) back to source
>>>>
>>>> Superstep 2.) Source receives messages from neighbors that contain
>>>> number of mutual neighbors. These are summed up and divided by the
>>>> number of possible connections between neighbors to get the local
>>>> clustering coefficient (between 0 and 1). This is set as the source
>>>> vertex's value.
>>>>
>>>> On Mon, Aug 12, 2013 at 7:28 AM, Kyle Orlando <kyle.r.orlando@gmail.com>
>>>> wrote:
>>>> > Alrighty, here's my code:
>>>> >
>>>> >           public void compute(
>>>> >                       Vertex<IntWritable, DoubleWritable,
>>>> NullWritable> vertex,
>>>> >                       Iterable<IntArrayListWritable> messages)
throws
>>>> IOException {
>>>> >
>>>> >                   if (isSource(vertex)) {
>>>> >                           if (getSuperstep() == 0) {
>>>> >                                   System.out.println("\nSUPERSTEP
>>>> 0\n------------------------------");
>>>> >                                   IntArrayListWritable
>>>> sourceNeighbors = new IntArrayListWritable();
>>>> >
>>>> >                                   vertex.setValue(new
>>>> DoubleWritable(0));
>>>> >
>>>> >                                   for (Edge<IntWritable,
>>>> NullWritable> edge : vertex.getEdges()) {
>>>> >                                           IntWritable targetVertex =
>>>> new
>>>> > IntWritable(edge.getTargetVertexId().get());
>>>> >
>>>> sourceNeighbors.add(targetVertex);
>>>> >                                   }
>>>> >
>>>> >                                   for (IntWritable neighbor :
>>>> sourceNeighbors) {
>>>> >                                           sendMessage(neighbor,
>>>> sourceNeighbors);
>>>> >                                           System.out.println("Vertex
>>>> " + vertex.getId() + " sends " +
>>>> > sourceNeighbors + " to " + neighbor);
>>>> >                                   }
>>>> >
>>>> >                           } else if (getSuperstep() == 2) {
>>>> >                                   int total = vertex.getNumEdges();
>>>> >                                   int possibleCombos = total * (total
>>>> - 1);
>>>> >                                   int mutualNeighbors = 0;
>>>> >                                   double coefficient = 0;
>>>> >                                   int count = 1;
>>>> >
>>>> >                                   System.out.println("\nSUPERSTEP
>>>> 2\n------------------------------");
>>>> >
>>>> >                                   for (IntArrayListWritable message
:
>>>> messages) {
>>>> >                                           System.out.println("Message
>>>> " + count + " contains: " + message);
>>>> >                                           mutualNeighbors +=
>>>> message.get(0).get();
>>>> >                                           count++;
>>>> >                                   }
>>>> >
>>>> >                                   if (possibleCombos > 0) {
>>>> >                                           coefficient =
>>>> (double)mutualNeighbors / possibleCombos;
>>>> >                                           vertex.setValue(new
>>>> DoubleWritable(coefficient));
>>>> >                                   }
>>>> >
>>>> >                           }
>>>> >                   } else {
>>>> >                           vertex.setValue(new DoubleWritable(0));
>>>> >
>>>> >                           if (getSuperstep() == 1) {
>>>> >                                   int mutualNeighbors = 0;
>>>> >
>>>> >                                   System.out.println("\nSUPERSTEP
>>>> 1\n------------------------------");
>>>> >                                   IntArrayListWritable
>>>> sourceNeighbors = messages.iterator().next();
>>>> >                                   System.out.println("Vertex " +
>>>> vertex.getId() + " receives " +
>>>> > sourceNeighbors);
>>>> >
>>>> >                                   for (Edge<IntWritable,
>>>> NullWritable> edge : vertex.getEdges()) {
>>>> >                                           IntWritable targetVertex =
>>>> new
>>>> > IntWritable(edge.getTargetVertexId().get());
>>>> >                                           if
>>>> (sourceNeighbors.contains(targetVertex)) mutualNeighbors++;
>>>> >                                   }
>>>> >
>>>> >                                   IntArrayListWritable neighborCount
>>>> = new IntArrayListWritable();
>>>> >                                   neighborCount.add(new
>>>> IntWritable(mutualNeighbors));
>>>> >
>>>> >                                   sendMessage(new
>>>> IntWritable(SOURCE_ID.get(getConf())), neighborCount);
>>>> >                                   System.out.println("Vertex " +
>>>> vertex.getId() + " sends " +
>>>> > neighborCount + " to Vertex " + SOURCE_ID.get(getConf()));
>>>> >
>>>> >                           }
>>>> >                   }
>>>> >
>>>> >                 vertex.voteToHalt();
>>>> >           }
>>>> >
>>>> >
>>>> > On Fri, Aug 9, 2013 at 10:15 PM, Claudio Martella
>>>> > <claudio.martella@gmail.com> wrote:
>>>> >> you'd have to show us the code in the compute method. your problem
>>>> might be
>>>> >> caused by object reusal.
>>>> >>
>>>> >>
>>>> >> On Fri, Aug 9, 2013 at 9:05 PM, Kyle Orlando <
>>>> kyle.r.orlando@gmail.com>
>>>> >> wrote:
>>>> >>>
>>>> >>> Hello,
>>>> >>>
>>>> >>> I am trying to write code to compute the local clustering
>>>> coefficient
>>>> >>> of a vertex/some vertices, and to do this I send a message that
>>>> >>> contains a list of the source's neighbors to each of its neighbors.
>>>> >>> This is in, of course, an IntArrayListWritable.  I check the
list
>>>> that
>>>> >>> I am sending before invoking sendMessage(), and it appears to
be
>>>> >>> correct. However, when I use message.iterator.next() or something
>>>> >>> similar, the IntArrayListWritable objects seem to repeat or
>>>> something.
>>>> >>> I did some logging, and here was the output:
>>>> >>>
>>>> >>> SUPERSTEP 0
>>>> >>> ------------------------------
>>>> >>> Vertex 1 sends [2, 3, 4] to Vertex 2
>>>> >>> Vertex 1 sends [2, 3, 4] to Vertex 3
>>>> >>> Vertex 1 sends [2, 3, 4] to Vertex 4
>>>> >>>
>>>> >>> SUPERSTEP 1
>>>> >>> ------------------------------
>>>> >>> Vertex 2 receives [2, 3, 4]
>>>> >>> Vertex 2 sends [1] to Vertex 1
>>>> >>>
>>>> >>> SUPERSTEP 1
>>>> >>> ------------------------------
>>>> >>> Vertex 3 receives [2, 3, 4, 2, 3, 4]
>>>> >>> Vertex 3 sends [1] to Vertex 1
>>>> >>>
>>>> >>> SUPERSTEP 1
>>>> >>> ------------------------------
>>>> >>> Vertex 4 receives [2, 3, 4, 2, 3, 4, 2, 3, 4]
>>>> >>> Vertex 4 sends [0] to Vertex 1
>>>> >>>
>>>> >>> SUPERSTEP 2
>>>> >>> ------------------------------
>>>> >>> Message 1 contains: [1]
>>>> >>> Message 2 contains: [1, 1, 1]
>>>> >>> Message 3 contains: [1, 1, 1, 1, 1, 0]
>>>> >>>
>>>> >>> What is happening?
>>>> >>>
>>>> >>> --
>>>> >>> Kyle Orlando
>>>> >>> Computer Engineering Major
>>>> >>> University of Maryland
>>>> >>
>>>> >>
>>>> >>
>>>> >>
>>>> >> --
>>>> >>    Claudio Martella
>>>> >>    claudio.martella@gmail.com
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > Kyle Orlando
>>>> > Computer Engineering Major
>>>> > University of Maryland
>>>>
>>>>
>>>>
>>>> --
>>>> Kyle Orlando
>>>> Computer Engineering Major
>>>> University of Maryland
>>>>
>>>
>>>
>>>
>>> --
>>>    Claudio Martella
>>>    claudio.martella@gmail.com
>>>
>>
>>
>>
>> --
>> Kyle Orlando
>> Computer Engineering Major
>> University of Maryland
>>
>
>
>
> --
>    Claudio Martella
>    claudio.martella@gmail.com
>



-- 
Kyle Orlando
Computer Engineering Major
University of Maryland

Mime
View raw message