incubator-giraph-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jake Mannix <jake.man...@gmail.com>
Subject Re: Message processing
Date Thu, 08 Sep 2011 21:19:46 GMT
On Thu, Sep 8, 2011 at 2:01 PM, Avery Ching <aching@apache.org> wrote:

>  Yes, the messages are delivered to inMessages (map of id -> list of
> messages).  Then they are transferred to the vertices just before the
> computation on a worker.
>

So just to make sure I understand correctly (I think I do, but reasoning
about distributed async operations makes my head hurt a bit), I'm going to
sound dumb and repeat this back to you: they show up in an asynchronous
manner into BasicRPCCommunications, where they get buffered up into the
inMessages map (one list for each vertex on the local node), where the
GraphMapper then loops through and gives them to each Vertex before they do
their compute() calls?


On 9/8/11 1:55 PM, Jake Mannix wrote:
>
> So in particular, each GraphMapper has a BasicRPCCommunications object (and
> a CommunicationsInterface proxy object for each of the other nodes), and
> when other nodes call putMsg(vertexId, message) on their respective proxies,
> it shows up in the target vertex's BasicRPCCommunions putMsg() call, is that
> right?
>
> On Thu, Sep 8, 2011 at 1:38 PM, Avery Ching <aching@apache.org> wrote:
>
>> If I'm understanding your question correctly, the iterator is created in
>> GraphMapper, just before compute() is called.  Messages are tranferred to
>> the vertices prior to the compute portion superstep.
>>
>> Avery
>>
>>
>> On 9/8/11 12:59 PM, Jake Mannix wrote:
>>
>>> Question about the message passing API:
>>>
>>>  BaseVertex#sendMsg(I id, M msg)
>>>
>>> sends messages.  And
>>>
>>>  BaseVertex#compute(Iterator<M> msgIterator)
>>>
>>> deals with them in a big iterator provided by the framework.  My question
>>> is where the Iterator of messages is created?  If I wanted to say, subvert
>>> the processing to avoid buffering them up into one big List, and instead
>>> handle them as they come in, where would I look in the code to see where
>>> it's buffered up?
>>>
>>>  -jake
>>>
>>>
>>>
>>
>
>

Mime
View raw message