hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lee fei <chirs...@gmail.com>
Subject Re: Can hama monitor the num of alive nodes or num of tasks in last superstep?
Date Fri, 30 Nov 2012 18:22:56 GMT
If I use an aggregator, every lived vertex will directly access and modify
 the aggregator in RAM , and during the sync phase, the aggregator in every
groomserver will do a reduce operation. Did I got it? and if so, there are
less communication cost compare to send to message to a master vertex.

On Sat, Dec 1, 2012 at 2:09 AM, Thomas Jungblut
<thomas.jungblut@gmail.com>wrote:

> Yes, that is exactly the way aggregators should be used ;)
>
> 2012/11/30 lee fei <chirsfei@gmail.com>
>
> > Hi Thomas,
> >   Beautiful solution! I am wonder if I maintain an aggregator which in
> > every superstep every lived vertexes can add 1 to it, and the master
> vertex
> > check this aggregator in next superstep, if the increment  is zero, there
> > mean no lived vertexes in last superstep.
> > Due to the sum of lived vertexes in one superstep maybe vary large,
> > maintain an aggregator can be a more efficient way and save
> communication's
> > cost?
> >
> > On Sat, Dec 1, 2012 at 1:31 AM, Thomas Jungblut
> > <thomas.jungblut@gmail.com>wrote:
> >
> > > You can get the number of tasks/peers via the BSPPeer object that you
> can
> > > obtain inside your vertex via getPeer().getNumPeers().
> > > Suraj already told you the overall procedure, but I would consult you
> to
> > > use the state machine of a vertex. So computation stops if no message
> has
> > > been send and no vertex is active any more.
> > > In case of a graph you can always send a message to a "master" vertex,
> > e.G.
> > > with the ID of 0. Therefore you can send a message to a vertex key with
> > the
> > > method sendMessage(V destinationVertexID, M msg).
> > >
> > > Good luck!
> > >
> > >
> > > 2012/11/30 Du Zhengjun <zhengjunbase@gmail.com>
> > >
> > > > Yes you are right. I am planning to use the vertex API, and the
> 'node'
> > > > imply a vertex in a graph.
> > > >
> > > > Sorry for my ambiguous express.
> > > >
> > > > Thank you for  your quick reply :)
> > > >
> > > >
> > > > On Sat, Dec 1, 2012 at 12:47 AM, Thomas Jungblut
> > > > <thomas.jungblut@gmail.com>wrote:
> > > >
> > > > > Wait a second, can you tell us what kind of API are you using? I
> > assume
> > > > you
> > > > > use the vertex API in the graph package right?
> > > > >
> > > > > 2012/11/30 Suraj Menon <surajsmenon@apache.org>
> > > > >
> > > > > > Hi lee,
> > > > > >
> > > > > > I am assuming 'node' here implies BSP peer.
> > > > > >
> > > > > > You would have this feature once I get HAMA-652 (
> > > > > > https://issues.apache.org/jira/browse/HAMA-652) working. For
now
> > you
> > > > can
> > > > > > do
> > > > > > something similar to what is done is graph processing.
> > > > > > Keep sending a vote to halt(message) to every other peers, when
> you
> > > > want
> > > > > to
> > > > > > halt, until in the next superstep you find that the number of
> votes
> > > ==
> > > > > > number of peers. This is a limitation in the current
> > implementation.
> > > We
> > > > > > need all the peers running together and peer.sync won't proceed
> > until
> > > > all
> > > > > > the peers enter and leave the synchronization barrier. Please
> check
> > > the
> > > > > > progress on HAMA-652, it should get fixed in near future.
> > > > > >
> > > > > > Thanks,
> > > > > > Suraj
> > > > > >
> > > > > > On Fri, Nov 30, 2012 at 11:05 AM, lee fei <chirsfei@gmail.com>
> > > wrote:
> > > > > >
> > > > > > > Thank you for you help~
> > > > > > >
> > > > > > >    I need to know if there is any node active in last superstep
> > to
> > > > > > > decide which the sub program should to process in the compute(
> )
> > > > > > function.
> > > > > > > So if I can get the sum of alive nodes(not halted) in last
> > > superstep
> > > > or
> > > > > > the
> > > > > > > sum of messages sent in last superstep, I can easily make
the
> > > > decision.
> > > > > > >
> > > > > > > Is there a way to do that?
> > > > > > >
> > > > > > > On Fri, Nov 30, 2012 at 8:30 PM, Edward <edward@udanax.org>
> > wrote:
> > > > > > >
> > > > > > > > It should be always equal to launched tasks num. Instead,
we
> > can
> > > > > count
> > > > > > > > failure tasks num. Why do you need?
> > > > > > > >
> > > > > > > > Sent from my iPhone
> > > > > > > >
> > > > > > > > On Nov 30, 2012, at 6:29 PM, lee fei <chirsfei@gmail.com>
> > wrote:
> > > > > > > >
> > > > > > > > > Hi,
> > > > > > > > >
> > > > > > > > >  Did hama have record the sum of alive nodes
or sum of
> tasks
> > in
> > > > > last
> > > > > > > > > superstep? Is there a way to get this kinds of
values?
> > > > > > > > >
> > > > > > > > > Thank you for you attention:D
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message