hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thomas Jungblut <thomas.jungb...@gmail.com>
Subject Re: Can hama monitor the num of alive nodes or num of tasks in last superstep?
Date Fri, 30 Nov 2012 18:26:26 GMT
Exactly. A aggregator lives in each task, they will be reduced on a master
task by the framework.

2012/11/30 lee fei <chirsfei@gmail.com>

> If I use an aggregator, every lived vertex will directly access and modify
>  the aggregator in RAM , and during the sync phase, the aggregator in every
> groomserver will do a reduce operation. Did I got it? and if so, there are
> less communication cost compare to send to message to a master vertex.
>
> On Sat, Dec 1, 2012 at 2:09 AM, Thomas Jungblut
> <thomas.jungblut@gmail.com>wrote:
>
> > Yes, that is exactly the way aggregators should be used ;)
> >
> > 2012/11/30 lee fei <chirsfei@gmail.com>
> >
> > > Hi Thomas,
> > >   Beautiful solution! I am wonder if I maintain an aggregator which in
> > > every superstep every lived vertexes can add 1 to it, and the master
> > vertex
> > > check this aggregator in next superstep, if the increment  is zero,
> there
> > > mean no lived vertexes in last superstep.
> > > Due to the sum of lived vertexes in one superstep maybe vary large,
> > > maintain an aggregator can be a more efficient way and save
> > communication's
> > > cost?
> > >
> > > On Sat, Dec 1, 2012 at 1:31 AM, Thomas Jungblut
> > > <thomas.jungblut@gmail.com>wrote:
> > >
> > > > You can get the number of tasks/peers via the BSPPeer object that you
> > can
> > > > obtain inside your vertex via getPeer().getNumPeers().
> > > > Suraj already told you the overall procedure, but I would consult you
> > to
> > > > use the state machine of a vertex. So computation stops if no message
> > has
> > > > been send and no vertex is active any more.
> > > > In case of a graph you can always send a message to a "master"
> vertex,
> > > e.G.
> > > > with the ID of 0. Therefore you can send a message to a vertex key
> with
> > > the
> > > > method sendMessage(V destinationVertexID, M msg).
> > > >
> > > > Good luck!
> > > >
> > > >
> > > > 2012/11/30 Du Zhengjun <zhengjunbase@gmail.com>
> > > >
> > > > > Yes you are right. I am planning to use the vertex API, and the
> > 'node'
> > > > > imply a vertex in a graph.
> > > > >
> > > > > Sorry for my ambiguous express.
> > > > >
> > > > > Thank you for  your quick reply :)
> > > > >
> > > > >
> > > > > On Sat, Dec 1, 2012 at 12:47 AM, Thomas Jungblut
> > > > > <thomas.jungblut@gmail.com>wrote:
> > > > >
> > > > > > Wait a second, can you tell us what kind of API are you using?
I
> > > assume
> > > > > you
> > > > > > use the vertex API in the graph package right?
> > > > > >
> > > > > > 2012/11/30 Suraj Menon <surajsmenon@apache.org>
> > > > > >
> > > > > > > Hi lee,
> > > > > > >
> > > > > > > I am assuming 'node' here implies BSP peer.
> > > > > > >
> > > > > > > You would have this feature once I get HAMA-652 (
> > > > > > > https://issues.apache.org/jira/browse/HAMA-652) working.
For
> now
> > > you
> > > > > can
> > > > > > > do
> > > > > > > something similar to what is done is graph processing.
> > > > > > > Keep sending a vote to halt(message) to every other peers,
when
> > you
> > > > > want
> > > > > > to
> > > > > > > halt, until in the next superstep you find that the number
of
> > votes
> > > > ==
> > > > > > > number of peers. This is a limitation in the current
> > > implementation.
> > > > We
> > > > > > > need all the peers running together and peer.sync won't
proceed
> > > until
> > > > > all
> > > > > > > the peers enter and leave the synchronization barrier.
Please
> > check
> > > > the
> > > > > > > progress on HAMA-652, it should get fixed in near future.
> > > > > > >
> > > > > > > Thanks,
> > > > > > > Suraj
> > > > > > >
> > > > > > > On Fri, Nov 30, 2012 at 11:05 AM, lee fei <chirsfei@gmail.com>
> > > > wrote:
> > > > > > >
> > > > > > > > Thank you for you help~
> > > > > > > >
> > > > > > > >    I need to know if there is any node active in last
> superstep
> > > to
> > > > > > > > decide which the sub program should to process in
the
> compute(
> > )
> > > > > > > function.
> > > > > > > > So if I can get the sum of alive nodes(not halted)
in last
> > > > superstep
> > > > > or
> > > > > > > the
> > > > > > > > sum of messages sent in last superstep, I can easily
make the
> > > > > decision.
> > > > > > > >
> > > > > > > > Is there a way to do that?
> > > > > > > >
> > > > > > > > On Fri, Nov 30, 2012 at 8:30 PM, Edward <edward@udanax.org>
> > > wrote:
> > > > > > > >
> > > > > > > > > It should be always equal to launched tasks num.
Instead,
> we
> > > can
> > > > > > count
> > > > > > > > > failure tasks num. Why do you need?
> > > > > > > > >
> > > > > > > > > Sent from my iPhone
> > > > > > > > >
> > > > > > > > > On Nov 30, 2012, at 6:29 PM, lee fei <chirsfei@gmail.com>
> > > wrote:
> > > > > > > > >
> > > > > > > > > > Hi,
> > > > > > > > > >
> > > > > > > > > >  Did hama have record the sum of alive nodes
or sum of
> > tasks
> > > in
> > > > > > last
> > > > > > > > > > superstep? Is there a way to get this kinds
of values?
> > > > > > > > > >
> > > > > > > > > > Thank you for you attention:D
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message