hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thomas Jungblut <thomas.jungb...@googlemail.com>
Subject Re: counters and supersteps
Date Wed, 07 Dec 2011 10:00:11 GMT
Wait a second please:
The localrunner is going to piggyback to the client console with a field
called "superStepCount".
This is just getting updated when a new taskstatus comes in, and if it is
greater than superStepCount it updates the variable.
There is no summation.

Are you speaking about the LocalBSPRunner Tommaso?

2011/12/7 Edward J. Yoon <edwardyoon@apache.org>

> Counters seems synchronized. Below small patch will fix this problem.
>
> Index:
> src/main/java/org/apache/hama/bsp/LocalBSPRunner.java===================================================================---
> src/main/java/org/apache/hama/bsp/LocalBSPRunner.java   (revision
> 1211275)+++ src/main/java/org/apache/hama/bsp/LocalBSPRunner.java
> (working
> copy)@@ -213,7 +213,6 @@     private int id;     private BSP bsp;
> private RawSplit[] splits;-    private static final Counters counters
> = new Counters();      public BSPRunner(Configuration conf, BSPJob
> job, int id, RawSplit[] splits) {       super();@@ -244,7 +243,7 @@
>     BSPPeerImpl peer = new BSPPeerImpl(job, conf, new TaskAttemptID(
>         new TaskID(job.getJobID(), id), id), new LocalUmbilical(),
> id,-          splitname, realBytes, counters);+          splitname,
> realBytes, new Counters());        bsp.setConf(conf);       try {
>
> On Wed, Dec 7, 2011 at 6:05 PM, Tommaso Teofili
> <tommaso.teofili@gmail.com> wrote:
> > for an example take a look at the RandBenchTest which runs RandBench with
> > 200 supersteps  and it ends up with :
> >
> > ...
> > 11/12/07 09:59:07 INFO examples.RandBench$RandBSP: local:19 to local:7 :
> 10
> >
> > 11/12/07 09:59:07 INFO examples.RandBench$RandBSP: local:19 to local:7 :
> 10
> >
> > 11/12/07 09:59:07 INFO examples.RandBench$RandBSP: local:19 to local:7 :
> 10
> >
> > 11/12/07 09:59:08 INFO bsp.BSPJobClient: Current supersteps number: 4000
> >
> > 11/12/07 09:59:08 INFO bsp.BSPJobClient: The total number of supersteps:
> > 4000
> >
> >
> > which is exactly 200*cluster.getMaxTasks() on my laptop.
> >
> > Tommaso
> >
> >
> >
> >
> > 2011/12/7 Thomas Jungblut <thomas.jungblut@googlemail.com>
> >
> >> We are so buggy :D
> >>
> >> 2011/12/7 Edward J. Yoon <edwardyoon@apache.org>
> >>
> >> > OMG!
> >> >
> >> > On Wed, Dec 7, 2011 at 5:31 PM, Tommaso Teofili
> >> > <tommaso.teofili@gmail.com> wrote:
> >> > > Hi all,
> >> > >
> >> > > after the recent updates in trunk it seems to me the count of the
> >> number
> >> > of
> >> > > supersteps is not working correctly anymore.
> >> > > I have a sample algorithm with 3 supersteps split to 200 tasks and
I
> >> got
> >> > a
> >> > > count of 600 supersteps in the logger.
> >> > >
> >> > > Anyone experiencing the same issue?
> >> > > Tommaso
> >> >
> >> >
> >> >
> >> > --
> >> > Best Regards, Edward J. Yoon
> >> > @eddieyoon
> >> >
> >>
> >>
> >>
> >> --
> >> Thomas Jungblut
> >> Berlin <thomas.jungblut@gmail.com>
> >>
>
>
>
> --
> Best Regards, Edward J. Yoon
> @eddieyoon
>



-- 
Thomas Jungblut
Berlin <thomas.jungblut@gmail.com>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message