incubator-hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tommaso Teofili <tommaso.teof...@gmail.com>
Subject Re: counters and supersteps
Date Wed, 07 Dec 2011 10:07:29 GMT
right, it looks like Edward's patch fixes it.

...

11/12/07 11:06:06 INFO examples.RandBench$RandBSP: local:18 to local:10 : 10

11/12/07 11:06:06 INFO examples.RandBench$RandBSP: local:18 to local:10 : 10

11/12/07 11:06:06 INFO examples.RandBench$RandBSP: local:18 to local:10 : 10

11/12/07 11:06:08 INFO bsp.BSPJobClient: Current supersteps number: 200

11/12/07 11:06:08 INFO bsp.BSPJobClient: The total number of supersteps: 200

Job Finished in 6.38 seconds
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.56 sec


Thanks,
Tommaso


2011/12/7 Thomas Jungblut <thomas.jungblut@googlemail.com>

> Ah sorry, but since this is a shared object, this will be incremented by
> all tasks.
> So your fix will fix it ;)
> Sorry.
>
> 2011/12/7 Thomas Jungblut <thomas.jungblut@googlemail.com>
>
> > Wait a second please:
> > The localrunner is going to piggyback to the client console with a field
> > called "superStepCount".
> > This is just getting updated when a new taskstatus comes in, and if it is
> > greater than superStepCount it updates the variable.
> > There is no summation.
> >
> > Are you speaking about the LocalBSPRunner Tommaso?
> >
> >
> > 2011/12/7 Edward J. Yoon <edwardyoon@apache.org>
> >
> >> Counters seems synchronized. Below small patch will fix this problem.
> >>
> >> Index:
> >>
> src/main/java/org/apache/hama/bsp/LocalBSPRunner.java===================================================================---
> >> src/main/java/org/apache/hama/bsp/LocalBSPRunner.java   (revision
> >> 1211275)+++ src/main/java/org/apache/hama/bsp/LocalBSPRunner.java
> >> (working
> >> copy)@@ -213,7 +213,6 @@     private int id;     private BSP bsp;
> >> private RawSplit[] splits;-    private static final Counters counters
> >> = new Counters();      public BSPRunner(Configuration conf, BSPJob
> >> job, int id, RawSplit[] splits) {       super();@@ -244,7 +243,7 @@
> >>     BSPPeerImpl peer = new BSPPeerImpl(job, conf, new TaskAttemptID(
> >>         new TaskID(job.getJobID(), id), id), new LocalUmbilical(),
> >> id,-          splitname, realBytes, counters);+          splitname,
> >> realBytes, new Counters());        bsp.setConf(conf);       try {
> >>
> >> On Wed, Dec 7, 2011 at 6:05 PM, Tommaso Teofili
> >> <tommaso.teofili@gmail.com> wrote:
> >> > for an example take a look at the RandBenchTest which runs RandBench
> >> with
> >> > 200 supersteps  and it ends up with :
> >> >
> >> > ...
> >> > 11/12/07 09:59:07 INFO examples.RandBench$RandBSP: local:19 to local:7
> >> : 10
> >> >
> >> > 11/12/07 09:59:07 INFO examples.RandBench$RandBSP: local:19 to local:7
> >> : 10
> >> >
> >> > 11/12/07 09:59:07 INFO examples.RandBench$RandBSP: local:19 to local:7
> >> : 10
> >> >
> >> > 11/12/07 09:59:08 INFO bsp.BSPJobClient: Current supersteps number:
> 4000
> >> >
> >> > 11/12/07 09:59:08 INFO bsp.BSPJobClient: The total number of
> supersteps:
> >> > 4000
> >> >
> >> >
> >> > which is exactly 200*cluster.getMaxTasks() on my laptop.
> >> >
> >> > Tommaso
> >> >
> >> >
> >> >
> >> >
> >> > 2011/12/7 Thomas Jungblut <thomas.jungblut@googlemail.com>
> >> >
> >> >> We are so buggy :D
> >> >>
> >> >> 2011/12/7 Edward J. Yoon <edwardyoon@apache.org>
> >> >>
> >> >> > OMG!
> >> >> >
> >> >> > On Wed, Dec 7, 2011 at 5:31 PM, Tommaso Teofili
> >> >> > <tommaso.teofili@gmail.com> wrote:
> >> >> > > Hi all,
> >> >> > >
> >> >> > > after the recent updates in trunk it seems to me the count
of the
> >> >> number
> >> >> > of
> >> >> > > supersteps is not working correctly anymore.
> >> >> > > I have a sample algorithm with 3 supersteps split to 200
tasks
> and
> >> I
> >> >> got
> >> >> > a
> >> >> > > count of 600 supersteps in the logger.
> >> >> > >
> >> >> > > Anyone experiencing the same issue?
> >> >> > > Tommaso
> >> >> >
> >> >> >
> >> >> >
> >> >> > --
> >> >> > Best Regards, Edward J. Yoon
> >> >> > @eddieyoon
> >> >> >
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Thomas Jungblut
> >> >> Berlin <thomas.jungblut@gmail.com>
> >> >>
> >>
> >>
> >>
> >> --
> >> Best Regards, Edward J. Yoon
> >> @eddieyoon
> >>
> >
> >
> >
> > --
> > Thomas Jungblut
> > Berlin <thomas.jungblut@gmail.com>
> >
>
>
>
> --
> Thomas Jungblut
> Berlin <thomas.jungblut@gmail.com>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message