incubator-hama-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Thomas Jungblut <thomas.jungb...@googlemail.com>
Subject Re: counters and supersteps
Date Wed, 07 Dec 2011 10:01:54 GMT
Ah sorry, but since this is a shared object, this will be incremented by
all tasks.
So your fix will fix it ;)
Sorry.

2011/12/7 Thomas Jungblut <thomas.jungblut@googlemail.com>

> Wait a second please:
> The localrunner is going to piggyback to the client console with a field
> called "superStepCount".
> This is just getting updated when a new taskstatus comes in, and if it is
> greater than superStepCount it updates the variable.
> There is no summation.
>
> Are you speaking about the LocalBSPRunner Tommaso?
>
>
> 2011/12/7 Edward J. Yoon <edwardyoon@apache.org>
>
>> Counters seems synchronized. Below small patch will fix this problem.
>>
>> Index:
>> src/main/java/org/apache/hama/bsp/LocalBSPRunner.java===================================================================---
>> src/main/java/org/apache/hama/bsp/LocalBSPRunner.java   (revision
>> 1211275)+++ src/main/java/org/apache/hama/bsp/LocalBSPRunner.java
>> (working
>> copy)@@ -213,7 +213,6 @@     private int id;     private BSP bsp;
>> private RawSplit[] splits;-    private static final Counters counters
>> = new Counters();      public BSPRunner(Configuration conf, BSPJob
>> job, int id, RawSplit[] splits) {       super();@@ -244,7 +243,7 @@
>>     BSPPeerImpl peer = new BSPPeerImpl(job, conf, new TaskAttemptID(
>>         new TaskID(job.getJobID(), id), id), new LocalUmbilical(),
>> id,-          splitname, realBytes, counters);+          splitname,
>> realBytes, new Counters());        bsp.setConf(conf);       try {
>>
>> On Wed, Dec 7, 2011 at 6:05 PM, Tommaso Teofili
>> <tommaso.teofili@gmail.com> wrote:
>> > for an example take a look at the RandBenchTest which runs RandBench
>> with
>> > 200 supersteps  and it ends up with :
>> >
>> > ...
>> > 11/12/07 09:59:07 INFO examples.RandBench$RandBSP: local:19 to local:7
>> : 10
>> >
>> > 11/12/07 09:59:07 INFO examples.RandBench$RandBSP: local:19 to local:7
>> : 10
>> >
>> > 11/12/07 09:59:07 INFO examples.RandBench$RandBSP: local:19 to local:7
>> : 10
>> >
>> > 11/12/07 09:59:08 INFO bsp.BSPJobClient: Current supersteps number: 4000
>> >
>> > 11/12/07 09:59:08 INFO bsp.BSPJobClient: The total number of supersteps:
>> > 4000
>> >
>> >
>> > which is exactly 200*cluster.getMaxTasks() on my laptop.
>> >
>> > Tommaso
>> >
>> >
>> >
>> >
>> > 2011/12/7 Thomas Jungblut <thomas.jungblut@googlemail.com>
>> >
>> >> We are so buggy :D
>> >>
>> >> 2011/12/7 Edward J. Yoon <edwardyoon@apache.org>
>> >>
>> >> > OMG!
>> >> >
>> >> > On Wed, Dec 7, 2011 at 5:31 PM, Tommaso Teofili
>> >> > <tommaso.teofili@gmail.com> wrote:
>> >> > > Hi all,
>> >> > >
>> >> > > after the recent updates in trunk it seems to me the count of
the
>> >> number
>> >> > of
>> >> > > supersteps is not working correctly anymore.
>> >> > > I have a sample algorithm with 3 supersteps split to 200 tasks
and
>> I
>> >> got
>> >> > a
>> >> > > count of 600 supersteps in the logger.
>> >> > >
>> >> > > Anyone experiencing the same issue?
>> >> > > Tommaso
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Best Regards, Edward J. Yoon
>> >> > @eddieyoon
>> >> >
>> >>
>> >>
>> >>
>> >> --
>> >> Thomas Jungblut
>> >> Berlin <thomas.jungblut@gmail.com>
>> >>
>>
>>
>>
>> --
>> Best Regards, Edward J. Yoon
>> @eddieyoon
>>
>
>
>
> --
> Thomas Jungblut
> Berlin <thomas.jungblut@gmail.com>
>



-- 
Thomas Jungblut
Berlin <thomas.jungblut@gmail.com>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message