hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nan Zhu <zhunans...@gmail.com>
Subject Re: response time measurement
Date Sat, 17 Nov 2012 14:19:30 GMT
yes, you can do that 

and actually there some counters implementations, 

  public static enum Counter {  
    NUM_FAILED_MAPS, 
    NUM_FAILED_REDUCES,
    TOTAL_LAUNCHED_MAPS,
    TOTAL_LAUNCHED_REDUCES,
    OTHER_LOCAL_MAPS,
    DATA_LOCAL_MAPS,
    RACK_LOCAL_MAPS,
    SLOTS_MILLIS_MAPS,
    SLOTS_MILLIS_REDUCES,
    FALLOW_SLOTS_MILLIS_MAPS,
    FALLOW_SLOTS_MILLIS_REDUCES,
    MAPS_RESPONSE_TIME,
    REDUCE_START_TIME,
    REDUCE_RESPONSE_TIME
  }

Best, 

-- 
Nan Zhu
School of Computer Science,
McGill University



On Saturday, 17 November, 2012 at 9:12 AM, Samaneh Shokuhi wrote:

> Hi,
> I checked the JobInProgress.java class, i see as you said there is no
> counter as "Total Job Time" there, but there are two different variables
> ,one called "finishTime" another called "startTime" .I guess if i add
> (finishTime - startTime) to logJobSummary method that may give us the
> "Total Job Time" .Am i right ?
> 
> Best,
> Samaneh
> 
> 
> 
> 
> On Tue, Nov 13, 2012 at 1:41 AM, Nan Zhu <zhunansjtu@gmail.com (mailto:zhunansjtu@gmail.com)>
wrote:
> 
> > There is no such a counter in default Hadoop implementation
> > 
> > you can instrument your own code in JobInProgress.java to calculate it,
> > 
> > there is also an implemented counter in JobInProgress, MAP_SLOTS_MILLS,
> > considering both the map tasks occupying multiple slots, but this value
> > also increase when the task is failed,
> > 
> > Best,
> > 
> > --
> > Nan Zhu
> > School of Computer Science,
> > McGill University
> > 
> > 
> > On Monday, 12 November, 2012 at 6:14 PM, Samaneh Shokuhi wrote:
> > > Hi,
> > > I need to do some experience on hadoop source code.
> > > i have modified part of it in MapTask.java class and need to compare the
> > > response time for mapper /job before and after modification .Is it
> > > reported somewhere to see that?
> > > 
> > > 
> > > Cheers,
> > > Samaneh
> > > 
> > 
> > 
> 
> 
> 



Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message