hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nan Zhu <zhunans...@gmail.com>
Subject Re: Access Hadoop Counters
Date Tue, 30 Oct 2012 15:51:51 GMT
Hi, Harsh 

thanks for your reply, 

yes, I'm modifying the source code of MR, 

I have dig it out, just as jip.getMapCounters().getCounter(FileInputFormat.Counter.BYTES_READ);

Thx 

-- 
Nan Zhu
School of Computer Science,
McGill University



On Tuesday, October 30, 2012 at 12:51 AM, Harsh J wrote:

> Hi,
> 
> Are you attempting to modify MR sources? JobInProgress is a system
> class, not to be used by users directly. If you seek MR project-side
> development help, its better to reach the mapreduce-dev@ lists
> instead.
> 
> That said, if you want the file/hdfs bytes counters, you can access
> them via the RunningJob or Job APIs directly (just give the group and
> counter names).
> 
> On Tue, Oct 30, 2012 at 9:19 AM, Nan Zhu <zhunansjtu@gmail.com (mailto:zhunansjtu@gmail.com)>
wrote:
> > Hi, all
> > 
> > I noticed that in FileInputFormat and FileOutputFormat classes, there are
> > counters(Counter's name?) for input and output data size,
> > 
> > my question is that given a JobInProgress object, e.g. jip, after I get the
> > corresponding object by
> > 
> > FileInputFormat fif = (FileInputFormat) jip.getConf().getInputFormat();
> > FileOutputFormat fof = (FileOutputFormat)
> > jip.getConf().getOutputputFormat();
> > 
> > How can I access the values of the counters?
> > 
> > Best,
> > 
> > --
> > Nan Zhu
> > School of Computer Science,
> > McGill University
> > 
> 
> 
> 
> 
> -- 
> Harsh J
> 
> 



Mime
View raw message