hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Devaraj k <devara...@huawei.com>
Subject RE: Reading the value of a counter
Date Mon, 26 Sep 2011 19:34:00 GMT
Hi  Steve,

public void map(Text key, Text val, OutputCollector<Text, Text> output,
      Reporter reporter) throws IOException {
      reporter.incrCounter(RecordCounters.TYPE_A, 1);

  You can increment the counter based on your business logic in mappers or reducers as shown

If you launch your job with JobClient.runJob(), the information printed on stdout when the
job completes will contain the values of all the counters. Both runJob() and submitJob() will
return a RunningJob object that refers to the running job. The RunningJob.getCounters() method
will return a Counters object that contains the values of all the counters so that you can
query them programmatically. The Counters.getCounter(Enum key) method returns the value of
a particular counter.

From: Steve Lewis [lordjoe2000@gmail.com]
Sent: Tuesday, September 27, 2011 12:47 AM
To: mapreduce-user
Subject: Reading the value of a counter

 Assume I have a class which extends Configured implements Tool

Assume I run the job using the run method.

Within my job I increment a counter. When I finish the run I want the final value of the counter.

in the code which calls the run method (which in my system is running outside of the cluster
) how to I get the final value of
my counter after the job is complete. Assume I run 0.20? . A code sampel would be very usefil

Also a job prints to stderr a report of Reduce input groups, Combine output records,Map input

11/09/26 11:15:42 INFO mapred.JobClient:     Reduce input groups=26308
11/09/26 11:15:42 INFO mapred.JobClient:     Combine output records=0
11/09/26 11:15:42 INFO mapred.JobClient:     Map input records=12606891

Is there a way for my code to access these numbers after the job has completed?

Steven M. Lewis PhD
4221 105th Ave NE
Kirkland, WA 98033
206-384-1340 (cell)
Skype lordjoe_com

View raw message