hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anthony Urso <antho...@cs.ucla.edu>
Subject Re: Dynamic counters in Hadoop M/R jobs...
Date Tue, 20 Jul 2010 00:10:39 GMT
The new Mapper class passes a Map.Context object to the map() method.
>From this you can get a StatusReporter object  which  can produce a
named Counter object and increment it.

http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapreduce/Mapper.Context.html
http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapreduce/StatusReporter.html
http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapreduce/Counter.html

Alternatively, the old APIs are supposed to be temporarily
de-deprecated in the next stable release, so you can use them as
described by that book.

On Mon, Jul 19, 2010 at 11:22 AM, Michael Segel
<michael_segel@hotmail.com> wrote:
>
> Hi,
>
> In looking at "The Definitive Guide"  pgs 211-218 looking at the documentation for counters
have a question about how to use the
>
> dynamic (String, String) methods to create/access counters.
>
> Looking at the IdentityTableMap class, the following map() method exists():
> public void map(ImmutableBytesWritable key,
>                RowResult value,
>                org.apache.hadoop.mapred.OutputCollector<ImmutableBytesWritable,RowResult>
output,
>                org.apache.hadoop.mapred.Reporter reporter)
>         throws IOExceptionThis method is deprecated in 20.5
>
> So how can you get the Reporter so you can use the 'reporter.incrCounter(String group,String
counter, long amount)'
> method. (See pg 214 code fragment)
>
> I must be missing something.
>
> Thx
>
> -Mike
>
>
>
> _________________________________________________________________
> The New Busy is not the too busy. Combine all your e-mail accounts with Hotmail.
> http://www.windowslive.com/campaign/thenewbusy?tile=multiaccount&ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_4

Mime
View raw message