hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From JAX <jayunit...@gmail.com>
Subject Re: Accessing global Counters
Date Fri, 20 Apr 2012 23:51:45 GMT
No reducers can't access mapper counters.
---> maybe theres a way to intermediately put counters in the distributed cache???

Jay Vyas 
MMSB
UCHC

On Apr 20, 2012, at 1:24 PM, Robert Evans <evans@yahoo-inc.com> wrote:

> There was a discussion about this several months ago
> 
> http://mail-archives.apache.org/mod_mbox/hadoop-mapreduce-user/201112.mbox/%3CCADYHM8xiw8_bF=ZQe-bAgdFZ6r3tOb0AOF9ViOZgtZEQGkPZVA@mail.gmail.com%3E
> 
> The conclusion is that if you want to read them from the reducer you are going to have
to do something special until someone finds time to implement it as part of.
> 
> https://issues.apache.org/jira/browse/MAPREDUCE-3520
> 
> --Bobby Evans
> 
> 
> On 4/20/12 11:36 AM, "Amith D K" <amithdk@huawei.com> wrote:
> 
> Yes U can use user defined counter as Jagat suggeted.
> 
> Counter can be enum as Jagat described or any string which are called dynamic counters.
> 
> It is easier to use Enum counter than dynamic counters, finally it depends on your use
case :)
> 
> Amith
> ________________________________________
> From: Jagat [jagatsingh@gmail.com]
> Sent: Saturday, April 21, 2012 12:25 AM
> To: common-user@hadoop.apache.org
> Subject: Re: Accessing global Counters
> 
> Hi
> 
> You can create your own counters like
> 
> enum CountFruits {
> Apple,
> Mango,
> Banana
> }
> 
> 
> And in your mapper class when you see condition to increment , you can use
> Reporter incrCounter method to do the same.
> 
> http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapred/Reporter.html#incrCounter(java.lang.Enum,%20long)
> 
> e.g
> // I saw Apple increment it by one
> reporter.incrCounter(CountFruits.Apple,1);
> 
> Now you can access them using job.getCounters
> 
> http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/mapreduce/Job.html#getCounters()
> 
> Hope this helps
> 
> Regards,
> 
> Jagat Singh
> 
> 
> On Fri, Apr 20, 2012 at 9:43 PM, Gayatri Rao <rgayatri1@gmail.com> wrote:
> 
>> Hi All,
>> 
>> Is there a way for me to set global counters in Mapper and access them from
>> reducer?
>> Could you suggest how I can acheve this?
>> 
>> Thanks
>> Gayatri
>> 
> 

Mime
View raw message