hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Segel <michael_se...@hotmail.com>
Subject Re: Increment Counters in HBase during MapReduce
Date Tue, 19 Jun 2012 12:45:53 GMT
Sure, why not? 

You can always open a connection to the counter table in your Mapper.setup() method and then
increment the counters within the Mapper.map() method.

Your update of the counter is an artifact and not the output of the Mapper.map() method.


On Jun 18, 2012, at 7:49 PM, Sid Kumar wrote:

> Hi everyone,
> 
>    I have a use case in HBase that I was wondering if someone may have
> stumbled upon. I am maintaining an ad impressions table with columns that
> are counters for certain metrics. I started using the incrementColumnValue
> method part of the HTable API to update these metrics and that works great.
>    I was wondering if this function could be used from a MapReduce job.
> The TableOutputFormat supports only Delete and Put operations. Using the
> Incremental counters saves me from doing any aggregations in my Map Reduce
> code. Ideally i would like to just call this function in my mapper and
> wouldn't even need a Reducer.
>    Has anyone run into this use case? I would also love to know if there
> are any better alternatives of solving this too. Any info would be great.
> 
> Thanks
> Sid


Mime
View raw message