hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Daniel Cryans <jdcry...@apache.org>
Subject Re: Increment Counters in HBase during MapReduce
Date Tue, 19 Jun 2012 16:55:47 GMT
This question was answered here already:
http://mail-archives.apache.org/mod_mbox/hbase-user/201101.mbox/%3CAANLkTinnW2d7DMCyFu3ptv1Hu_i3XqK_1pDSgD5NT_Lk@mail.gmail.com%3E

Counters are not idempotent, this can be hard to manage.

J-D

On Mon, Jun 18, 2012 at 5:49 PM, Sid Kumar <sqlsid101@gmail.com> wrote:
> Hi everyone,
>
>    I have a use case in HBase that I was wondering if someone may have
> stumbled upon. I am maintaining an ad impressions table with columns that
> are counters for certain metrics. I started using the incrementColumnValue
> method part of the HTable API to update these metrics and that works great.
>    I was wondering if this function could be used from a MapReduce job.
> The TableOutputFormat supports only Delete and Put operations. Using the
> Incremental counters saves me from doing any aggregations in my Map Reduce
> code. Ideally i would like to just call this function in my mapper and
> wouldn't even need a Reducer.
>    Has anyone run into this use case? I would also love to know if there
> are any better alternatives of solving this too. Any info would be great.
>
> Thanks
> Sid

Mime
View raw message