hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Marc Spaggiari <jean-m...@spaggiari.org>
Subject Re: How to implement increment in an idempotent manner
Date Fri, 18 Mar 2016 10:16:29 GMT
What's about the other option where each bolt increment it's own column and
at theend ou aggregate those few columns together?

2016-03-18 6:14 GMT-04:00 Sreeram <sreeram.v@gmail.com>:

> The incremented field is more like an amount field that will be storing the
> aggregate amount. Since the field will be incremented concurrently by
> multiple bolts running in parallel, storing the value before increment and
> then doing a put in case of replay will not help.
>
> Reason to have this field is to pre-compute a certain aggregate amount and
> materialize it in the Hbase table.
>
> On Fri, Mar 18, 2016 at 3:30 PM, Jean-Marc Spaggiari <
> jean-marc@spaggiari.org> wrote:
>
> > At the beginning of your Storm bolt process can you not do a put of "0"?
> So
> > it start back from scratch? Or else you will need to query the value, and
> > keep the value to put it back if you need to replay your bolt
> >
> > Other option is, you increment a specific difference column and at the
> end
> > if you are successful with your bolt you increment te initial column with
> > the new total counter?
> >
> > JMS
> >
> > 2016-03-18 5:33 GMT-04:00 Sreeram <sreeram.v@gmail.com>:
> >
> > > Hi,
> > >
> > >  I am looking for suggestions from community on implementing HBase
> > > increment in a idempotent manner.
> > >
> > >  My use case is a storm Hbase bolt that atomically increments a HBase
> > > counter. Replay of the storm bolt results in a double increment.
> > >
> > >  Any suggestion on the approach to be taken is welcome.
> > >
> > >  Thank you.
> > >
> > >  Regards,
> > >  Sreeram
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message