hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bryan Beaudreault <bbeaudrea...@hubspot.com>
Subject Re: hbase increments and hadoop attempts
Date Mon, 25 Mar 2013 14:22:43 GMT
Increments are not idempotent, so yes you will double increment the set of
increments that succeeded in the first attempt(s).  If you care about that
you're better off not using the Increment interface and instead having 2
jobs: one that does a Get of the current value and adds the offset then
passes through to the next job.  The next job would simply save the locally
incremented values with a series of Puts.  This way you can re-run either
job any number of times without double-incrementing anything.

On Mon, Mar 25, 2013 at 10:17 AM, prakash kadel <prakash.kadel@gmail.com>wrote:

> hi everyone,
>   when i launch my mapreduce jobs to increment counters in hbase i
> sometimes have maps
> with multiple attempts like:
> attempt_201303251722_0161_m_000074_0
> attempt_201303251722_0161_m_000074_1
> if there are multiple attempts running and if the first one gets
> completed successful,
> what will happen to the increments made by the second attempt
> will there be unnecessary increments due to the second attempt running?
> thank you.
> Sincerely
> Prakash

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message