hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Billy Pearson" <sa...@pearsonwholesale.com>
Subject Re: incrementColumnValue
Date Wed, 12 Jan 2011 08:07:19 GMT
Thanks for that info did not thank about it that way but good reason.

Billy


> Hey,
>
> It is not possible, nor alas would it be a good idea.  Speculative
> execution can cause jobs to run twice with the "results" discarded
> from one.  The hbase output format doesnt really have a good way to
> 'discard' results, since we are outputting to a table not to a file
> that can be tossed.
>
> Futhermore, failures will cause job reruns, and the ICV is not exactly
> what you'd call idempotent.  You can instantiate HTable and call ICV
> directly yourself in either the map or reduce phase, but again, not
> recommended.
>
> You can also summarize your data and use a secondary process to
> execute a roll up of ICVs... if the number isnt too massive this might
> be acceptable.
>
> On Tue, Jan 11, 2011 at 4:07 PM, Billy Pearson
> <sales@pearsonwholesale.com> wrote:
>> Is there a way to make a mapreduce job and use incrementColumnValue in 
>> place
>> of Put?
>>
>> I am trying to move a job over from thrift and have to be able to use
>> incrementColumnValue
>> as a output but I can not seams to work it out with out calling HTable 
>> every
>> map.
>>
>> small example would be nice if anyone uses it now
>> Billy
>>
>>
>>
> 



Mime
View raw message