hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dan Harvey <danharve...@gmail.com>
Subject Re: async table updates?
Date Wed, 23 Feb 2011 23:37:13 GMT
You could use something like the row log library from the Lilly project to queue the processing
from one table to the other every time you do a put in HBase. As the queue is in HBase itself
too it's all atomic, I'm sure you could get it working with minimal lag between the two tables.

http://www.lilyproject.org/lily/about/playground/hbaserowlog.html

On 23 Feb 2011, at 22:40, Ryan Rawson <ryanobjc@gmail.com> wrote:

> In thrift there is a 'oneway' or 'async' or 'fire and forget' call
> type.  I cant recommend those kinds of approaches, since once your
> system runs into problems you have no feedback.  So if you are asking
> for a one shot, no reply "assume it worked" call, we don't have one
> (nor would I wish that hell on anyone).
> 
> So it boils down to your application architecture.  I'm sure you can
> figure out how to get notifications via callbacks (with background
> threads or threadpools).
> 
> Or did you mean something else?
> -ryan
> 
> On Wed, Feb 23, 2011 at 2:31 PM, Vishal Kapoor
> <vishal.kapoor.in@gmail.com> wrote:
>> I have two tables called LIVE and MASTER.
>> LIVE reports about the MASTER activity and I need to process records in LIVE
>> almost real time( some business logic)
>> 
>> if I need to store the activity of entities reported by LIVE rows in MASTER
>> say in ACTIVITY:LAST_REPORTED
>> 
>> I could process my data in LIVE and at the same time update the MASTER's
>> "LIVE REPORTED" attribute with value of current timestamp.
>> but I was wondering if I can update the MASTER table in a async call so that
>> I don't compromise on the speed.
>> was wondering if anyone tried to achieve this or even if it is advisable...
>> 
>> 
>> thanks,
>> Vishal Kapoor
>> 

Mime
View raw message