hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Wei Tan <w...@us.ibm.com>
Subject Re: Coprocessor / threading model
Date Tue, 15 Jan 2013 18:44:40 GMT
Andrew, could you explain more, why doing cross-table operation is an 
anti-pattern of using CP?
Durability might be an issue, as far as I understand. Thanks,

Best Regards,

From:   Andrew Purtell <apurtell@apache.org>
To:     "user@hbase.apache.org" <user@hbase.apache.org>, 
Date:   01/12/2013 09:39 PM
Subject:        Re: Coprocessor / threading model

> In pre-put, I trigger another Put() in an external table (to build the
secondary index).

We should probably call this a Coprocessor anti-pattern.

Coprocessors are meant to operate on the region to which they are
associated. They are a way you can extend HBase function while it operates
in region on data for the region. Think of them as loadable kernel 
They are not a general purpose server side platform for programming as if
you are building a HBase client (with HTable, etc.). Just because you can
do this doesn't mean you should.

On Sat, Jan 12, 2013 at 4:06 PM, Adrien Mogenet 

> Hi there,
> I'm experiencing some issues with CP. I'm trying to implement an 
> solution (inspired by Annop's slides). In pre-put, I trigger another 
> in an external table (to build the secondary index). It works perfect 
> one client, but when I'm inserting data from 2 separate clients, I met
> issues with HTable object (the one used in pre-Put()), because it's not
> thread-safe. I decided to move on TablePool and that fixed my issue.
> But if I increase the write-load (and concurrency) HBase is throwing a 
> exception because it can't create new native threads. Looking at HBase
> metrics "threads count", I see that roughly 3500 threads are created.
> I'm looking for documentation about how CPs are working with threads :
> what/when should I protect against concurrency issues ? How may I solve 
> issue ?
> Help is welcome :-)
> --
> Adrien Mogenet
> http://www.mogenet.me

Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message