Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (nike.apache.org: domain of andrew.purtell@gmail.com
 designates 209.85.220.47 as permitted sender)
References: 
 <CADt=Q7=vRLuH5JOB+2ouz=GUBcEdEsL09XscYi4OXYC4cmq_Yw@mail.gmail.com>
 <CA+RK=_BBnnz2LOCwpvsHz=JCtg3v_8wOyjMAe29kjROyEpEqVQ@mail.gmail.com>
 <CALte62yVGf3LL0hpG666EB-QygdpRM+A9=tPmcWh7frG37-eMQ@mail.gmail.com>
Mime-Version: 1.0 (1.0)
In-Reply-To: 
 <CALte62yVGf3LL0hpG666EB-QygdpRM+A9=tPmcWh7frG37-eMQ@mail.gmail.com>
Content-Type: text/plain;
	charset=us-ascii
Content-Transfer-Encoding: quoted-printable
Message-Id: <C9820C24-A89F-42CE-8B29-6EA321469964@gmail.com>
Cc: "user@hbase.apache.org" <user@hbase.apache.org>
From: Andrew Purtell <andrew.purtell@gmail.com>
Subject: Re: Coprocessor / threading model
Date: Sat, 12 Jan 2013 19:58:30 -0800
To: "user@hbase.apache.org" <user@hbase.apache.org>

Yes, especially if the cross region communication is in process.

On Jan 12, 2013, at 6:48 PM, Ted Yu <yuzhihong@gmail.com> wrote:

> bq. Coprocessors are meant to operate on the region to which they are
> associated.
>=20
> For Anoop's case, the secondary table(s) have their regions aligned with
> the corresponding region from primary table. Meaning, related regions are
> served by the same region server.
> Would writes to such regions of secondary table(s) be acceptable ?
>=20
> Thanks
>=20
> On Sat, Jan 12, 2013 at 6:39 PM, Andrew Purtell <apurtell@apache.org> wrot=
e:
>=20
>>> In pre-put, I trigger another Put() in an external table (to build the
>> secondary index).
>>=20
>> We should probably call this a Coprocessor anti-pattern.
>>=20
>> Coprocessors are meant to operate on the region to which they are
>> associated. They are a way you can extend HBase function while it operate=
s
>> in region on data for the region. Think of them as loadable kernel module=
s.
>> They are not a general purpose server side platform for programming as if=

>> you are building a HBase client (with HTable, etc.). Just because you can=

>> do this doesn't mean you should.
>>=20
>>=20
>> On Sat, Jan 12, 2013 at 4:06 PM, Adrien Mogenet <adrien.mogenet@gmail.com=

>>> wrote:
>>=20
>>> Hi there,
>>>=20
>>> I'm experiencing some issues with CP. I'm trying to implement an indexin=
g
>>> solution (inspired by Annop's slides). In pre-put, I trigger another
>> Put()
>>> in an external table (to build the secondary index). It works perfect fo=
r
>>> one client, but when I'm inserting data from 2 separate clients, I met
>>> issues with HTable object (the one used in pre-Put()), because it's not
>>> thread-safe. I decided to move on TablePool and that fixed my issue.
>>>=20
>>> But if I increase the write-load (and concurrency) HBase is throwing a
>> OOM
>>> exception because it can't create new native threads. Looking at HBase
>>> metrics "threads count", I see that roughly 3500 threads are created.
>>>=20
>>> I'm looking for documentation about how CPs are working with threads :
>>> what/when should I protect against concurrency issues ? How may I solve
>> my
>>> issue ?
>>>=20
>>> Help is welcome :-)
>>>=20
>>> --
>>> Adrien Mogenet
>>> 06.59.16.64.22
>>> http://www.mogenet.me
>>=20
>>=20
>>=20
>> --
>> Best regards,
>>=20
>>   - Andy
>>=20
>> Problems worthy of attack prove their worth by hitting back. - Piet Hein
>> (via Tom White)
>>=20