Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (nike.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
  s=s1024; d=yahoo.com;
  h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:In-Reply-To:MIME-Version:Content-Type;
  b=QjUp73gAKBQaFR+OLgt3g9KhVbxk1jFRh6mlWdHRy24Sven75ZY9K/HpICxxnC6QNWa8NKT8iqGf9Pe4ZEB7cNg701SfzciiM2LFDVwfq2Pv9+OKgqAJFUhACBqaQL5yajhur/DrlUaK77eBqi1FEjy7+kCzwC9TDPyjHk2zWQI=;
References: <1377852561.60463.YahooMailNeo@web181606.mail.ne1.yahoo.com>
 <DC5EBE7F3610EB4CA5C7E92D76873E15180DD42AA6@exchange2007.carrieriq.com>
Message-ID: <1377896170.4191.YahooMailNeo@web181604.mail.ne1.yahoo.com>
Date: Fri, 30 Aug 2013 13:56:10 -0700 (PDT)
From: Kiru Pakkirisamy <kirupakkirisamy@yahoo.com>
Reply-To: Kiru Pakkirisamy <kirupakkirisamy@yahoo.com>
Subject: Re: Coprocessor concurrency 
To: "user@hbase.apache.org" <user@hbase.apache.org>
In-Reply-To: 
 <DC5EBE7F3610EB4CA5C7E92D76873E15180DD42AA6@exchange2007.carrieriq.com>
MIME-Version: 1.0
Content-Type: multipart/alternative;
 boundary="2017413661-1111410278-1377896170=:4191"

--2017413661-1111410278-1377896170=:4191
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable

Vlad,=0AI am running in separate processes. But I think I found at least pa=
rt of the problem.=0AIt is the HTable connection setup that is affecting it=
=0AIf I setup all the connections in all the processes for all the tables t=
hen and then call the endpoints it goes through ok.=0AAt least for the exam=
ple below.=0ABut with my real code where one endpoint does some computation=
 of atleast 400-500msec. It is degrading exponentially.=0A(I will reduce th=
e handler count and try ). Thanks and appreciate your response.=0AThese thi=
ngs are blocking me from doing real scalability testing. We got single clie=
nt down to 1.3 sec (target is under a sec) and now I want to move on to sca=
lability.=0A=0ARegards,=0A- kiru=0A=0A=0A________________________________=
=0A From: Vladimir Rodionov <vrodionov@carrieriq.com>=0ATo: "user@hbase.apa=
che.org" <user@hbase.apache.org>; Kiru Pakkirisamy <kirupakkirisamy@yahoo.c=
om> =0ASent: Friday, August 30, 2013 12:34 PM=0ASubject: RE: Coprocessor co=
ncurrency =0A =0A=0ACan you check RS thread stack traces during your test? =
If there are some concurrency/contention issues they will pop up in a stack=
 traces.=0AAlso, make sure that you run your client apps in a separate proc=
esses.=0A=0ABest regards,=0AVladimir Rodionov=0APrincipal Platform Engineer=
=0ACarrier IQ, www.carrieriq.com=0Ae-mail: vrodionov@carrieriq.com=0A=0A___=
_____________________________________=0AFrom: Kiru Pakkirisamy [kirupakkiri=
samy@yahoo.com]=0ASent: Friday, August 30, 2013 1:49 AM=0ATo: hbase mailing=
 list=0ASubject: Coprocessor concurrency=0A=0ASee below a null endpoint whi=
ch takes 60-70msec on my 4 node ec2 cluster for a table with 45 regions.=0A=
When I run 64 concurrent clients for this the latency jumps to 3000-3700 ms=
ec.=0A=0A(zookeeper maxClientCnxs is set to 0 (unlimited) and hbase regions=
erver handler count is 800). I hope I am not missing any configs for concur=
rency)=0A=0A---------------------------------------------------------------=
---------------------------------------------------------------------------=
----------------=0A=0Apackage com.serendio.hbase.coprocessor;=0A=0Aimport j=
ava.io.IOException;=0Aimport java.util.Map;=0A=0Aimport org.apache.hadoop.h=
base.coprocessor.BaseEndpointCoprocessor;=0A=0A//Aggregation implementation=
 at a region.=0Apublic class SearchEndpoint extends BaseEndpointCoprocessor=
 implements=0ASearchProtocol {=0A@Override=0Apublic Map<String, Double> foo=
(Map<String, Double> input, int topN)=0Athrows IOException {=0A// Implement=
 your logic here=0A// Map<String, Double> ret =3D new HashMap<String, Doubl=
e>();=0A// ret.put("foo", 1.0);=0Areturn null;=0A}=0A}=0A=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0A=0Apackage com.serendio.hbase.coprocess=
or;=0A=0Aimport java.io.IOException;=0Aimport java.util.Map;=0A=0Aimport or=
g.apache.hadoop.hbase.ipc.CoprocessorProtocol;=0A=0Apublic interface Search=
Protocol extends CoprocessorProtocol {=0A=0Apublic Map<String, Double> foo(=
Map<String, Double> input,=0Aint topN) throws IOException;=0A}=0A=0A=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=0Apackag=
e com.serendio.hbase.coprocessor;=0A=0Aimport java.io.IOException;=0Aimport=
 java.util.HashMap;=0Aimport java.util.Map;=0A=0Aimport org.apache.hadoop.h=
base.HBaseConfiguration;=0Aimport org.apache.hadoop.hbase.client.HTable;=0A=
import org.apache.hadoop.hbase.client.HTableInterface;=0Aimport org.apache.=
hadoop.hbase.client.coprocessor.Batch.Call;=0A=0Apublic class Search {=0Apu=
blic static final String TEST_TABLE =3D "c4";=0Apublic static final String =
TEST_FAMILY =3D "info";=0Apublic static final String TEST_QUALIFIER =3D "c1=
";=0A=0A/**=0A* ``=0A*=0A* @param args=0A*/=0Apublic static void main(Strin=
g[] args) {=0A=0Aorg.apache.hadoop.conf.Configuration conf =3D HBaseConfigu=
ration.create();=0AHTableInterface table =3D null;=0Atry {=0Atable =3D new =
HTable(conf, TEST_TABLE);=0A=0A} catch (IOException e2) {=0A// TODO Auto-ge=
nerated catch block=0Ae2.printStackTrace();=0A}=0A=0AMap<byte[], Map<String=
, Double>> results =3D null;=0Afinal Map<String, Double> input =3D new Hash=
Map<String, Double>();=0Ainput.put("test", 1.0);=0Afinal int topN =3D 10;=
=0Along start =3D System.currentTimeMillis();=0Atry {=0Aresults =3D table.c=
oprocessorExec(SearchProtocol.class, null, null,=0Anew Call<SearchProtocol,=
 Map<String, Double>>() {=0A=0A@Override=0Apublic Map<String, Double> call(=
SearchProtocol instance)=0Athrows IOException {=0A// TODO Auto-generated me=
thod stub=0Areturn instance.foo(input, topN);=0A}=0A});=0A} catch (IOExcept=
ion e1) {=0A// TODO Auto-generated catch block=0Ae1.printStackTrace();=0A} =
catch (Throwable e1) {=0A// TODO Auto-generated catch block=0Ae1.printStack=
Trace();=0A}=0Along end =3D System.currentTimeMillis();=0ASystem.out.printl=
n("Total search elapsed time: "=0A+ Long.toString(end - start));=0A}=0A=0A}=
=0A=0A=0A=0ARegards,=0A- kiru=0A=0AConfidentiality Notice:=A0 The informati=
on contained in this message, including any attachments hereto, may be conf=
idential and is intended to be read only by the individual or entity to who=
m this message is addressed. If the reader of this message is not the inten=
ded recipient or an agent or designee of the intended recipient, please not=
e that any review, use, disclosure or distribution of this message or its a=
ttachments, in any form, is strictly prohibited.=A0 If you have received th=
is message in error, please immediately notify the sender and/or Notificati=
ons@carrieriq.com and delete or destroy any copy of this message and its at=
tachments.
--2017413661-1111410278-1377896170=:4191--