Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (nike.apache.org: domain of anoopsj@huawei.com designates
 119.145.14.65 as permitted sender)
From: Anoop Sam John <anoopsj@huawei.com>
To: "user@hbase.apache.org" <user@hbase.apache.org>
Subject: RE: Coprocessor tests under busy insertions
Thread-Topic: Coprocessor tests under busy insertions
Thread-Index: AQHNePXRcel63EG0H0ueK6TozJ/eKpdXHgVA
Date: Mon, 13 Aug 2012 04:01:23 +0000
Message-ID: 
 <0CE69E9126D0344088798A3B7F7F80863A4C55A0@szxeml531-mbs.china.huawei.com>
References: <85E959BE-2DDA-49FE-88E5-09E9DA76C88E@gmail.com>
In-Reply-To: <85E959BE-2DDA-49FE-88E5-09E9DA76C88E@gmail.com>
Accept-Language: en-US, zh-CN
Content-Language: en-US
Content-Type: text/plain; charset="Windows-1252"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0

Can u paste your CP implementation here [prePut/ postPut?]
Are u doing check for the table in CP hook? U need to only handle the hooks=
 while it is being called for your table. Remember that your index table al=
so have these same hooks.

-Anoop-
________________________________________
From: Henry JunYoung KIM [henry.jykim@gmail.com]
Sent: Monday, August 13, 2012 7:18 AM
To: user@hbase.apache.org
Subject: Coprocessor tests under busy insertions

Hi, hbase users.

now, I am testing coprocessors to create secondary indexes in background.
coprocessors itself is packaged in base 0.92.1 I am using.

the scenario I want to describe is this one.

the main table is 'blog' which is having a field named 'userId'.
from this field I want to create secondary index to map 'userId' and it's '=
url'.

I put RegionObserver implementations in my secondary index creator.

the situation I got from hbase is this log.

------------
12/08/13 10:37:08 WARN client.HConnectionManager$HConnectionImplementation:=
 Failed all from region=3Dblog,user6447991910946051755,1344821177585.7d4cbd=
4a9817ab7cb5c6219498d854a4., hostname=3Dsearch-ddm-test5, port=3D60020
java.util.concurrent.ExecutionException: java.net.SocketTimeoutException: C=
all to search-ddm-test5/xx.xx.xx.xx:60020 failed on socket timeout exceptio=
n: java.net.SocketTimeoutException: 60000 millis timeout while waiting for =
channel to be ready for read. ch : java.nio.channels.SocketChannel[connecte=
d local=3D/xx.xx.xx.xx:53733 remote=3Dsearch-ddm-test5/xx.xx.xx.xx:60020]
        at java.util.concurrent.FutureTask$Sync.innerGet(FutureTask.java:22=
2)
        at java.util.concurrent.FutureTask.get(FutureTask.java:83)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImp=
lementation.processBatchCallback(HConnectionManager.java:1557)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImp=
lementation.processBatch(HConnectionManager.java:1409)
        at org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:9=
43)
        at org.apache.hadoop.hbase.client.HTable.doPut(HTable.java:820)
        at org.apache.hadoop.hbase.client.HTable.put(HTable.java:795)
        at com.yahoo.ycsb.db.HBaseClient.update(HBaseClient.java:321)
        at com.yahoo.ycsb.DBWrapper.update(DBWrapper.java:126)
        at com.yahoo.ycsb.workloads.CoreWorkload.doTransactionUpdate(CoreWo=
rkload.java:628)
        at com.yahoo.ycsb.workloads.CoreWorkload.doTransaction(CoreWorkload=
.java:483)
        at com.yahoo.ycsb.ClientThread.run(Client.java:233)
Caused by: java.net.SocketTimeoutException: Call to search-ddm-test5/10.11.=
239.220:60020 failed on socket timeout exception: java.net.SocketTimeoutExc=
eption: 60000 millis timeout while waiting for channel to be ready for read=
. ch : java.nio.channels.SocketChannel[connected local=3D/xx.xx.xx.xx:53733=
 remote=3Dsearch-ddm-test5/xx.xx.xx.xx:60020]
        at org.apache.hadoop.hbase.ipc.HBaseClient.wrapException(HBaseClien=
t.java:949)
        at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:92=
2)
        at org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(Wri=
tableRpcEngine.java:150)
        at $Proxy5.multi(Unknown Source)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImp=
lementation$3$1.call(HConnectionManager.java:1386)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImp=
lementation$3$1.call(HConnectionManager.java:1384)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImp=
lementation.getRegionServerWithoutRetries(HConnectionManager.java:1365)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImp=
lementation$3.call(HConnectionManager.java:1383)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImp=
lementation$3.call(HConnectionManager.java:1381)
        at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:30=
3)
        at java.util.concurrent.FutureTask.run(FutureTask.java:138)
        at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoo=
lExecutor.java:886)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExe=
cutor.java:908)
        at java.lang.Thread.run(Thread.java:662)
Caused by: java.net.SocketTimeoutException: 60000 millis timeout while wait=
ing for channel to be ready for read. ch : java.nio.channels.SocketChannel[=
connected local=3D/xx.xx.xx.xx:53733 remote=3Dsearch-ddm-test5/xx.xx.xx.xx:=
60020]
        at org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeo=
ut.java:164)
        at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.j=
ava:155)
        at org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.j=
ava:128)
        at java.io.FilterInputStream.read(FilterInputStream.java:116)
        at org.apache.hadoop.hbase.ipc.HBaseClient$Connection$PingInputStre=
am.read(HBaseClient.java:311)
        at java.io.BufferedInputStream.fill(BufferedInputStream.java:218)
        at java.io.BufferedInputStream.read(BufferedInputStream.java:237)
        at java.io.DataInputStream.readInt(DataInputStream.java:370)
        at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.receiveRespon=
se(HBaseClient.java:571)
        at org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClie=
nt.java:505)
 200 sec: 237725 operations; 0 current ops/sec;
 210 sec: 237725 operations; 0 current ops/sec;
 220 sec: 237725 operations; 0 current ops/sec;
=85.
=85.
=85
 560 sec: 237725 operations; 0 current ops/sec;
 570 sec: 237725 operations; 0 current ops/sec;
 580 sec: 237725 operations; 0 current ops/sec;
 590 sec: 237725 operations; 0 current ops/sec;
 600 sec: 237725 operations; 0 current ops/sec;
 610 sec: 237725 operations; 0 current ops/sec;
 620 sec: 237725 operations; 0 current ops/sec;
 630 sec: 237725 operations; 0 current ops/sec;
 640 sec: 237725 operations; 0 current ops/sec;
 650 sec: 237725 operations; 0 current ops/sec;
 660 sec: 237725 operations; 0 current ops/sec;
 670 sec: 237725 operations; 0 current ops/sec;
 680 sec: 237725 operations; 0 current ops/sec;
 690 sec: 237725 operations; 0 current ops/sec;
 700 sec: 237725 operations; 0 current ops/sec;
 710 sec: 237725 operations; 0 current ops/sec;
 720 sec: 237725 operations; 0 current ops/sec;
 730 sec: 237725 operations; 0 current ops/sec;
 740 sec: 237725 operations; 0 current ops/sec;
 750 sec: 237725 operations; 0 current ops/sec;
 760 sec: 237725 operations; 0 current ops/sec;
------------

there was several normal insertions about 10000 TPS.
but, after this operations, no more insertions, just warnings from hbase.

in my implementations,
I didn't use HTablePool, Counter, only thing I used is just to put the mapp=
ing information into a secondary index table.

so, the question is.
1. does coprocessors not reliable yet?
2. how does it's performance?
3. is there any references which can help us to make it safe usages?

thanks for your concerns.

- Henry=