hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: 60000 millis timeout while waiting for channel to be ready for read
Date Tue, 09 Jul 2013 15:02:39 GMT
Which HBase release are you using ?

On Tue, Jul 9, 2013 at 4:54 AM, shapoor <esmaili_607@yahoo.com> wrote:

> hello,
>
> i am doing a lot of saves in HBase. like 100,000,000 documents, each 100KB.
> before i start the program there are almost 18 connections after starting
> my
> 2 regions and one master cluster. the connections are zookeeper, hdfs and
> hbase. as the process of saving starts, i have repeatedly more connections
> (i guess for each time flushing) until i reach almost 80 connections and
> that's when the following exeption appears. now HBase manages to save the
> data somehow but it is not effective with so many connections. how do I
> save
> this problem??
>
> regards, shapoor
>
> 13/07/09 13:31:48 WARN client.HConnectionManager$HConnectionImplementation:
> Failed all from
>
> region=table2,doc-id-866604,1373369430484.09001c90b3d2a4c20b56c35bb976ff91.,
> hostname=kcs-testhadoop02, port=60020
> java.util.concurrent.ExecutionException: java.net.SocketTimeoutException:
> Call to kcs-testhadoop02/192.168.111.211:60020 failed on socket timeout
> exception: java.net.SocketTimeoutException: 60000 millis timeout while
> waiting for channel to be ready for read. ch :
> java.nio.channels.SocketChannel[connected local=/192.168.111.72:55354
> remote=kcs-testhadoop02/192.168.111.211:60020]
>         at java.util.concurrent.FutureTask$Sync.innerGet(Unknown Source)
>         at java.util.concurrent.FutureTask.get(Unknown Source)
>         at
>
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:1598)
>         at
>
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatch(HConnectionManager.java:1450)
>         at
> org.apache.hadoop.hbase.client.HTable.flushCommits(HTable.java:916)
>         at at.knowcenter.backends.HBaseStorage.flush(HBaseStorage.java:324)
>         at at.knowcenter.evaltool.Evaluate.save(Evaluate.java:155)
>         at
> at.knowcenter.evaltool.Evaluate.performSaveEvaluation(Evaluate.java:100)
>         at at.knowcenter.evaltool.Evaluate.evaluate(Evaluate.java:77)
>         at
> at.knowcenter.evaltool.EvaluationTool.execute(EvaluationTool.java:144)
>         at
> at.knowcenter.evaltool.EvaluationTool.main(EvaluationTool.java:199)
> Caused by: java.net.SocketTimeoutException: Call to
> kcs-testhadoop02/192.168.111.211:60020 failed on socket timeout exception:
> java.net.SocketTimeoutException: 60000 millis timeout while waiting for
> channel to be ready for read. ch :
> java.nio.channels.SocketChannel[connected
> local=/192.168.111.72:55354 remote=kcs-testhadoop02/192.168.111.211:60020]
>         at
>
> org.apache.hadoop.hbase.ipc.HBaseClient.wrapException(HBaseClient.java:1026)
>         at
> org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:999)
>         at
>
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:86)
>         at com.sun.proxy.$Proxy6.multi(Unknown Source)
>         at
>
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3$1.call(HConnectionManager.java:1427)
>         at
>
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3$1.call(HConnectionManager.java:1425)
>         at
>
> org.apache.hadoop.hbase.client.ServerCallable.withoutRetries(ServerCallable.java:215)
>         at
>
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3.call(HConnectionManager.java:1434)
>         at
>
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$3.call(HConnectionManager.java:1422)
>         at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
>         at java.util.concurrent.FutureTask.run(Unknown Source)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown
> Source)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown
> Source)
>         at java.lang.Thread.run(Unknown Source)
> Caused by: java.net.SocketTimeoutException: 60000 millis timeout while
> waiting for channel to be ready for read. ch :
> java.nio.channels.SocketChannel[connected local=/192.168.111.72:55354
> remote=kcs-testhadoop02/192.168.111.211:60020]
>         at
>
> org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:164)
>         at
> org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:155)
>         at
> org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:128)
>         at java.io.FilterInputStream.read(Unknown Source)
>         at
>
> org.apache.hadoop.hbase.ipc.HBaseClient$Connection$PingInputStream.read(HBaseClient.java:373)
>         at java.io.BufferedInputStream.fill(Unknown Source)
>         at java.io.BufferedInputStream.read(Unknown Source)
>         at java.io.DataInputStream.readInt(Unknown Source)
>         at
>
> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.receiveResponse(HBaseClient.java:646)
>         at
>
> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.run(HBaseClient.java:580)
>
>
>
> --
> View this message in context:
> http://apache-hbase.679495.n3.nabble.com/60000-millis-timeout-while-waiting-for-channel-to-be-ready-for-read-tp4047612.html
> Sent from the HBase Developer mailing list archive at Nabble.com.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message