hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Hbase batch get and SocketTimeoutException
Date Thu, 09 Jul 2015 15:53:05 GMT
Looks like server side was not able to process all the Get's fast enough.

Which release of hbase are you using ?

Did you adjust any config parameters for hbase ?

Thanks

On Thu, Jul 9, 2015 at 12:22 AM, psobolewski <psobolewski89@gmail.com>
wrote:

> I use java and i want to do batch get like this
>
>     final List<Get> gets = uids.stream()
>                     .map(uid -> new Get(toBytes(uid)))
>                     .collect(Collectors.toList());
>
>     Configuration configuration = HBaseConfiguration.create();
>
>     conf.set("hbase.zookeeper.quorum", quorum);
>     conf.set("hbase.zookeeper.property.clientPort",
> properties.getString("HBASE_CONFIGURATION_ZOOKEEPER_CLIENTPORT"));
>     conf.set("zookeeper.znode.parent",
> properties.getString("HBASE_CONFIGURATION_ZOOKEEPER_ZNODE_PARENT"));
>
>     HTable table = new HTable(configuration, tableName);
>     return table.get(gets);
>
> When get list has 10K gets everything is ok.
>
> When i try to do 100K gets in one batch i have exception:
>
>     java.lang.RuntimeException:
> org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed
> 100000 actions: SocketTimeoutException: 100000 times,
>     Caused by:
> org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed
> 100000 actions: SocketTimeoutException: 100000 times,
>             at
>
> org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.makeException(AsyncProcess.java:203)
> ~[hbase-query-layer-r575958b.jar:?]
>             at
>
> org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.access$500(AsyncProcess.java:187)
> ~[hbase-query-layer-r575958b.jar:?]
>             at
>
> org.apache.hadoop.hbase.client.AsyncProcess.getErrors(AsyncProcess.java:922)
> ~[hbase-query-layer-r575958b.jar:?]
>             at
>
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:2402)
> ~[hbase-query-layer-r575958b.jar:?]
>             at
> org.apache.hadoop.hbase.client.HTable.batchCallback(HTable.java:868)
> ~[hbase-query-layer-r575958b.jar:?]
>             at
> org.apache.hadoop.hbase.client.HTable.batchCallback(HTable.java:883)
> ~[hbase-query-layer-r575958b.jar:?]
>             at org.apache.hadoop.hbase.client.HTable.batch(HTable.java:858)
> ~[hbase-query-layer-r575958b.jar:?]
>             at org.apache.hadoop.hbase.client.HTable.get(HTable.java:825)
> ~[hbase-query-layer-r575958b.jar:?]
>             at
> hbase_query_layer.hbase.HbaseConnector.get(HbaseConnector.java:89)
> ~[hbase-query-layer-r575958b.jar:?]
>             ... 15 more
>
> What is wrong?
>
>
>
>
> --
> View this message in context:
> http://apache-hbase.679495.n3.nabble.com/Hbase-batch-get-and-SocketTimeoutException-tp4073054.html
> Sent from the HBase Developer mailing list archive at Nabble.com.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message