hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vladimir Rodionov <vladrodio...@gmail.com>
Subject Re: Hbase batch get and SocketTimeoutException
Date Fri, 10 Jul 2015 00:55:41 GMT
Yes, this is socket timeout due to very large batch which RS can not
process before timeout expires.
Just do not do 100K batch, or increase SocketTimeout (RPC config in
hbase-site.xml)

-Vlad

On Thu, Jul 9, 2015 at 8:53 AM, Ted Yu <yuzhihong@gmail.com> wrote:

> Looks like server side was not able to process all the Get's fast enough.
>
> Which release of hbase are you using ?
>
> Did you adjust any config parameters for hbase ?
>
> Thanks
>
> On Thu, Jul 9, 2015 at 12:22 AM, psobolewski <psobolewski89@gmail.com>
> wrote:
>
> > I use java and i want to do batch get like this
> >
> >     final List<Get> gets = uids.stream()
> >                     .map(uid -> new Get(toBytes(uid)))
> >                     .collect(Collectors.toList());
> >
> >     Configuration configuration = HBaseConfiguration.create();
> >
> >     conf.set("hbase.zookeeper.quorum", quorum);
> >     conf.set("hbase.zookeeper.property.clientPort",
> > properties.getString("HBASE_CONFIGURATION_ZOOKEEPER_CLIENTPORT"));
> >     conf.set("zookeeper.znode.parent",
> > properties.getString("HBASE_CONFIGURATION_ZOOKEEPER_ZNODE_PARENT"));
> >
> >     HTable table = new HTable(configuration, tableName);
> >     return table.get(gets);
> >
> > When get list has 10K gets everything is ok.
> >
> > When i try to do 100K gets in one batch i have exception:
> >
> >     java.lang.RuntimeException:
> > org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException:
> Failed
> > 100000 actions: SocketTimeoutException: 100000 times,
> >     Caused by:
> > org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException:
> Failed
> > 100000 actions: SocketTimeoutException: 100000 times,
> >             at
> >
> >
> org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.makeException(AsyncProcess.java:203)
> > ~[hbase-query-layer-r575958b.jar:?]
> >             at
> >
> >
> org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.access$500(AsyncProcess.java:187)
> > ~[hbase-query-layer-r575958b.jar:?]
> >             at
> >
> >
> org.apache.hadoop.hbase.client.AsyncProcess.getErrors(AsyncProcess.java:922)
> > ~[hbase-query-layer-r575958b.jar:?]
> >             at
> >
> >
> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:2402)
> > ~[hbase-query-layer-r575958b.jar:?]
> >             at
> > org.apache.hadoop.hbase.client.HTable.batchCallback(HTable.java:868)
> > ~[hbase-query-layer-r575958b.jar:?]
> >             at
> > org.apache.hadoop.hbase.client.HTable.batchCallback(HTable.java:883)
> > ~[hbase-query-layer-r575958b.jar:?]
> >             at
> org.apache.hadoop.hbase.client.HTable.batch(HTable.java:858)
> > ~[hbase-query-layer-r575958b.jar:?]
> >             at org.apache.hadoop.hbase.client.HTable.get(HTable.java:825)
> > ~[hbase-query-layer-r575958b.jar:?]
> >             at
> > hbase_query_layer.hbase.HbaseConnector.get(HbaseConnector.java:89)
> > ~[hbase-query-layer-r575958b.jar:?]
> >             ... 15 more
> >
> > What is wrong?
> >
> >
> >
> >
> > --
> > View this message in context:
> >
> http://apache-hbase.679495.n3.nabble.com/Hbase-batch-get-and-SocketTimeoutException-tp4073054.html
> > Sent from the HBase Developer mailing list archive at Nabble.com.
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message