hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From psobolewski <psobolewsk...@gmail.com>
Subject Hbase batch get and SocketTimeoutException
Date Thu, 09 Jul 2015 07:22:17 GMT
I use java and i want to do batch get like this

    final List<Get> gets = uids.stream()
                    .map(uid -> new Get(toBytes(uid)))
                    .collect(Collectors.toList());
    
    Configuration configuration = HBaseConfiguration.create();
    
    conf.set("hbase.zookeeper.quorum", quorum);
    conf.set("hbase.zookeeper.property.clientPort",
properties.getString("HBASE_CONFIGURATION_ZOOKEEPER_CLIENTPORT"));
    conf.set("zookeeper.znode.parent",
properties.getString("HBASE_CONFIGURATION_ZOOKEEPER_ZNODE_PARENT"));
    
    HTable table = new HTable(configuration, tableName);
    return table.get(gets);
        
When get list has 10K gets everything is ok.

When i try to do 100K gets in one batch i have exception:

    java.lang.RuntimeException:
org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed
100000 actions: SocketTimeoutException: 100000 times, 
    Caused by:
org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed
100000 actions: SocketTimeoutException: 100000 times, 
            at
org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.makeException(AsyncProcess.java:203)
~[hbase-query-layer-r575958b.jar:?]
            at
org.apache.hadoop.hbase.client.AsyncProcess$BatchErrors.access$500(AsyncProcess.java:187)
~[hbase-query-layer-r575958b.jar:?]
            at
org.apache.hadoop.hbase.client.AsyncProcess.getErrors(AsyncProcess.java:922)
~[hbase-query-layer-r575958b.jar:?]
            at
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.processBatchCallback(HConnectionManager.java:2402)
~[hbase-query-layer-r575958b.jar:?]
            at
org.apache.hadoop.hbase.client.HTable.batchCallback(HTable.java:868)
~[hbase-query-layer-r575958b.jar:?]
            at
org.apache.hadoop.hbase.client.HTable.batchCallback(HTable.java:883)
~[hbase-query-layer-r575958b.jar:?]
            at org.apache.hadoop.hbase.client.HTable.batch(HTable.java:858)
~[hbase-query-layer-r575958b.jar:?]
            at org.apache.hadoop.hbase.client.HTable.get(HTable.java:825)
~[hbase-query-layer-r575958b.jar:?]
            at
hbase_query_layer.hbase.HbaseConnector.get(HbaseConnector.java:89)
~[hbase-query-layer-r575958b.jar:?]
            ... 15 more

What is wrong?




--
View this message in context: http://apache-hbase.679495.n3.nabble.com/Hbase-batch-get-and-SocketTimeoutException-tp4073054.html
Sent from the HBase Developer mailing list archive at Nabble.com.

Mime
View raw message