hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From <Hariharan_Sethura...@Dell.com>
Subject RE: HBase client: refreshing the connection
Date Tue, 09 Jun 2015 05:17:07 GMT
Thanks Nicolas.
We will have only one master server locally in our application, tried  with retries =7 and
pause=400ms. It came down to 8 mins which we are ok.

Thanks,
Hari

-----Original Message-----
From: Nicolas Liochon [mailto:nkeywal@gmail.com]
Sent: Tuesday, June 02, 2015 8:05 PM
To: user
Cc: C, Yuling
Subject: Re: HBase client: refreshing the connection

(removing dev list)
> We observed there is an zxid mismatch on hbase server logs.
This looks like a bug. But the 0.94.15 is quite old now...

For hbase.client.retries.number=3, HBase uses an exponential back-off time, so setting the
#retry to 3 will reduce the hanging time to a few dozen of seconds max, likely less. 3 is
too small for most cases because many situations will lead to a retry (typically if the region
location cache is not up to date, that's a retry). Lowering pause is good as well (iirc it's
1s in 0.94, reducing it to 500ms should be ok).

On Mon, Jun 1, 2015 at 10:21 AM, wrote:

> Hi All,
>
> We are using 0.94.15 in our Opendaylight/TSDR project currently.
>
> We observed put operation hanged for 20 mins (with all default
> timeouts) and then throws an IOException. Even when we re-attempt the
> same put operation, it hangs for 20 mins again. We observed there is
> an zxid mismatch on hbase server logs.
>
> We wanted to get clarified for the following items.
>
> 1) Reducing this hanging time from 20 mins to 5 mins: Looks there are
> many timeout configuration (hbase-client, zookeeper, client.pause etc)
> and it slightly confusing how they are all calculated with backoff
> series. If I add the configuration hbase.client.retries.number=3 in
> hbase-site.xml will bring down it to 5 mins?
>
>
>
> 2) When we receive this exception, we deletedAllConnections and
> subsequent put operation succeeded. We wish to continue this approach.
> Following is our code where we create HTable.
>
> HTableInterface htableResult = null;
>
> htableResult = htableMap.get(tableName);
>
> ..
>
> if (htableResult == null) {
>
> if (htablePool == null ||
> htablePool.getTable(tableName) == null) {
>
> htablePool = new HTablePool(getConfiguration(),
> poolSize);
>
> }
>
> if ( htablePool != null){
>
> htableResult = htablePool.getTable(tableName);
>
> ..
>
> }
>
> }
>
> htableMap.put(tableName, htableResult);
>
>
>
> We create 5 tables in our application. Will there be 5 HConnection
> totally and each HConnection for each Table? If yes, how do I delete a
> connection for the given table as most of the delete(All)Connections
> in HConnectionManager are deprecated in 0.94.15. No alternatives given
> in the java doc. Even if we use deleteConnection, it asks for conf
> which doesn't bind to any table, correct?
>
> deleteConnection
> @Deprecated
> public static void
> deleteConnection(org.apache.hadoop.conf.Configuration
> conf)
> Deprecated.
> Delete connection information for the instance specified by configuration.
> If there are no more references to it, this will then close connection
> to the zookeeper ensemble and let go of all resources.
> Parameters:
> conf - configuration whose identity is used to find HConnection instance.
>
>
> deleteAllConnections
> @Deprecated
> public static void deleteAllConnections(boolean stopProxy) Deprecated.
> use deleteAllConnections() instead Delete information for all
> connections.
> Parameters:
> stopProxy - No longer used. This parameter is ignored.
>
>
> deleteAllConnections
> @Deprecated
> public static void deleteAllConnections() Deprecated.
> Delete information for all connections.
> Throws:
> IOException
>
> Thanks,
> Hari
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message