hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kurt Young (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-13960) HConnection stuck with UnknownHostException
Date Thu, 25 Jun 2015 01:14:05 GMT

    [ https://issues.apache.org/jira/browse/HBASE-13960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14600490#comment-14600490

Kurt Young commented on HBASE-13960:

In RpcClient::createBlockingRpcChannel, when new BlockingRpcChannelImplementation(), check
the isa and throw IOException if error occurred, exception thrown through  HConnectionImplementation::getClient,
and let RegionServerCallable::prepare fail, client we try again later
i think this maybe enough, but i haven't check all the details of callers who called RpcClient::createBlockingRpcChannel

> HConnection stuck with UnknownHostException 
> --------------------------------------------
>                 Key: HBASE-13960
>                 URL: https://issues.apache.org/jira/browse/HBASE-13960
>             Project: HBase
>          Issue Type: Bug
>          Components: hbase
>    Affects Versions: 0.98.8
>            Reporter: Kurt Young
> when put/get from hbase, if we meet a temporary dns failure causes resolve RS's host,
the error will never recovered. put/get will failed with UnknownHostException forever. 
> I checked the code, and the reason maybe:
> 1. when RegionServerCallable or MultiServerCallable prepare(), it gets a  ClientService.BlockingInterface
stub from Hconnection
> 2. In HConnectionImplementation::getClient, it caches the stub with a BlockingRpcChannelImplementation
> 3. In BlockingRpcChannelImplementation(), 
>      this.isa = new InetSocketAddress(sn.getHostname(), sn.getPort()); If we meet a 
temporary dns failure then the "address" in isa will be null.
> 4. then we launch the real rpc call, the following stack is:
> Caused by: java.net.UnknownHostException: unknown host: r101072047.sqa.zmf.tbsite.net
> 	at org.apache.hadoop.hbase.ipc.RpcClient$Connection.<init>(RpcClient.java:385)
> 	at org.apache.hadoop.hbase.ipc.RpcClient.createConnection(RpcClient.java:351)
> 	at org.apache.hadoop.hbase.ipc.RpcClient.getConnection(RpcClient.java:1523)
> 	at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1435)
> 	at org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1654)
> 	at org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1712)
> Besides, i noticed there is a protection in RpcClient:
> if (remoteId.getAddress().isUnresolved()) {
>         throw new UnknownHostException("unknown host: " + remoteId.getAddress().getHostName());
>       }
> shouldn't we do something when this situation occurred? 

This message was sent by Atlassian JIRA

View raw message