hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sean Busbey (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-15018) Inconsistent way of handling TimeoutException in the rpc client implementations
Date Wed, 30 Dec 2015 06:41:49 GMT

    [ https://issues.apache.org/jira/browse/HBASE-15018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15074686#comment-15074686
] 

Sean Busbey commented on HBASE-15018:
-------------------------------------

I pushed this on master, but when I cherry-pick back to branch-1 I consistently get a failure
from TestGenerateDelegationToken:

{code}
Running org.apache.hadoop.hbase.security.token.TestGenerateDelegationToken
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 270.558 sec <<< FAILURE!
- in org.apache.hadoop.hbase.security.token.TestGenerateDelegationToken
org.apache.hadoop.hbase.security.token.TestGenerateDelegationToken  Time elapsed: 270.548
sec  <<< ERROR!
java.lang.RuntimeException: Master not initialized after 200000ms seconds
	at org.apache.hadoop.hbase.util.JVMClusterUtil.startup(JVMClusterUtil.java:225)
	at org.apache.hadoop.hbase.LocalHBaseCluster.startup(LocalHBaseCluster.java:447)
	at org.apache.hadoop.hbase.security.token.TestGenerateDelegationToken.setUp(TestGenerateDelegationToken.java:129)

{code}

> Inconsistent way of handling TimeoutException in the rpc client implementations
> -------------------------------------------------------------------------------
>
>                 Key: HBASE-15018
>                 URL: https://issues.apache.org/jira/browse/HBASE-15018
>             Project: HBase
>          Issue Type: Bug
>          Components: Client, IPC/RPC
>    Affects Versions: 2.0.0, 1.1.0, 1.2.0
>            Reporter: Ashish Singhi
>            Assignee: Ashish Singhi
>             Fix For: 2.0.0, 1.2.0, 1.3.0, 1.1.3
>
>         Attachments: HBASE-15018-v1(1).patch, HBASE-15018-v1.patch, HBASE-15018-v2.patch,
HBASE-15018-v3.patch, HBASE-15018.patch, HBASE-15018.patch
>
>
> If there is any rpc timeout when using RpcClientImpl then we wrap the exception in IOE
and throw it,
> {noformat}
> 2015-11-16 04:05:24,935 WARN [main-EventThread.replicationSource,peer2] regionserver.HBaseInterClusterReplicationEndpoint:
Can't replicate because of a local or network error:
> java.io.IOException: Call to host-XX:16040 failed on local exception: org.apache.hadoop.hbase.ipc.CallTimeoutException:
Call id=510, waitTime=180001, operationTimeout=180000 expired.
> at org.apache.hadoop.hbase.ipc.RpcClientImpl.wrapException(RpcClientImpl.java:1271)
> at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1239)
> at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:213)
> at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:287)
> at org.apache.hadoop.hbase.protobuf.generated.AdminProtos$AdminService$BlockingStub.replicateWALEntry(AdminProtos.java:25690)
> at org.apache.hadoop.hbase.protobuf.ReplicationProtbufUtil.replicateWALEntry(ReplicationProtbufUtil.java:77)
> at org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint$Replicator.call(HBaseInterClusterReplicationEndpoint.java:322)
> at org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint$Replicator.call(HBaseInterClusterReplicationEndpoint.java:308)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hadoop.hbase.ipc.CallTimeoutException: Call id=510, waitTime=180001,
operationTimeout=180000 expired.
> at org.apache.hadoop.hbase.ipc.Call.checkAndSetTimeout(Call.java:70)
> at org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1213)
> ... 10 more
> {noformat}
> But that isn't case with AsyncRpcClient, we don't wrap and throw CallTimeoutException
as it is.
> {noformat}
> 2015-12-21 14:27:33,093 WARN  [RS_OPEN_REGION-host-XX:16201-0.replicationSource.host-XX%2C16201%2C1450687255593,1]
regionserver.HBaseInterClusterReplicationEndpoint: Can't replicate because of a local or network
error: 
> org.apache.hadoop.hbase.ipc.CallTimeoutException: callId=2, method=ReplicateWALEntry,
rpcTimeout=600000, param {TODO: class org.apache.hadoop.hbase.protobuf.generated.AdminProtos$ReplicateWALEntryRequest}
> 	at org.apache.hadoop.hbase.ipc.AsyncRpcClient.call(AsyncRpcClient.java:257)
> 	at org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:217)
> 	at org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:295)
> 	at org.apache.hadoop.hbase.protobuf.generated.AdminProtos$AdminService$BlockingStub.replicateWALEntry(AdminProtos.java:23707)
> 	at org.apache.hadoop.hbase.protobuf.ReplicationProtbufUtil.replicateWALEntry(ReplicationProtbufUtil.java:73)
> 	at org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint$Replicator.call(HBaseInterClusterReplicationEndpoint.java:387)
> 	at org.apache.hadoop.hbase.replication.regionserver.HBaseInterClusterReplicationEndpoint$Replicator.call(HBaseInterClusterReplicationEndpoint.java:370)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> 	at java.lang.Thread.run(Thread.java:745)
> {noformat}
> I think we should have same behavior across both the implementations.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message