hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "stack (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-8764) Some MasterMonitorCallable should retry
Date Wed, 24 Jul 2013 21:03:50 GMT

     [ https://issues.apache.org/jira/browse/HBASE-8764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

stack updated HBASE-8764:
-------------------------

    Attachment: 8764v3.txt

Add master retying.  Uses same mechanism as regionserver retrying.  Add test too of the exception
seen here proving we now retry.

Main change is in HBaseAdmin where we get rid of executable methods and all do executeCallable
instead.  Here is main change:

{code}
  9 -  private <V> V executeCallable(Callable<V> function) throws IOException
{
 10 +  private <V> V executeCallable(MasterCallable<V> callable) throws IOException
{
 11 +    RpcRetryingCaller<V> caller = new RpcRetryingCaller<V>();
 12      try {
 13 -      return function.call();
 14 -    } catch (RemoteException re) {
 15 -      throw re.unwrapRemoteException();
 16 -    } catch (IOException e) {
 17 -      throw e;
 18 -    } catch (ServiceException se) {
 19 -      throw ProtobufUtil.getRemoteException(se);
 20 -    } catch (Exception e) {
 21 -      // This should not happen...
 22 -      throw new IOException("Unexpected exception when calling master", e);
 23 +      return caller.callWithRetries(callable, getConfiguration());
 24 +    } finally {
 25 +      callable.close();
 26      }
 27    }
{code}

The exceptions are handled inside in the callWithRetries.
                
> Some MasterMonitorCallable should retry
> ---------------------------------------
>
>                 Key: HBASE-8764
>                 URL: https://issues.apache.org/jira/browse/HBASE-8764
>             Project: HBase
>          Issue Type: Bug
>          Components: IPC/RPC
>    Affects Versions: 0.95.1
>            Reporter: Elliott Clark
>            Assignee: stack
>             Fix For: 0.95.2
>
>         Attachments: 8764.txt, 8764v2.txt, 8764v3.txt
>
>
> Calls in the admin that only get status should re-try.
> got a call stack like:
> {code}
> org.apache.hadoop.hbase.exceptions.PleaseHoldException: org.apache.hadoop.hbase.exceptions.PleaseHoldException:
Master is initializing
> 	at org.apache.hadoop.hbase.master.HMaster.checkInitialized(HMaster.java:2266)
> 	at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1610)
> 	at org.apache.hadoop.hbase.master.HMaster.createTable(HMaster.java:1646)
> 	at org.apache.hadoop.hbase.protobuf.generated.MasterAdminProtos$MasterAdminService$2.callBlockingMethod(MasterAdminProtos.java:20930)
> 	at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2122)
> 	at org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1829)
> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
> 	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> 	at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
> 	at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:90)
> 	at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:79)
> 	at org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRemoteException(ProtobufUtil.java:230)
> 	at org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:2705)
> 	at org.apache.hadoop.hbase.client.HBaseAdmin.execute(HBaseAdmin.java:2674)
> 	at org.apache.hadoop.hbase.client.HBaseAdmin.createTableAsync(HBaseAdmin.java:524)
> 	at org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:417)
> 	at org.apache.hadoop.hbase.client.HBaseAdmin.createTable(HBaseAdmin.java:349)
> 	at org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Generator.createSchema(IntegrationTestBigLinkedList.java:437)
> 	at org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Generator.runGenerator(IntegrationTestBigLinkedList.java:471)
> 	at org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Generator.run(IntegrationTestBigLinkedList.java:505)
> 	at org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Loop.runGenerator(IntegrationTestBigLinkedList.java:698)
> 	at org.apache.hadoop.hbase.test.IntegrationTestBigLinkedList$Loop.run(IntegrationTestBigLinkedList.java:748)
> 	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> 	at org.apache.hadoop.hbase.test.IntegrationTestBigLinkedListWithChaosMonkey.testContinuousIngest(IntegrationTestBigLinkedListWithChaosMonkey.java:80)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> 	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> 	at java.lang.reflect.Method.invoke(Method.java:601)
> 	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
> 	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> 	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
> 	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> 	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
> 	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
> 	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
> 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
> 	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
> 	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> 	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> 	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> 	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> 	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> 	at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> 	at org.junit.runners.Suite.runChild(Suite.java:127)
> 	at org.junit.runners.Suite.runChild(Suite.java:26)
> 	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
> 	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
> 	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
> 	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
> 	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
> 	at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> 	at org.junit.runner.JUnitCore.run(JUnitCore.java:160)
> 	at org.junit.runner.JUnitCore.run(JUnitCore.java:138)
> 	at org.junit.runner.JUnitCore.run(JUnitCore.java:117)
> 	at org.apache.hadoop.hbase.IntegrationTestsDriver.doWork(IntegrationTestsDriver.java:111)
> 	at org.apache.hadoop.hbase.util.AbstractHBaseTool.run(AbstractHBaseTool.java:108)
> 	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
> 	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
> 	at org.apache.hadoop.hbase.IntegrationTestsDriver.main(IntegrationTestsDriver.java:47)
> {code}

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message