hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gregory Chanan (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-6151) Master can die if RegionServer throws ServerNotRunningYet
Date Sat, 02 Jun 2012 00:54:23 GMT

     [ https://issues.apache.org/jira/browse/HBASE-6151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Gregory Chanan updated HBASE-6151:
----------------------------------

    Description: 
See, for example:

{noformat}
2012-05-23 16:49:22,745 FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception.
Starting shutdown.
org.apache.hadoop.hbase.ipc.ServerNotRunningException: org.apache.hadoop.hbase.ipc.ServerNotRunningException:
Server is not running yet
	at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1038)

	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
	at org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException(RemoteExceptionHandler.java:96)
	at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1240)
	at org.apache.hadoop.hbase.catalog.CatalogTracker.getCachedConnection(CatalogTracker.java:444)
	at org.apache.hadoop.hbase.catalog.CatalogTracker.getMetaServerConnection(CatalogTracker.java:343)
	at org.apache.hadoop.hbase.catalog.CatalogTracker.verifyMetaRegionLocation(CatalogTracker.java:540)
	at org.apache.hadoop.hbase.master.HMaster.assignRootAndMeta(HMaster.java:474)
	at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:412)
{noformat}

The HRegionServer calls HBaseServer:
{code}
  public void start() {
    startThreads();
    openServer();
  }
{code}

but the server can start accepting RPCs once the threads have been started, but if they do,
they throw ServerNotRunningException until openServer runs.  We should probably
1) Catch the remote exception and retry on the master
2) Look into whether the start() behavior of HBaseServer makes any sense.  Why would you start
accepting RPCs only to throw back ServerNotRunningException?


  was:
See, for example:

{noformat}
2012-05-23 16:49:22,745 FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception.
Starting shutdown.
org.apache.hadoop.hbase.ipc.ServerNotRunningException: org.apache.hadoop.hbase.ipc.ServerNotRunningException:
Server is not running yet
	at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1038)

	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
	at org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException(RemoteExceptionHandler.java:96)
	at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1240)
	at org.apache.hadoop.hbase.catalog.CatalogTracker.getCachedConnection(CatalogTracker.java:444)
	at org.apache.hadoop.hbase.catalog.CatalogTracker.getMetaServerConnection(CatalogTracker.java:343)
	at org.apache.hadoop.hbase.catalog.CatalogTracker.verifyMetaRegionLocation(CatalogTracker.java:540)
	at org.apache.hadoop.hbase.master.HMaster.assignRootAndMeta(HMaster.java:474)
	at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:412)
{noformat}

{code}
The HRegionServer calls HBaseServer:
  public void start() {
    startThreads();
    openServer();
  }
{code}

but the server can start accepting RPCs once the threads have been started, but if they do,
they throw ServerNotRunningException until openServer runs.  We should probably
1) Catch the remote exception and retry on the master
2) Look into whether the start() behavior of HBaseServer makes any sense.  Why would you start
accepting RPCs only to throw back ServerNotRunningException?


    
> Master can die if RegionServer throws ServerNotRunningYet
> ---------------------------------------------------------
>
>                 Key: HBASE-6151
>                 URL: https://issues.apache.org/jira/browse/HBASE-6151
>             Project: HBase
>          Issue Type: Bug
>          Components: ipc
>    Affects Versions: 0.90.7, 0.92.2, 0.96.0, 0.94.1
>            Reporter: Gregory Chanan
>            Assignee: Gregory Chanan
>
> See, for example:
> {noformat}
> 2012-05-23 16:49:22,745 FATAL org.apache.hadoop.hbase.master.HMaster: Unhandled exception.
Starting shutdown.
> org.apache.hadoop.hbase.ipc.ServerNotRunningException: org.apache.hadoop.hbase.ipc.ServerNotRunningException:
Server is not running yet
> 	at org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1038)
> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> 	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
> 	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> 	at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> 	at org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException(RemoteExceptionHandler.java:96)
> 	at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getHRegionConnection(HConnectionManager.java:1240)
> 	at org.apache.hadoop.hbase.catalog.CatalogTracker.getCachedConnection(CatalogTracker.java:444)
> 	at org.apache.hadoop.hbase.catalog.CatalogTracker.getMetaServerConnection(CatalogTracker.java:343)
> 	at org.apache.hadoop.hbase.catalog.CatalogTracker.verifyMetaRegionLocation(CatalogTracker.java:540)
> 	at org.apache.hadoop.hbase.master.HMaster.assignRootAndMeta(HMaster.java:474)
> 	at org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:412)
> {noformat}
> The HRegionServer calls HBaseServer:
> {code}
>   public void start() {
>     startThreads();
>     openServer();
>   }
> {code}
> but the server can start accepting RPCs once the threads have been started, but if they
do, they throw ServerNotRunningException until openServer runs.  We should probably
> 1) Catch the remote exception and retry on the master
> 2) Look into whether the start() behavior of HBaseServer makes any sense.  Why would
you start accepting RPCs only to throw back ServerNotRunningException?

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message