hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "王仲远" <zhywangch...@163.com>
Subject Problems with Hbase Restart
Date Wed, 13 Jan 2010 12:32:42 GMT
I have two problems when I use hbase 0.20.2.

I used several machines to make a cluster and deployed hadoop and hbase on them. I followed
the "Getting Started" and installed Hbase 0.20.2. I could start hbase sucessfully and created
table and inserted data.

However, when I accessed the master page: http://10.37.17.252:60010/master.jsp , I found the
first problem: the master node also appear on Region Servers and appear twice. But I didn't
set the master node as regionserver in conf/regionservers: 
Region Servers
Address Start Code Load 
127.0.0.1:60030 1263383321075 requests=0, regions=0, usedHeap=0, maxHeap=0 
127.0.0.1:60030 1263383321096 requests=0, regions=0, usedHeap=0, maxHeap=0 
WAMDM1.ruc.edu.cn:60030 1263383350174 requests=0, regions=0, usedHeap=24, maxHeap=991 
WAMDM2.ruc.edu.cn:60030 1263383320980 requests=0, regions=1, usedHeap=32, maxHeap=991 
WAMDM3.ruc.edu.cn:60030 1263383320985 requests=0, regions=1, usedHeap=33, maxHeap=991 
WAMDM4.ruc.edu.cn:60030 1263383322246 requests=0, regions=0, usedHeap=33, maxHeap=991 

However, the hbase seemed to work fine. But When I try to restart the hbase, I found the second
problem: I run bin/stop-hbase.sh and then run bin/ start-hbase.sh. But at this time, the hbase
go down. When I tried to access the master page: http://10.37.17.252:60010/master.jsp , I
got the following error message:

HTTP ERROR: 500
Trying to contact region server null for region , row '', but failed after 3 attempts.
Exceptions:
org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed setting up proxy to /10.37.17.248:60020
after attempts=1
org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed setting up proxy to /10.37.17.248:60020
after attempts=1
org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed setting up proxy to /10.37.17.248:60020
after attempts=1

RequestURI=/master.jsp

Caused by:
org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to contact region server
null for region , row '', but failed after 3 attempts.
Exceptions:
org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed setting up proxy to /10.37.17.248:60020
after attempts=1
org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed setting up proxy to /10.37.17.248:60020
after attempts=1
org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed setting up proxy to /10.37.17.248:60020
after attempts=1

	at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:1002)
	at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:55)
	at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:28)
	at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.listTables(HConnectionManager.java:433)
	at org.apache.hadoop.hbase.client.HBaseAdmin.listTables(HBaseAdmin.java:127)
	at org.apache.hadoop.hbase.generated.master.master_jsp._jspService(master_jsp.java:125)
	at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
	at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:502)
	at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:363)
	at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
	at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
	at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
	at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:417)
	at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
	at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
	at org.mortbay.jetty.Server.handle(Server.java:324)
	at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:534)
	at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:864)
	at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:533)
	at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:207)
	at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:403)
	at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
	at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:522)

Powered by Jetty://


The error messages in the master log:
2010-01-13 18:34:04,424 DEBUG org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Read ZNode
/hbase/root-region-server got 10.37.17.248:60020
2010-01-13 18:34:04,425 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /10.37.17.248:60020
could not be reached after 1 tries, giving up.
2010-01-13 18:34:04,425 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$TableServers:
Root region location changed. Sleeping.
2010-01-13 18:34:06,425 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$TableServers:
Wake. Retry finding root region.
2010-01-13 18:34:06,429 DEBUG org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Read ZNode
/hbase/root-region-server got 10.37.17.248:60020
2010-01-13 18:34:06,430 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /10.37.17.248:60020
could not be reached after 1 tries, giving up.
2010-01-13 18:34:06,430 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$TableServers:
Root region location changed. Sleeping.
2010-01-13 18:34:08,430 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$TableServers:
Wake. Retry finding root region.
2010-01-13 18:34:08,434 DEBUG org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Read ZNode
/hbase/root-region-server got 10.37.17.248:60020
2010-01-13 18:34:08,435 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /10.37.17.248:60020
could not be reached after 1 tries, giving up.
2010-01-13 18:34:08,435 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$TableServers:
locateRegionInMeta attempt 0 of 3 failed; retrying after sleep of 2000
org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed setting up proxy to /10.37.17.248:60020
after attempts=1
        at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:424)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getHRegionConnection(HConnectionManager.java:865)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getHRegionConnection(HConnectionManager.java:881)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRootRegion(HConnectionManager.java:936)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:581)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:557)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegionInMeta(HConnectionManager.java:631)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:590)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:557)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionLocation(HConnectionManager.java:407)
        at org.apache.hadoop.hbase.client.ServerCallable.instantiateServer(ServerCallable.java:57)
        at org.apache.hadoop.hbase.client.ScannerCallable.instantiateServer(ScannerCallable.java:62)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:988)
        at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:55)
        at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:28)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.listTables(HConnectionManager.java:433)
        at org.apache.hadoop.hbase.client.HBaseAdmin.listTables(HBaseAdmin.java:127)
        at org.apache.hadoop.hbase.generated.master.master_jsp._jspService(master_jsp.java:125)
        at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
        at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:502)
        at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:363)
        at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
        at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
        at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
        at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:417)
        at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
        at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
        at org.mortbay.jetty.Server.handle(Server.java:324)
        at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:534)
        at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:864)
        at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:533)
        at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:207)
        at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:403)
        at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
        at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:522)
2010-01-13 18:34:08,451 DEBUG org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Read ZNode
/hbase/root-region-server got 10.37.17.248:60020
2010-01-13 18:34:08,451 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /10.37.17.248:60020
could not be reached after 1 tries, giving up.
2010-01-13 18:34:08,451 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$TableServers:
Root region location changed. Sleeping.
2010-01-13 18:34:10,451 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$TableServers:
Wake. Retry finding root region.
2010-01-13 18:34:10,456 DEBUG org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Read ZNode
/hbase/root-region-server got 10.37.17.248:60020
2010-01-13 18:34:10,465 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /10.37.17.248:60020
could not be reached after 1 tries, giving up.
2010-01-13 18:34:10,465 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$TableServers:
Root region location changed. Sleeping.
2010-01-13 18:34:12,465 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$TableServers:
Wake. Retry finding root region.
2010-01-13 18:34:12,469 DEBUG org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Read ZNode
/hbase/root-region-server got 10.37.17.248:60020
2010-01-13 18:34:12,470 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /10.37.17.248:60020
could not be reached after 1 tries, giving up.
2010-01-13 18:34:14,474 DEBUG org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Read ZNode
/hbase/root-region-server got 10.37.17.248:60020
2010-01-13 18:34:14,475 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /10.37.17.248:60020
could not be reached after 1 tries, giving up.
2010-01-13 18:34:14,475 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$TableServers:
Root region location changed. Sleeping.
2010-01-13 18:34:16,475 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$TableServers:
Wake. Retry finding root region.
2010-01-13 18:34:16,480 DEBUG org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Read ZNode
/hbase/root-region-server got 10.37.17.248:60020
2010-01-13 18:34:16,480 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /10.37.17.248:60020
could not be reached after 1 tries, giving up.
2010-01-13 18:34:16,480 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$TableServers:
Root region location changed. Sleeping.
2010-01-13 18:34:18,480 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$TableServers:
Wake. Retry finding root region.
2010-01-13 18:34:18,485 DEBUG org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Read ZNode
/hbase/root-region-server got 10.37.17.248:60020
2010-01-13 18:34:18,486 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /10.37.17.248:60020
could not be reached after 1 tries, giving up.
2010-01-13 18:34:18,486 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$TableServers:
locateRegionInMeta attempt 0 of 3 failed; retrying after sleep of 2000
org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed setting up proxy to /10.37.17.248:60020
after attempts=1
        at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:424)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getHRegionConnection(HConnectionManager.java:865)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getHRegionConnection(HConnectionManager.java:881)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRootRegion(HConnectionManager.java:936)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:581)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:557)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegionInMeta(HConnectionManager.java:631)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:590)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.relocateRegion(HConnectionManager.java:563)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionLocation(HConnectionManager.java:407)
        at org.apache.hadoop.hbase.client.ServerCallable.instantiateServer(ServerCallable.java:57)
        at org.apache.hadoop.hbase.client.ScannerCallable.instantiateServer(ScannerCallable.java:62)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:988)
        at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:55)
        at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:28)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.listTables(HConnectionManager.java:433)
        at org.apache.hadoop.hbase.client.HBaseAdmin.listTables(HBaseAdmin.java:127)
        at org.apache.hadoop.hbase.generated.master.master_jsp._jspService(master_jsp.java:125)
        at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
        at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:502)
        at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:363)
        at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
        at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
        at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
        at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:417)
        at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
        at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
        at org.mortbay.jetty.Server.handle(Server.java:324)
        at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:534)
        at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:864)
        at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:533)
        at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:207)
        at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:403)
        at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
        at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:522)
2010-01-13 18:34:18,500 DEBUG org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Read ZNode
/hbase/root-region-server got 10.37.17.248:60020
2010-01-13 18:34:18,501 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /10.37.17.248:60020
could not be reached after 1 tries, giving up.
2010-01-13 18:34:18,501 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$TableServers:
Root region location changed. Sleeping.
2010-01-13 18:34:20,501 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$TableServers:
Wake. Retry finding root region.
2010-01-13 18:34:20,506 DEBUG org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Read ZNode
/hbase/root-region-server got 10.37.17.248:60020
2010-01-13 18:34:20,506 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /10.37.17.248:60020
could not be reached after 1 tries, giving up.
2010-01-13 18:34:20,506 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$TableServers:
Root region location changed. Sleeping.
2010-01-13 18:34:22,507 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$TableServers:
Wake. Retry finding root region.
2010-01-13 18:34:22,511 DEBUG org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Read ZNode
/hbase/root-region-server got 10.37.17.248:60020
2010-01-13 18:34:22,511 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /10.37.17.248:60020
could not be reached after 1 tries, giving up.
2010-01-13 18:34:24,516 DEBUG org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Read ZNode
/hbase/root-region-server got 10.37.17.248:60020
2010-01-13 18:34:24,516 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /10.37.17.248:60020
could not be reached after 1 tries, giving up.
2010-01-13 18:34:24,516 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$TableServers:
Root region location changed. Sleeping.
2010-01-13 18:34:26,517 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$TableServers:
Wake. Retry finding root region.
2010-01-13 18:34:26,521 DEBUG org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Read ZNode
/hbase/root-region-server got 10.37.17.248:60020
2010-01-13 18:34:26,522 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /10.37.17.248:60020
could not be reached after 1 tries, giving up.
2010-01-13 18:34:26,522 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$TableServers:
Root region location changed. Sleeping.
2010-01-13 18:34:28,522 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$TableServers:
Wake. Retry finding root region.
2010-01-13 18:34:28,526 DEBUG org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Read ZNode
/hbase/root-region-server got 10.37.17.248:60020
2010-01-13 18:34:28,527 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /10.37.17.248:60020
could not be reached after 1 tries, giving up.
2010-01-13 18:34:28,527 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$TableServers:
locateRegionInMeta attempt 0 of 3 failed; retrying after sleep of 2000
org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed setting up proxy to /10.37.17.248:60020
after attempts=1
        at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:424)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getHRegionConnection(HConnectionManager.java:865)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getHRegionConnection(HConnectionManager.java:881)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRootRegion(HConnectionManager.java:936)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:581)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:557)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegionInMeta(HConnectionManager.java:631)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:590)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.relocateRegion(HConnectionManager.java:563)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionLocation(HConnectionManager.java:407)
        at org.apache.hadoop.hbase.client.ServerCallable.instantiateServer(ServerCallable.java:57)
        at org.apache.hadoop.hbase.client.ScannerCallable.instantiateServer(ScannerCallable.java:62)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:988)
        at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:55)
        at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:28)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.listTables(HConnectionManager.java:433)
        at org.apache.hadoop.hbase.client.HBaseAdmin.listTables(HBaseAdmin.java:127)
        at org.apache.hadoop.hbase.generated.master.master_jsp._jspService(master_jsp.java:125)
        at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
        at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:502)
        at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:363)
        at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
        at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
        at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
        at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:417)
        at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
        at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
        at org.mortbay.jetty.Server.handle(Server.java:324)
        at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:534)
        at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:864)
        at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:533)
        at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:207)
        at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:403)
        at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
        at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:522)
2010-01-13 18:34:28,547 DEBUG org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Read ZNode
/hbase/root-region-server got 10.37.17.248:60020
2010-01-13 18:34:28,547 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /10.37.17.248:60020
could not be reached after 1 tries, giving up.
2010-01-13 18:34:28,547 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$TableServers:
Root region location changed. Sleeping.
2010-01-13 18:34:30,547 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$TableServers:
Wake. Retry finding root region.
2010-01-13 18:34:30,552 DEBUG org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Read ZNode
/hbase/root-region-server got 10.37.17.248:60020
2010-01-13 18:34:30,552 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /10.37.17.248:60020
could not be reached after 1 tries, giving up.
2010-01-13 18:34:30,552 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$TableServers:
Root region location changed. Sleeping.
2010-01-13 18:34:31,575 INFO org.apache.hadoop.hbase.master.ServerManager: 5 region servers,
0 dead, average load 0.16666666666666666
2010-01-13 18:34:32,552 DEBUG org.apache.hadoop.hbase.client.HConnectionManager$TableServers:
Wake. Retry finding root region.
2010-01-13 18:34:32,557 DEBUG org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Read ZNode
/hbase/root-region-server got 10.37.17.248:60020
2010-01-13 18:34:32,558 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /10.37.17.248:60020
could not be reached after 1 tries, giving up.
2010-01-13 18:34:32,740 INFO org.apache.hadoop.hbase.master.BaseScanner: RegionManager.rootScanner
scanning meta region {server: 10.37.17.248:60020, regionname: -ROOT-,,0, startKey: <>}
2010-01-13 18:34:32,741 INFO org.apache.hadoop.hbase.master.BaseScanner: All 0 .META. region(s)
scanned
2010-01-13 18:34:32,741 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /10.37.17.248:60020
could not be reached after 1 tries, giving up.
2010-01-13 18:34:32,741 WARN org.apache.hadoop.hbase.master.BaseScanner: Scan ROOT region
org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed setting up proxy to /10.37.17.248:60020
after attempts=1
        at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:424)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getHRegionConnection(HConnectionManager.java:865)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getHRegionConnection(HConnectionManager.java:881)
        at org.apache.hadoop.hbase.master.BaseScanner.scanRegion(BaseScanner.java:156)
        at org.apache.hadoop.hbase.master.RootScanner.scanRoot(RootScanner.java:54)
        at org.apache.hadoop.hbase.master.RootScanner.maintenanceScan(RootScanner.java:79)
        at org.apache.hadoop.hbase.master.BaseScanner.chore(BaseScanner.java:136)
        at org.apache.hadoop.hbase.Chore.run(Chore.java:68)
2010-01-13 18:35:31,575 INFO org.apache.hadoop.hbase.master.ServerManager: 5 region servers,
0 dead, average load 0.16666666666666666
2010-01-13 18:35:32,740 INFO org.apache.hadoop.hbase.master.BaseScanner: RegionManager.rootScanner
scanning meta region {server: 10.37.17.248:60020, regionname: -ROOT-,,0, startKey: <>}
2010-01-13 18:35:32,741 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /10.37.17.248:60020
could not be reached after 1 tries, giving up.
2010-01-13 18:35:32,741 WARN org.apache.hadoop.hbase.master.BaseScanner: Scan ROOT region
org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed setting up proxy to /10.37.17.248:60020
after attempts=1
        at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:424)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getHRegionConnection(HConnectionManager.java:865)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getHRegionConnection(HConnectionManager.java:881)
        at org.apache.hadoop.hbase.master.BaseScanner.scanRegion(BaseScanner.java:156)
        at org.apache.hadoop.hbase.master.RootScanner.scanRoot(RootScanner.java:54)
        at org.apache.hadoop.hbase.master.RootScanner.maintenanceScan(RootScanner.java:79)
        at org.apache.hadoop.hbase.master.BaseScanner.chore(BaseScanner.java:136)
        at org.apache.hadoop.hbase.Chore.run(Chore.java:68)
2010-01-13 18:35:32,742 INFO org.apache.hadoop.hbase.master.BaseScanner: All 0 .META. region(s)
scanned
2010-01-13 18:36:31,575 INFO org.apache.hadoop.hbase.master.ServerManager: 5 region servers,
0 dead, average load 0.16666666666666666
2010-01-13 18:36:32,740 INFO org.apache.hadoop.hbase.master.BaseScanner: RegionManager.rootScanner
scanning meta region {server: 10.37.17.248:60020, regionname: -ROOT-,,0, startKey: <>}
2010-01-13 18:36:32,741 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /10.37.17.248:60020
could not be reached after 1 tries, giving up.
2010-01-13 18:36:32,741 WARN org.apache.hadoop.hbase.master.BaseScanner: Scan ROOT region
org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed setting up proxy to /10.37.17.248:60020
after attempts=1
        at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:424)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getHRegionConnection(HConnectionManager.java:865)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getHRegionConnection(HConnectionManager.java:881)
        at org.apache.hadoop.hbase.master.BaseScanner.scanRegion(BaseScanner.java:156)
        at org.apache.hadoop.hbase.master.RootScanner.scanRoot(RootScanner.java:54)
        at org.apache.hadoop.hbase.master.RootScanner.maintenanceScan(RootScanner.java:79)
        at org.apache.hadoop.hbase.master.BaseScanner.chore(BaseScanner.java:136)
        at org.apache.hadoop.hbase.Chore.run(Chore.java:68)
2010-01-13 18:36:32,742 INFO org.apache.hadoop.hbase.master.BaseScanner: All 0 .META. region(s)
scanned
2010-01-13 18:37:31,575 INFO org.apache.hadoop.hbase.master.ServerManager: 5 region servers,
0 dead, average load 0.16666666666666666
2010-01-13 18:37:32,741 INFO org.apache.hadoop.hbase.master.BaseScanner: RegionManager.rootScanner
scanning meta region {server: 10.37.17.248:60020, regionname: -ROOT-,,0, startKey: <>}
2010-01-13 18:37:32,741 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /10.37.17.248:60020
could not be reached after 1 tries, giving up.
2010-01-13 18:37:32,741 WARN org.apache.hadoop.hbase.master.BaseScanner: Scan ROOT region
org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed setting up proxy to /10.37.17.248:60020
after attempts=1
        at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:424)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getHRegionConnection(HConnectionManager.java:865)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getHRegionConnection(HConnectionManager.java:881)
        at org.apache.hadoop.hbase.master.BaseScanner.scanRegion(BaseScanner.java:156)
        at org.apache.hadoop.hbase.master.RootScanner.scanRoot(RootScanner.java:54)
        at org.apache.hadoop.hbase.master.RootScanner.maintenanceScan(RootScanner.java:79)
        at org.apache.hadoop.hbase.master.BaseScanner.chore(BaseScanner.java:136)
        at org.apache.hadoop.hbase.Chore.run(Chore.java:68)
2010-01-13 18:37:32,742 INFO org.apache.hadoop.hbase.master.BaseScanner: All 0 .META. region(s)
scanned
2010-01-13 18:38:31,575 INFO org.apache.hadoop.hbase.master.ServerManager: 5 region servers,
0 dead, average load 0.16666666666666666
2010-01-13 18:38:32,740 INFO org.apache.hadoop.hbase.master.BaseScanner: RegionManager.rootScanner
scanning meta region {server: 10.37.17.248:60020, regionname: -ROOT-,,0, startKey: <>}
2010-01-13 18:38:32,741 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /10.37.17.248:60020
could not be reached after 1 tries, giving up.
2010-01-13 18:38:32,741 WARN org.apache.hadoop.hbase.master.BaseScanner: Scan ROOT region
org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed setting up proxy to /10.37.17.248:60020
after attempts=1
        at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:424)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getHRegionConnection(HConnectionManager.java:865)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getHRegionConnection(HConnectionManager.java:881)
        at org.apache.hadoop.hbase.master.BaseScanner.scanRegion(BaseScanner.java:156)
        at org.apache.hadoop.hbase.master.RootScanner.scanRoot(RootScanner.java:54)
        at org.apache.hadoop.hbase.master.RootScanner.maintenanceScan(RootScanner.java:79)
        at org.apache.hadoop.hbase.master.BaseScanner.chore(BaseScanner.java:136)
        at org.apache.hadoop.hbase.Chore.run(Chore.java:68)
2010-01-13 18:38:32,742 INFO org.apache.hadoop.hbase.master.BaseScanner: All 0 .META. region(s)
scanned
2010-01-13 18:39:31,575 INFO org.apache.hadoop.hbase.master.ServerManager: 5 region servers,
0 dead, average load 0.16666666666666666
2010-01-13 18:39:32,740 INFO org.apache.hadoop.hbase.master.BaseScanner: RegionManager.rootScanner
scanning meta region {server: 10.37.17.248:60020, regionname: -ROOT-,,0, startKey: <>}
2010-01-13 18:39:32,741 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /10.37.17.248:60020
could not be reached after 1 tries, giving up.
2010-01-13 18:39:32,741 WARN org.apache.hadoop.hbase.master.BaseScanner: Scan ROOT region
org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed setting up proxy to /10.37.17.248:60020
after attempts=1
        at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:424)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getHRegionConnection(HConnectionManager.java:865)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getHRegionConnection(HConnectionManager.java:881)
        at org.apache.hadoop.hbase.master.BaseScanner.scanRegion(BaseScanner.java:156)
        at org.apache.hadoop.hbase.master.RootScanner.scanRoot(RootScanner.java:54)
        at org.apache.hadoop.hbase.master.RootScanner.maintenanceScan(RootScanner.java:79)
        at org.apache.hadoop.hbase.master.BaseScanner.chore(BaseScanner.java:136)
        at org.apache.hadoop.hbase.Chore.run(Chore.java:68)
2010-01-13 18:39:32,742 INFO org.apache.hadoop.hbase.master.BaseScanner: All 0 .META. region(s)
scanned
2010-01-13 18:40:31,575 INFO org.apache.hadoop.hbase.master.ServerManager: 5 region servers,
0 dead, average load 0.16666666666666666
2010-01-13 18:40:32,740 INFO org.apache.hadoop.hbase.master.BaseScanner: RegionManager.rootScanner
scanning meta region {server: 10.37.17.248:60020, regionname: -ROOT-,,0, startKey: <>}
2010-01-13 18:40:32,740 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /10.37.17.248:60020
could not be reached after 1 tries, giving up.
2010-01-13 18:40:32,741 WARN org.apache.hadoop.hbase.master.BaseScanner: Scan ROOT region
org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed setting up proxy to /10.37.17.248:60020
after attempts=1
        at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:424)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getHRegionConnection(HConnectionManager.java:865)
        at org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getHRegionConnection(HConnectionManager.java:881)
        at org.apache.hadoop.hbase.master.BaseScanner.scanRegion(BaseScanner.java:156)
        at org.apache.hadoop.hbase.master.RootScanner.scanRoot(RootScanner.java:54)
        at org.apache.hadoop.hbase.master.RootScanner.maintenanceScan(RootScanner.java:79)
        at org.apache.hadoop.hbase.master.BaseScanner.chore(BaseScanner.java:136)
        at org.apache.hadoop.hbase.Chore.run(Chore.java:68)
.....


Can somebody help me. Thank you.
------------------
Zhongyuan
2010-01-13



Mime
View raw message