hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From stack <st...@duboce.net>
Subject Re: Re: Problems with Hbase Restart
Date Thu, 14 Jan 2010 19:24:18 GMT
Its an exception thrown when you have logging set to DEBUG. Its just an
emission to give a sense of where zk setup is during startup.  The issue of
nonode for the master is soon resolved after master succeeds writing its
address into zk and startup proceeds.

You are not the first to finger this exception as possible issue.  Its
confusing.  Let us try and change this one at least before we cut the new
0.20.3RC.

St.Ack

2010/1/14 王仲远 <zhywangchina@163.com>

> Hi, J-D,
>   Thank you for your response and suggestion. I found the problem was
> caused by /etc/hosts in 10.37.248. In the file, the first line is:
> 127.0.0.1       WAMDM1  localhost.localdomain   localhost
> and We used the hostname in our regionservers file.
>
> Then I had corrected the hosts file and solved the problems.
>
> However, when I restarted the hbase, I get some exceptions as follow:
> java.io.IOException: org.apache.zookeeper.KeeperException$NoNodeException:
> KeeperErrorCode = NoNode for /hbase/master
>        at
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.readAddressOrThrow(ZooKeeperWrapper.java:332)
>        at
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.readAddress(ZooKeeperWrapper.java:318)
>        at
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper.readMasterAddress(ZooKeeperWrapper.java:250)
>        at
> org.apache.hadoop.hbase.master.ZKMasterAddressWatcher.waitForMasterAddressAvailability(ZKMasterAddressWatcher.java:82)
>        at
> org.apache.hadoop.hbase.master.HMaster.writeAddressToZooKeeper(HMaster.java:267)
>        at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:254)
>        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
>        at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
>        at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
>
> But it seemed that Hbase still worked well.  What's the matter with this
> exception?
>
> Any suggestion? Thank you~~
>
>
> Zhongyuan
> 2010-01-15
>
>
> -------------------------------------------------------------
> From: Jean-Daniel Cryans
> Sent: 2010-01-14 01:40:33
> To: hbase-user
> CC:
> Subject: Re: Problems with Hbase Restart
>
> It seems your master's region server reported itself as 127.0.0.1
> Then there's something wrong with 10.37.17.248, you should check this
> machine's region server log.
> J-D
> 2010/1/13 王仲远 <zhywangchina@163.com>:
> > I have two problems when I use hbase 0.20.2.
> >
> > I used several machines to make a cluster and deployed hadoop and hbase
> on them. I followed the "Getting Started" and installed Hbase 0.20.2. I
> could start hbase sucessfully and created table and inserted data.
> >
> > However, when I accessed the master page:
> http://10.37.17.252:60010/master.jsp , I found the first problem: the
> master node also appear on Region Servers and appear twice. But I didn't set
> the master node as regionserver in conf/regionservers:
> > Region Servers
> > Address Start Code Load
> > 127.0.0.1:60030 1263383321075 requests=0, regions=0, usedHeap=0,
> maxHeap=0
> > 127.0.0.1:60030 1263383321096 requests=0, regions=0, usedHeap=0,
> maxHeap=0
> > WAMDM1.ruc.edu.cn:60030 1263383350174 requests=0, regions=0,
> usedHeap=24, maxHeap=991
> > WAMDM2.ruc.edu.cn:60030 1263383320980 requests=0, regions=1,
> usedHeap=32, maxHeap=991
> > WAMDM3.ruc.edu.cn:60030 1263383320985 requests=0, regions=1,
> usedHeap=33, maxHeap=991
> > WAMDM4.ruc.edu.cn:60030 1263383322246 requests=0, regions=0,
> usedHeap=33, maxHeap=991
> >
> > However, the hbase seemed to work fine. But When I try to restart the
> hbase, I found the second problem: I run bin/stop-hbase.sh and then run bin/
> start-hbase.sh. But at this time, the hbase go down. When I tried to access
> the master page: http://10.37.17.252:60010/master.jsp , I got the
> following error message:
> >
> > HTTP ERROR: 500
> > Trying to contact region server null for region , row '', but failed
> after 3 attempts.
> > Exceptions:
> > org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed setting
> up proxy to /10.37.17.248:60020 after attempts=1
> > org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed setting
> up proxy to /10.37.17.248:60020 after attempts=1
> > org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed setting
> up proxy to /10.37.17.248:60020 after attempts=1
> >
> > RequestURI=/master.jsp
> >
> > Caused by:
> > org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to
> contact region server null for region , row '', but failed after 3 attempts.
> > Exceptions:
> > org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed setting
> up proxy to /10.37.17.248:60020 after attempts=1
> > org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed setting
> up proxy to /10.37.17.248:60020 after attempts=1
> > org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed setting
> up proxy to /10.37.17.248:60020 after attempts=1
> >
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:1002)
> >        at
> org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:55)
> >        at
> org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:28)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.listTables(HConnectionManager.java:433)
> >        at
> org.apache.hadoop.hbase.client.HBaseAdmin.listTables(HBaseAdmin.java:127)
> >        at
> org.apache.hadoop.hbase.generated.master.master_jsp._jspService(master_jsp.java:125)
> >        at
> org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97)
> >        at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> >        at
> org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:502)
> >        at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:363)
> >        at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> >        at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
> >        at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
> >        at
> org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:417)
> >        at
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
> >        at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> >        at org.mortbay.jetty.Server.handle(Server.java:324)
> >        at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:534)
> >        at
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:864)
> >        at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:533)
> >        at
> org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:207)
> >        at
> org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:403)
> >        at
> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
> >        at
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:522)
> >
> > Powered by Jetty://
> >
> >
> > The error messages in the master log:
> > 2010-01-13 18:34:04,424 DEBUG
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Read ZNode
> /hbase/root-region-server got 10.37.17.248:60020
> > 2010-01-13 18:34:04,425 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /
> 10.37.17.248:60020 could not be reached after 1 tries, giving up.
> > 2010-01-13 18:34:04,425 DEBUG
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers: Root region
> location changed. Sleeping.
> > 2010-01-13 18:34:06,425 DEBUG
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers: Wake. Retry
> finding root region.
> > 2010-01-13 18:34:06,429 DEBUG
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Read ZNode
> /hbase/root-region-server got 10.37.17.248:60020
> > 2010-01-13 18:34:06,430 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /
> 10.37.17.248:60020 could not be reached after 1 tries, giving up.
> > 2010-01-13 18:34:06,430 DEBUG
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers: Root region
> location changed. Sleeping.
> > 2010-01-13 18:34:08,430 DEBUG
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers: Wake. Retry
> finding root region.
> > 2010-01-13 18:34:08,434 DEBUG
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Read ZNode
> /hbase/root-region-server got 10.37.17.248:60020
> > 2010-01-13 18:34:08,435 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /
> 10.37.17.248:60020 could not be reached after 1 tries, giving up.
> > 2010-01-13 18:34:08,435 DEBUG
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers:
> locateRegionInMeta attempt 0 of 3 failed; retrying after sleep of 2000
> > org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed setting
> up proxy to /10.37.17.248:60020 after attempts=1
> >        at
> org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:424)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getHRegionConnection(HConnectionManager.java:865)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getHRegionConnection(HConnectionManager.java:881)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRootRegion(HConnectionManager.java:936)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:581)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:557)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegionInMeta(HConnectionManager.java:631)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:590)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:557)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionLocation(HConnectionManager.java:407)
> >        at
> org.apache.hadoop.hbase.client.ServerCallable.instantiateServer(ServerCallable.java:57)
> >        at
> org.apache.hadoop.hbase.client.ScannerCallable.instantiateServer(ScannerCallable.java:62)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:988)
> >        at
> org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:55)
> >        at
> org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:28)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.listTables(HConnectionManager.java:433)
> >        at
> org.apache.hadoop.hbase.client.HBaseAdmin.listTables(HBaseAdmin.java:127)
> >        at
> org.apache.hadoop.hbase.generated.master.master_jsp._jspService(master_jsp.java:125)
> >        at
> org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97)
> >        at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> >        at
> org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:502)
> >        at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:363)
> >        at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> >        at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
> >        at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
> >        at
> org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:417)
> >        at
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
> >        at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> >        at org.mortbay.jetty.Server.handle(Server.java:324)
> >        at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:534)
> >        at
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:864)
> >        at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:533)
> >        at
> org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:207)
> >        at
> org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:403)
> >        at
> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
> >        at
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:522)
> > 2010-01-13 18:34:08,451 DEBUG
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Read ZNode
> /hbase/root-region-server got 10.37.17.248:60020
> > 2010-01-13 18:34:08,451 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /
> 10.37.17.248:60020 could not be reached after 1 tries, giving up.
> > 2010-01-13 18:34:08,451 DEBUG
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers: Root region
> location changed. Sleeping.
> > 2010-01-13 18:34:10,451 DEBUG
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers: Wake. Retry
> finding root region.
> > 2010-01-13 18:34:10,456 DEBUG
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Read ZNode
> /hbase/root-region-server got 10.37.17.248:60020
> > 2010-01-13 18:34:10,465 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /
> 10.37.17.248:60020 could not be reached after 1 tries, giving up.
> > 2010-01-13 18:34:10,465 DEBUG
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers: Root region
> location changed. Sleeping.
> > 2010-01-13 18:34:12,465 DEBUG
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers: Wake. Retry
> finding root region.
> > 2010-01-13 18:34:12,469 DEBUG
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Read ZNode
> /hbase/root-region-server got 10.37.17.248:60020
> > 2010-01-13 18:34:12,470 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /
> 10.37.17.248:60020 could not be reached after 1 tries, giving up.
> > 2010-01-13 18:34:14,474 DEBUG
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Read ZNode
> /hbase/root-region-server got 10.37.17.248:60020
> > 2010-01-13 18:34:14,475 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /
> 10.37.17.248:60020 could not be reached after 1 tries, giving up.
> > 2010-01-13 18:34:14,475 DEBUG
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers: Root region
> location changed. Sleeping.
> > 2010-01-13 18:34:16,475 DEBUG
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers: Wake. Retry
> finding root region.
> > 2010-01-13 18:34:16,480 DEBUG
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Read ZNode
> /hbase/root-region-server got 10.37.17.248:60020
> > 2010-01-13 18:34:16,480 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /
> 10.37.17.248:60020 could not be reached after 1 tries, giving up.
> > 2010-01-13 18:34:16,480 DEBUG
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers: Root region
> location changed. Sleeping.
> > 2010-01-13 18:34:18,480 DEBUG
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers: Wake. Retry
> finding root region.
> > 2010-01-13 18:34:18,485 DEBUG
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Read ZNode
> /hbase/root-region-server got 10.37.17.248:60020
> > 2010-01-13 18:34:18,486 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /
> 10.37.17.248:60020 could not be reached after 1 tries, giving up.
> > 2010-01-13 18:34:18,486 DEBUG
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers:
> locateRegionInMeta attempt 0 of 3 failed; retrying after sleep of 2000
> > org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed setting
> up proxy to /10.37.17.248:60020 after attempts=1
> >        at
> org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:424)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getHRegionConnection(HConnectionManager.java:865)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getHRegionConnection(HConnectionManager.java:881)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRootRegion(HConnectionManager.java:936)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:581)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:557)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegionInMeta(HConnectionManager.java:631)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:590)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.relocateRegion(HConnectionManager.java:563)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionLocation(HConnectionManager.java:407)
> >        at
> org.apache.hadoop.hbase.client.ServerCallable.instantiateServer(ServerCallable.java:57)
> >        at
> org.apache.hadoop.hbase.client.ScannerCallable.instantiateServer(ScannerCallable.java:62)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:988)
> >        at
> org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:55)
> >        at
> org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:28)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.listTables(HConnectionManager.java:433)
> >        at
> org.apache.hadoop.hbase.client.HBaseAdmin.listTables(HBaseAdmin.java:127)
> >        at
> org.apache.hadoop.hbase.generated.master.master_jsp._jspService(master_jsp.java:125)
> >        at
> org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97)
> >        at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> >        at
> org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:502)
> >        at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:363)
> >        at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> >        at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
> >        at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
> >        at
> org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:417)
> >        at
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
> >        at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> >        at org.mortbay.jetty.Server.handle(Server.java:324)
> >        at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:534)
> >        at
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:864)
> >        at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:533)
> >        at
> org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:207)
> >        at
> org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:403)
> >        at
> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
> >        at
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:522)
> > 2010-01-13 18:34:18,500 DEBUG
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Read ZNode
> /hbase/root-region-server got 10.37.17.248:60020
> > 2010-01-13 18:34:18,501 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /
> 10.37.17.248:60020 could not be reached after 1 tries, giving up.
> > 2010-01-13 18:34:18,501 DEBUG
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers: Root region
> location changed. Sleeping.
> > 2010-01-13 18:34:20,501 DEBUG
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers: Wake. Retry
> finding root region.
> > 2010-01-13 18:34:20,506 DEBUG
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Read ZNode
> /hbase/root-region-server got 10.37.17.248:60020
> > 2010-01-13 18:34:20,506 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /
> 10.37.17.248:60020 could not be reached after 1 tries, giving up.
> > 2010-01-13 18:34:20,506 DEBUG
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers: Root region
> location changed. Sleeping.
> > 2010-01-13 18:34:22,507 DEBUG
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers: Wake. Retry
> finding root region.
> > 2010-01-13 18:34:22,511 DEBUG
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Read ZNode
> /hbase/root-region-server got 10.37.17.248:60020
> > 2010-01-13 18:34:22,511 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /
> 10.37.17.248:60020 could not be reached after 1 tries, giving up.
> > 2010-01-13 18:34:24,516 DEBUG
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Read ZNode
> /hbase/root-region-server got 10.37.17.248:60020
> > 2010-01-13 18:34:24,516 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /
> 10.37.17.248:60020 could not be reached after 1 tries, giving up.
> > 2010-01-13 18:34:24,516 DEBUG
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers: Root region
> location changed. Sleeping.
> > 2010-01-13 18:34:26,517 DEBUG
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers: Wake. Retry
> finding root region.
> > 2010-01-13 18:34:26,521 DEBUG
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Read ZNode
> /hbase/root-region-server got 10.37.17.248:60020
> > 2010-01-13 18:34:26,522 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /
> 10.37.17.248:60020 could not be reached after 1 tries, giving up.
> > 2010-01-13 18:34:26,522 DEBUG
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers: Root region
> location changed. Sleeping.
> > 2010-01-13 18:34:28,522 DEBUG
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers: Wake. Retry
> finding root region.
> > 2010-01-13 18:34:28,526 DEBUG
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Read ZNode
> /hbase/root-region-server got 10.37.17.248:60020
> > 2010-01-13 18:34:28,527 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /
> 10.37.17.248:60020 could not be reached after 1 tries, giving up.
> > 2010-01-13 18:34:28,527 DEBUG
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers:
> locateRegionInMeta attempt 0 of 3 failed; retrying after sleep of 2000
> > org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed setting
> up proxy to /10.37.17.248:60020 after attempts=1
> >        at
> org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:424)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getHRegionConnection(HConnectionManager.java:865)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getHRegionConnection(HConnectionManager.java:881)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRootRegion(HConnectionManager.java:936)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:581)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:557)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegionInMeta(HConnectionManager.java:631)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.locateRegion(HConnectionManager.java:590)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.relocateRegion(HConnectionManager.java:563)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionLocation(HConnectionManager.java:407)
> >        at
> org.apache.hadoop.hbase.client.ServerCallable.instantiateServer(ServerCallable.java:57)
> >        at
> org.apache.hadoop.hbase.client.ScannerCallable.instantiateServer(ScannerCallable.java:62)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getRegionServerWithRetries(HConnectionManager.java:988)
> >        at
> org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:55)
> >        at
> org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:28)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.listTables(HConnectionManager.java:433)
> >        at
> org.apache.hadoop.hbase.client.HBaseAdmin.listTables(HBaseAdmin.java:127)
> >        at
> org.apache.hadoop.hbase.generated.master.master_jsp._jspService(master_jsp.java:125)
> >        at
> org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97)
> >        at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> >        at
> org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:502)
> >        at
> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:363)
> >        at
> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
> >        at
> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
> >        at
> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
> >        at
> org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:417)
> >        at
> org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
> >        at
> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
> >        at org.mortbay.jetty.Server.handle(Server.java:324)
> >        at
> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:534)
> >        at
> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:864)
> >        at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:533)
> >        at
> org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:207)
> >        at
> org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:403)
> >        at
> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
> >        at
> org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:522)
> > 2010-01-13 18:34:28,547 DEBUG
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Read ZNode
> /hbase/root-region-server got 10.37.17.248:60020
> > 2010-01-13 18:34:28,547 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /
> 10.37.17.248:60020 could not be reached after 1 tries, giving up.
> > 2010-01-13 18:34:28,547 DEBUG
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers: Root region
> location changed. Sleeping.
> > 2010-01-13 18:34:30,547 DEBUG
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers: Wake. Retry
> finding root region.
> > 2010-01-13 18:34:30,552 DEBUG
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Read ZNode
> /hbase/root-region-server got 10.37.17.248:60020
> > 2010-01-13 18:34:30,552 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /
> 10.37.17.248:60020 could not be reached after 1 tries, giving up.
> > 2010-01-13 18:34:30,552 DEBUG
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers: Root region
> location changed. Sleeping.
> > 2010-01-13 18:34:31,575 INFO
> org.apache.hadoop.hbase.master.ServerManager: 5 region servers, 0 dead,
> average load 0.16666666666666666
> > 2010-01-13 18:34:32,552 DEBUG
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers: Wake. Retry
> finding root region.
> > 2010-01-13 18:34:32,557 DEBUG
> org.apache.hadoop.hbase.zookeeper.ZooKeeperWrapper: Read ZNode
> /hbase/root-region-server got 10.37.17.248:60020
> > 2010-01-13 18:34:32,558 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /
> 10.37.17.248:60020 could not be reached after 1 tries, giving up.
> > 2010-01-13 18:34:32,740 INFO org.apache.hadoop.hbase.master.BaseScanner:
> RegionManager.rootScanner scanning meta region {server: 10.37.17.248:60020,
> regionname: -ROOT-,,0, startKey: <>}
> > 2010-01-13 18:34:32,741 INFO org.apache.hadoop.hbase.master.BaseScanner:
> All 0 .META. region(s) scanned
> > 2010-01-13 18:34:32,741 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /
> 10.37.17.248:60020 could not be reached after 1 tries, giving up.
> > 2010-01-13 18:34:32,741 WARN org.apache.hadoop.hbase.master.BaseScanner:
> Scan ROOT region
> > org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed setting
> up proxy to /10.37.17.248:60020 after attempts=1
> >        at
> org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:424)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getHRegionConnection(HConnectionManager.java:865)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getHRegionConnection(HConnectionManager.java:881)
> >        at
> org.apache.hadoop.hbase.master.BaseScanner.scanRegion(BaseScanner.java:156)
> >        at
> org.apache.hadoop.hbase.master.RootScanner.scanRoot(RootScanner.java:54)
> >        at
> org.apache.hadoop.hbase.master.RootScanner.maintenanceScan(RootScanner.java:79)
> >        at
> org.apache.hadoop.hbase.master.BaseScanner.chore(BaseScanner.java:136)
> >        at org.apache.hadoop.hbase.Chore.run(Chore.java:68)
> > 2010-01-13 18:35:31,575 INFO
> org.apache.hadoop.hbase.master.ServerManager: 5 region servers, 0 dead,
> average load 0.16666666666666666
> > 2010-01-13 18:35:32,740 INFO org.apache.hadoop.hbase.master.BaseScanner:
> RegionManager.rootScanner scanning meta region {server: 10.37.17.248:60020,
> regionname: -ROOT-,,0, startKey: <>}
> > 2010-01-13 18:35:32,741 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /
> 10.37.17.248:60020 could not be reached after 1 tries, giving up.
> > 2010-01-13 18:35:32,741 WARN org.apache.hadoop.hbase.master.BaseScanner:
> Scan ROOT region
> > org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed setting
> up proxy to /10.37.17.248:60020 after attempts=1
> >        at
> org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:424)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getHRegionConnection(HConnectionManager.java:865)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getHRegionConnection(HConnectionManager.java:881)
> >        at
> org.apache.hadoop.hbase.master.BaseScanner.scanRegion(BaseScanner.java:156)
> >        at
> org.apache.hadoop.hbase.master.RootScanner.scanRoot(RootScanner.java:54)
> >        at
> org.apache.hadoop.hbase.master.RootScanner.maintenanceScan(RootScanner.java:79)
> >        at
> org.apache.hadoop.hbase.master.BaseScanner.chore(BaseScanner.java:136)
> >        at org.apache.hadoop.hbase.Chore.run(Chore.java:68)
> > 2010-01-13 18:35:32,742 INFO org.apache.hadoop.hbase.master.BaseScanner:
> All 0 .META. region(s) scanned
> > 2010-01-13 18:36:31,575 INFO
> org.apache.hadoop.hbase.master.ServerManager: 5 region servers, 0 dead,
> average load 0.16666666666666666
> > 2010-01-13 18:36:32,740 INFO org.apache.hadoop.hbase.master.BaseScanner:
> RegionManager.rootScanner scanning meta region {server: 10.37.17.248:60020,
> regionname: -ROOT-,,0, startKey: <>}
> > 2010-01-13 18:36:32,741 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /
> 10.37.17.248:60020 could not be reached after 1 tries, giving up.
> > 2010-01-13 18:36:32,741 WARN org.apache.hadoop.hbase.master.BaseScanner:
> Scan ROOT region
> > org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed setting
> up proxy to /10.37.17.248:60020 after attempts=1
> >        at
> org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:424)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getHRegionConnection(HConnectionManager.java:865)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getHRegionConnection(HConnectionManager.java:881)
> >        at
> org.apache.hadoop.hbase.master.BaseScanner.scanRegion(BaseScanner.java:156)
> >        at
> org.apache.hadoop.hbase.master.RootScanner.scanRoot(RootScanner.java:54)
> >        at
> org.apache.hadoop.hbase.master.RootScanner.maintenanceScan(RootScanner.java:79)
> >        at
> org.apache.hadoop.hbase.master.BaseScanner.chore(BaseScanner.java:136)
> >        at org.apache.hadoop.hbase.Chore.run(Chore.java:68)
> > 2010-01-13 18:36:32,742 INFO org.apache.hadoop.hbase.master.BaseScanner:
> All 0 .META. region(s) scanned
> > 2010-01-13 18:37:31,575 INFO
> org.apache.hadoop.hbase.master.ServerManager: 5 region servers, 0 dead,
> average load 0.16666666666666666
> > 2010-01-13 18:37:32,741 INFO org.apache.hadoop.hbase.master.BaseScanner:
> RegionManager.rootScanner scanning meta region {server: 10.37.17.248:60020,
> regionname: -ROOT-,,0, startKey: <>}
> > 2010-01-13 18:37:32,741 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /
> 10.37.17.248:60020 could not be reached after 1 tries, giving up.
> > 2010-01-13 18:37:32,741 WARN org.apache.hadoop.hbase.master.BaseScanner:
> Scan ROOT region
> > org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed setting
> up proxy to /10.37.17.248:60020 after attempts=1
> >        at
> org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:424)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getHRegionConnection(HConnectionManager.java:865)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getHRegionConnection(HConnectionManager.java:881)
> >        at
> org.apache.hadoop.hbase.master.BaseScanner.scanRegion(BaseScanner.java:156)
> >        at
> org.apache.hadoop.hbase.master.RootScanner.scanRoot(RootScanner.java:54)
> >        at
> org.apache.hadoop.hbase.master.RootScanner.maintenanceScan(RootScanner.java:79)
> >        at
> org.apache.hadoop.hbase.master.BaseScanner.chore(BaseScanner.java:136)
> >        at org.apache.hadoop.hbase.Chore.run(Chore.java:68)
> > 2010-01-13 18:37:32,742 INFO org.apache.hadoop.hbase.master.BaseScanner:
> All 0 .META. region(s) scanned
> > 2010-01-13 18:38:31,575 INFO
> org.apache.hadoop.hbase.master.ServerManager: 5 region servers, 0 dead,
> average load 0.16666666666666666
> > 2010-01-13 18:38:32,740 INFO org.apache.hadoop.hbase.master.BaseScanner:
> RegionManager.rootScanner scanning meta region {server: 10.37.17.248:60020,
> regionname: -ROOT-,,0, startKey: <>}
> > 2010-01-13 18:38:32,741 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /
> 10.37.17.248:60020 could not be reached after 1 tries, giving up.
> > 2010-01-13 18:38:32,741 WARN org.apache.hadoop.hbase.master.BaseScanner:
> Scan ROOT region
> > org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed setting
> up proxy to /10.37.17.248:60020 after attempts=1
> >        at
> org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:424)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getHRegionConnection(HConnectionManager.java:865)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getHRegionConnection(HConnectionManager.java:881)
> >        at
> org.apache.hadoop.hbase.master.BaseScanner.scanRegion(BaseScanner.java:156)
> >        at
> org.apache.hadoop.hbase.master.RootScanner.scanRoot(RootScanner.java:54)
> >        at
> org.apache.hadoop.hbase.master.RootScanner.maintenanceScan(RootScanner.java:79)
> >        at
> org.apache.hadoop.hbase.master.BaseScanner.chore(BaseScanner.java:136)
> >        at org.apache.hadoop.hbase.Chore.run(Chore.java:68)
> > 2010-01-13 18:38:32,742 INFO org.apache.hadoop.hbase.master.BaseScanner:
> All 0 .META. region(s) scanned
> > 2010-01-13 18:39:31,575 INFO
> org.apache.hadoop.hbase.master.ServerManager: 5 region servers, 0 dead,
> average load 0.16666666666666666
> > 2010-01-13 18:39:32,740 INFO org.apache.hadoop.hbase.master.BaseScanner:
> RegionManager.rootScanner scanning meta region {server: 10.37.17.248:60020,
> regionname: -ROOT-,,0, startKey: <>}
> > 2010-01-13 18:39:32,741 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /
> 10.37.17.248:60020 could not be reached after 1 tries, giving up.
> > 2010-01-13 18:39:32,741 WARN org.apache.hadoop.hbase.master.BaseScanner:
> Scan ROOT region
> > org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed setting
> up proxy to /10.37.17.248:60020 after attempts=1
> >        at
> org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:424)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getHRegionConnection(HConnectionManager.java:865)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getHRegionConnection(HConnectionManager.java:881)
> >        at
> org.apache.hadoop.hbase.master.BaseScanner.scanRegion(BaseScanner.java:156)
> >        at
> org.apache.hadoop.hbase.master.RootScanner.scanRoot(RootScanner.java:54)
> >        at
> org.apache.hadoop.hbase.master.RootScanner.maintenanceScan(RootScanner.java:79)
> >        at
> org.apache.hadoop.hbase.master.BaseScanner.chore(BaseScanner.java:136)
> >        at org.apache.hadoop.hbase.Chore.run(Chore.java:68)
> > 2010-01-13 18:39:32,742 INFO org.apache.hadoop.hbase.master.BaseScanner:
> All 0 .META. region(s) scanned
> > 2010-01-13 18:40:31,575 INFO
> org.apache.hadoop.hbase.master.ServerManager: 5 region servers, 0 dead,
> average load 0.16666666666666666
> > 2010-01-13 18:40:32,740 INFO org.apache.hadoop.hbase.master.BaseScanner:
> RegionManager.rootScanner scanning meta region {server: 10.37.17.248:60020,
> regionname: -ROOT-,,0, startKey: <>}
> > 2010-01-13 18:40:32,740 INFO org.apache.hadoop.ipc.HbaseRPC: Server at /
> 10.37.17.248:60020 could not be reached after 1 tries, giving up.
> > 2010-01-13 18:40:32,741 WARN org.apache.hadoop.hbase.master.BaseScanner:
> Scan ROOT region
> > org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed setting
> up proxy to /10.37.17.248:60020 after attempts=1
> >        at
> org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:424)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getHRegionConnection(HConnectionManager.java:865)
> >        at
> org.apache.hadoop.hbase.client.HConnectionManager$TableServers.getHRegionConnection(HConnectionManager.java:881)
> >        at
> org.apache.hadoop.hbase.master.BaseScanner.scanRegion(BaseScanner.java:156)
> >        at
> org.apache.hadoop.hbase.master.RootScanner.scanRoot(RootScanner.java:54)
> >        at
> org.apache.hadoop.hbase.master.RootScanner.maintenanceScan(RootScanner.java:79)
> >        at
> org.apache.hadoop.hbase.master.BaseScanner.chore(BaseScanner.java:136)
> >        at org.apache.hadoop.hbase.Chore.run(Chore.java:68)
> > .....
> >
> >
> > Can somebody help me. Thank you.
> > ------------------
> > Zhongyuan
> > 2010-01-13
> >
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message