hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Daniel Cryans <jdcry...@apache.org>
Subject Re: Thrift on regionservers - won't connect
Date Fri, 10 Feb 2012 22:47:47 GMT
So is the master alive?

On Fri, Feb 10, 2012 at 12:21 PM, Tomas Tillery <tomastillery@gmail.com> wrote:
> I stand corrected - The region servers were still connected because they
> had remained connected. Restarting them resulted in them also receiving a
> Connection refused response. Error log follows.
>
> 12/02/10 20:12:24 INFO zookeeper.ClientCnxn: Opening socket connection to
> server master/192.168.1.1:2181
> 12/02/10 20:12:24 INFO zookeeper.ClientCnxn: Socket connection established
> to master/192.168.1.1:2181, initiating session
> 12/02/10 20:12:24 INFO zookeeper.ClientCnxn: Session establishment complete
> on server master/192.168.1.1:2181, sessionid = 0x13568dc6dfe0005,
> negotiated timeout = 40000
> 12/02/10 20:12:24 INFO ipc.HBaseRpcMetrics: Initializing RPC Metrics with
> hostName=HRegionServer, port=60020
> 12/02/10 20:12:24 INFO zookeeper.ZooKeeper: Initiating client connection,
> connectString=master:2181 sessionTimeout=180000 watcher=regionserver:60020
> 12/02/10 20:12:24 INFO zookeeper.ClientCnxn: Opening socket connection to
> server master/192.168.1.1:2181
> 12/02/10 20:12:24 INFO zookeeper.ClientCnxn: Socket connection established
> to master/192.168.1.1:2181, initiating session
> 12/02/10 20:12:24 INFO zookeeper.ClientCnxn: Session establishment complete
> on server master/192.168.1.1:2181, sessionid = 0x13568dc6dfe000c,
> negotiated timeout = 40000
> 12/02/10 20:12:24 INFO regionserver.ShutdownHook: Installed shutdown hook
> thread: Shutdownhook:regionserver60020
> 12/02/10 20:12:24 INFO regionserver.MemStoreFlusher:
> globalMemStoreLimit=395.0m, globalMemStoreLimitLowMark=345.6m,
> maxHeap=987.5m
> 12/02/10 20:12:24 INFO regionserver.HRegionServer: Runs every 10000000ms
> 12/02/10 20:12:24 INFO regionserver.HRegionServer: Attempting connect to
> Master server at master:60000
> 12/02/10 20:13:24 WARN regionserver.HRegionServer: Unable to connect to
> master. Retrying. Error was:
> java.net.ConnectException: Connection refused
>        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
>        at
> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
>        at
> org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
>        at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>        at
> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:328)
>        at
> org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:883)
>        at
> org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:750)
>        at
> org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:257)
>        at $Proxy6.getProtocolVersion(Unknown Source)
>        at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:419)
>        at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:393)
>        at org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:444)
>        at
> org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:349)
>        at
> org.apache.hadoop.hbase.regionserver.HRegionServer.getMaster(HRegionServer.java:1477)
>        at
> org.apache.hadoop.hbase.regionserver.HRegionServer.reportForDuty(HRegionServer.java:1530)
>        at
> org.apache.hadoop.hbase.regionserver.HRegionServer.tryReportForDuty(HRegionServer.java:1514)
>        at
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:573)
>        at java.lang.Thread.run(Thread.java:619)
>
> On Thu, Feb 9, 2012 at 2:48 PM, Tomas Tillery <tomastillery@gmail.com>wrote:
>
>> That appears to be the case. Here's an excerpt from the regionserver log
>> on the same machine:
>>
>> 12/01/24 22:11:44 INFO regionserver.HRegionServer: Attempting connect to
>> Master server at master:60000
>> 12/01/24 22:11:44 INFO regionserver.HRegionServer: Connected to master at
>> master:60000
>> 12/01/24 22:11:44 INFO regionserver.HRegionServer: Telling master at
>> master:60000 that we are up
>> 12/01/24 22:11:44 INFO regionserver.HRegionServer: Master passed us
>> address to use. Was=region-1:60020, Now=region-1:60020
>>
>>
>> On Thu, Feb 9, 2012 at 2:43 PM, Jean-Daniel Cryans <jdcryans@apache.org>wrote:
>>
>>> 60010 is the web ui port by default, 60000 is the port the master uses
>>> for communication. Clients (like thrift) and region servers don't rely
>>> on a configured port because the master already posts it in zookeeper
>>> along with its address (that's how your thrift server found it).
>>>
>>> If your region servers are using master:60000 and they don't have any
>>> problem connecting to a port that doesn't respond, we have a bigger
>>> issue here :)
>>>
>>> J-D
>>>
>>> On Thu, Feb 9, 2012 at 11:38 AM, Tomas Tillery <tomastillery@gmail.com>
>>> wrote:
>>> > No, it isn't - as far as I know, it should be aiming for 60010, rather
>>> than
>>> > 60000 (though HBase is also bound to 60000, there's no telnet response
>>> > there). There's probably something obvious I'm not managing to see, but
>>> I
>>> > can't find a way to redirect it to the proper port.
>>> > Starting thrift with this install is a matter of invoking
>>> > /etc/init.d/hadoop-hbase-thrift. Since it isn't invoked through the
>>> shell
>>> > script, I'm not sure where to specify the port to connect to.
>>> >
>>> > On Thu, Feb 9, 2012 at 2:27 PM, Jean-Daniel Cryans <jdcryans@apache.org
>>> >wrote:
>>> >
>>> >> Connection refused is a java exception that usually means that either
>>> >> there's nothing listening on the port or it's firewall'd. Is
>>> >> "master:60000" really what you're expecting to see here? Should be
>>> >> easy to "telnet master 60000" from that machien to really verify that
>>> >> it works as expected.
>>> >>
>>> >> J-D
>>> >>
>>> >> On Thu, Feb 9, 2012 at 11:19 AM, Tomas Tillery <tomastillery@gmail.com
>>> >
>>> >> wrote:
>>> >> > Hello, all
>>> >> >
>>> >> > It's my understanding that it's best practice to colocate thrift
on
>>> each
>>> >> of
>>> >> > the regionservers, and use load balancing to spread queries between
>>> them.
>>> >> > However, I can't seem to get thrift to bind from the regionservers
>>> (only
>>> >> > from the hmaster).
>>> >> >
>>> >> > The machines have connectivity between them; regionservers connect
>>> and
>>> >> > communicate with the hmaster without any problems.
>>> >> >
>>> >> > I'm running Hadoop 0.20.2-cdh3u2, HBase 0.90.4-cdh3u2.
>>> >> >
>>> >> > Thrift can run succesfully from the HMaster, but not from any of
the
>>> >> > datanode/regionservers
>>> >> >
>>> >> > What appears to be the relevant content from
>>> >> > hbase-hbase-thrift-region-1.out below:
>>> >> >
>>> >> > 12/02/09 19:09:17 INFO
>>> >> client.HConnectionManager$HConnectionImplementation:
>>> >> > getMaster attempt 0 of 10 failed; retrying after sleep of 1000
>>> >> > java.net.ConnectException: Connection refused
>>> >> >        at sun.nio.ch.SocketChannelImpl.checkConnect(Native
Method)
>>> >> >        at
>>> >> >
>>> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
>>> >> >        at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:328)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:883)
>>> >> >        at
>>> >> > org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:750)
>>> >> >        at
>>> >> >
>>> org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:257)
>>> >> >        at $Proxy4.getProtocolVersion(Unknown Source)
>>> >> >        at
>>> >> org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:419)
>>> >> >        at
>>> >> org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:393)
>>> >> >        at
>>> >> org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:444)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:360)
>>> >> >        at
>>> >> > org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:94)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.thrift.ThriftServer$HBaseHandler.<init>(ThriftServer.java:194)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.thrift.ThriftServer$HBaseHandler.<init>(ThriftServer.java:188)
>>> >> >        at
>>> >> >
>>> org.apache.hadoop.hbase.thrift.ThriftServer.doMain(ThriftServer.java:870)
>>> >> >        at
>>> >> >
>>> org.apache.hadoop.hbase.thrift.ThriftServer.main(ThriftServer.java:927)
>>> >> > 12/02/09 19:09:18 INFO
>>> >> client.HConnectionManager$HConnectionImplementation:
>>> >> > getMaster attempt 1 of 10 failed; retrying after sleep of 1000
>>> >> > java.net.ConnectException: Connection refused
>>> >> >        at sun.nio.ch.SocketChannelImpl.checkConnect(Native
Method)
>>> >> >        at
>>> >> >
>>> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
>>> >> >        at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:328)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:883)
>>> >> >        at
>>> >> > org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:750)
>>> >> >        at
>>> >> >
>>> org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:257)
>>> >> >        at $Proxy4.getProtocolVersion(Unknown Source)
>>> >> >        at
>>> >> org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:419)
>>> >> >        at
>>> >> org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:393)
>>> >> >        at
>>> >> org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:444)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:360)
>>> >> >        at
>>> >> > org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:94)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.thrift.ThriftServer$HBaseHandler.<init>(ThriftServer.java:194)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.thrift.ThriftServer$HBaseHandler.<init>(ThriftServer.java:188)
>>> >> >        at
>>> >> >
>>> org.apache.hadoop.hbase.thrift.ThriftServer.doMain(ThriftServer.java:870)
>>> >> >        at
>>> >> >
>>> org.apache.hadoop.hbase.thrift.ThriftServer.main(ThriftServer.java:927)
>>> >> > 12/02/09 19:09:19 INFO
>>> >> client.HConnectionManager$HConnectionImplementation:
>>> >> > getMaster attempt 2 of 10 failed; retrying after sleep of 1000
>>> >> > java.net.ConnectException: Connection refused
>>> >> >        at sun.nio.ch.SocketChannelImpl.checkConnect(Native
Method)
>>> >> >        at
>>> >> >
>>> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
>>> >> >        at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:328)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:883)
>>> >> >        at
>>> >> > org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:750)
>>> >> >        at
>>> >> >
>>> org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:257)
>>> >> >        at $Proxy4.getProtocolVersion(Unknown Source)
>>> >> >        at
>>> >> org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:419)
>>> >> >        at
>>> >> org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:393)
>>> >> >        at
>>> >> org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:444)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:360)
>>> >> >        at
>>> >> > org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:94)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.thrift.ThriftServer$HBaseHandler.<init>(ThriftServer.java:194)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.thrift.ThriftServer$HBaseHandler.<init>(ThriftServer.java:188)
>>> >> >        at
>>> >> >
>>> org.apache.hadoop.hbase.thrift.ThriftServer.doMain(ThriftServer.java:870)
>>> >> >        at
>>> >> >
>>> org.apache.hadoop.hbase.thrift.ThriftServer.main(ThriftServer.java:927)
>>> >> > 12/02/09 19:09:20 INFO
>>> >> client.HConnectionManager$HConnectionImplementation:
>>> >> > getMaster attempt 3 of 10 failed; retrying after sleep of 2000
>>> >> > java.net.ConnectException: Connection refused
>>> >> >        at sun.nio.ch.SocketChannelImpl.checkConnect(Native
Method)
>>> >> >        at
>>> >> >
>>> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
>>> >> >        at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:328)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:883)
>>> >> >        at
>>> >> > org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:750)
>>> >> >        at
>>> >> >
>>> org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:257)
>>> >> >        at $Proxy4.getProtocolVersion(Unknown Source)
>>> >> >        at
>>> >> org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:419)
>>> >> >        at
>>> >> org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:393)
>>> >> >        at
>>> >> org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:444)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:360)
>>> >> >        at
>>> >> > org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:94)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.thrift.ThriftServer$HBaseHandler.<init>(ThriftServer.java:194)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.thrift.ThriftServer$HBaseHandler.<init>(ThriftServer.java:188)
>>> >> >        at
>>> >> >
>>> org.apache.hadoop.hbase.thrift.ThriftServer.doMain(ThriftServer.java:870)
>>> >> >        at
>>> >> >
>>> org.apache.hadoop.hbase.thrift.ThriftServer.main(ThriftServer.java:927)
>>> >> > 12/02/09 19:09:22 INFO
>>> >> client.HConnectionManager$HConnectionImplementation:
>>> >> > getMaster attempt 4 of 10 failed; retrying after sleep of 2000
>>> >> > java.net.ConnectException: Connection refused
>>> >> >        at sun.nio.ch.SocketChannelImpl.checkConnect(Native
Method)
>>> >> >        at
>>> >> >
>>> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
>>> >> >        at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:328)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:883)
>>> >> >        at
>>> >> > org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:750)
>>> >> >        at
>>> >> >
>>> org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:257)
>>> >> >        at $Proxy4.getProtocolVersion(Unknown Source)
>>> >> >        at
>>> >> org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:419)
>>> >> >        at
>>> >> org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:393)
>>> >> >        at
>>> >> org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:444)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:360)
>>> >> >        at
>>> >> > org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:94)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.thrift.ThriftServer$HBaseHandler.<init>(ThriftServer.java:194)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.thrift.ThriftServer$HBaseHandler.<init>(ThriftServer.java:188)
>>> >> >        at
>>> >> >
>>> org.apache.hadoop.hbase.thrift.ThriftServer.doMain(ThriftServer.java:870)
>>> >> >        at
>>> >> >
>>> org.apache.hadoop.hbase.thrift.ThriftServer.main(ThriftServer.java:927)
>>> >> > 12/02/09 19:09:24 INFO
>>> >> client.HConnectionManager$HConnectionImplementation:
>>> >> > getMaster attempt 5 of 10 failed; retrying after sleep of 4000
>>> >> > java.net.ConnectException: Connection refused
>>> >> >        at sun.nio.ch.SocketChannelImpl.checkConnect(Native
Method)
>>> >> >        at
>>> >> >
>>> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
>>> >> >        at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:328)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:883)
>>> >> >        at
>>> >> > org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:750)
>>> >> >        at
>>> >> >
>>> org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:257)
>>> >> >        at $Proxy4.getProtocolVersion(Unknown Source)
>>> >> >        at
>>> >> org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:419)
>>> >> >        at
>>> >> org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:393)
>>> >> >        at
>>> >> org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:444)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:360)
>>> >> >        at
>>> >> > org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:94)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.thrift.ThriftServer$HBaseHandler.<init>(ThriftServer.java:194)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.thrift.ThriftServer$HBaseHandler.<init>(ThriftServer.java:188)
>>> >> >        at
>>> >> >
>>> org.apache.hadoop.hbase.thrift.ThriftServer.doMain(ThriftServer.java:870)
>>> >> >        at
>>> >> >
>>> org.apache.hadoop.hbase.thrift.ThriftServer.main(ThriftServer.java:927)
>>> >> > 12/02/09 19:09:28 INFO
>>> >> client.HConnectionManager$HConnectionImplementation:
>>> >> > getMaster attempt 6 of 10 failed; retrying after sleep of 4000
>>> >> > java.net.ConnectException: Connection refused
>>> >> >        at sun.nio.ch.SocketChannelImpl.checkConnect(Native
Method)
>>> >> >        at
>>> >> >
>>> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
>>> >> >        at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:328)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:883)
>>> >> >        at
>>> >> > org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:750)
>>> >> >        at
>>> >> >
>>> org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:257)
>>> >> >        at $Proxy4.getProtocolVersion(Unknown Source)
>>> >> >        at
>>> >> org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:419)
>>> >> >        at
>>> >> org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:393)
>>> >> >        at
>>> >> org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:444)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:360)
>>> >> >        at
>>> >> > org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:94)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.thrift.ThriftServer$HBaseHandler.<init>(ThriftServer.java:194)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.thrift.ThriftServer$HBaseHandler.<init>(ThriftServer.java:188)
>>> >> >        at
>>> >> >
>>> org.apache.hadoop.hbase.thrift.ThriftServer.doMain(ThriftServer.java:870)
>>> >> >        at
>>> >> >
>>> org.apache.hadoop.hbase.thrift.ThriftServer.main(ThriftServer.java:927)
>>> >> > 12/02/09 19:09:32 INFO
>>> >> client.HConnectionManager$HConnectionImplementation:
>>> >> > getMaster attempt 7 of 10 failed; retrying after sleep of 8000
>>> >> > java.net.ConnectException: Connection refused
>>> >> >        at sun.nio.ch.SocketChannelImpl.checkConnect(Native
Method)
>>> >> >        at
>>> >> >
>>> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
>>> >> >        at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:328)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:883)
>>> >> >        at
>>> >> > org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:750)
>>> >> >        at
>>> >> >
>>> org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:257)
>>> >> >        at $Proxy4.getProtocolVersion(Unknown Source)
>>> >> >        at
>>> >> org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:419)
>>> >> >        at
>>> >> org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:393)
>>> >> >        at
>>> >> org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:444)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:360)
>>> >> >        at
>>> >> > org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:94)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.thrift.ThriftServer$HBaseHandler.<init>(ThriftServer.java:194)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.thrift.ThriftServer$HBaseHandler.<init>(ThriftServer.java:188)
>>> >> >        at
>>> >> >
>>> org.apache.hadoop.hbase.thrift.ThriftServer.doMain(ThriftServer.java:870)
>>> >> >        at
>>> >> >
>>> org.apache.hadoop.hbase.thrift.ThriftServer.main(ThriftServer.java:927)
>>> >> > 12/02/09 19:09:40 INFO
>>> >> client.HConnectionManager$HConnectionImplementation:
>>> >> > getMaster attempt 8 of 10 failed; retrying after sleep of 16000
>>> >> > java.net.ConnectException: Connection refused
>>> >> >        at sun.nio.ch.SocketChannelImpl.checkConnect(Native
Method)
>>> >> >        at
>>> >> >
>>> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
>>> >> >        at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:328)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:883)
>>> >> >        at
>>> >> > org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:750)
>>> >> >        at
>>> >> >
>>> org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:257)
>>> >> >        at $Proxy4.getProtocolVersion(Unknown Source)
>>> >> >        at
>>> >> org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:419)
>>> >> >        at
>>> >> org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:393)
>>> >> >        at
>>> >> org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:444)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:360)
>>> >> >        at
>>> >> > org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:94)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.thrift.ThriftServer$HBaseHandler.<init>(ThriftServer.java:194)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.thrift.ThriftServer$HBaseHandler.<init>(ThriftServer.java:188)
>>> >> >        at
>>> >> >
>>> org.apache.hadoop.hbase.thrift.ThriftServer.doMain(ThriftServer.java:870)
>>> >> >        at
>>> >> >
>>> org.apache.hadoop.hbase.thrift.ThriftServer.main(ThriftServer.java:927)
>>> >> > 12/02/09 19:09:56 INFO
>>> >> client.HConnectionManager$HConnectionImplementation:
>>> >> > getMaster attempt 9 of 10 failed; no more retrying.
>>> >> > java.net.ConnectException: Connection refused
>>> >> >        at sun.nio.ch.SocketChannelImpl.checkConnect(Native
Method)
>>> >> >        at
>>> >> >
>>> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:574)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
>>> >> >        at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:408)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.setupIOstreams(HBaseClient.java:328)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.ipc.HBaseClient.getConnection(HBaseClient.java:883)
>>> >> >        at
>>> >> > org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:750)
>>> >> >        at
>>> >> >
>>> org.apache.hadoop.hbase.ipc.HBaseRPC$Invoker.invoke(HBaseRPC.java:257)
>>> >> >        at $Proxy4.getProtocolVersion(Unknown Source)
>>> >> >        at
>>> >> org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:419)
>>> >> >        at
>>> >> org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:393)
>>> >> >        at
>>> >> org.apache.hadoop.hbase.ipc.HBaseRPC.getProxy(HBaseRPC.java:444)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:360)
>>> >> >        at
>>> >> > org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:94)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.thrift.ThriftServer$HBaseHandler.<init>(ThriftServer.java:194)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.thrift.ThriftServer$HBaseHandler.<init>(ThriftServer.java:188)
>>> >> >        at
>>> >> >
>>> org.apache.hadoop.hbase.thrift.ThriftServer.doMain(ThriftServer.java:870)
>>> >> >        at
>>> >> >
>>> org.apache.hadoop.hbase.thrift.ThriftServer.main(ThriftServer.java:927)
>>> >> > Exception in thread "main"
>>> >> > org.apache.hadoop.hbase.MasterNotRunningException: master:60000
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getMaster(HConnectionManager.java:396)
>>> >> >        at
>>> >> > org.apache.hadoop.hbase.client.HBaseAdmin.<init>(HBaseAdmin.java:94)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.thrift.ThriftServer$HBaseHandler.<init>(ThriftServer.java:194)
>>> >> >        at
>>> >> >
>>> >>
>>> org.apache.hadoop.hbase.thrift.ThriftServer$HBaseHandler.<init>(ThriftServer.java:188)
>>> >> >        at
>>> >> >
>>> org.apache.hadoop.hbase.thrift.ThriftServer.doMain(ThriftServer.java:870)
>>> >> >        at
>>> >> >
>>> org.apache.hadoop.hbase.thrift.ThriftServer.main(ThriftServer.java:927)
>>> >>
>>>
>>
>>

Mime
View raw message