hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Carlotta Hicks <Carlotta.Hi...@sas.com>
Subject Server in the failed servers list
Date Thu, 20 Nov 2014 21:08:36 GMT
Hi All - My machine has somehow made the failed servers list.  How do I get off the list?

CDH 5.0.1
HBase 0.12

org.apache.hadoop.hbase.client.RetriesExhaustedException: Failed after attempts=35, exceptions:
Thu Nov 20 15:38:01 EST 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@54930080, org.apache.hadoop.hbase.ipc.RpcClient$FailedServerException:
This server is in the failed servers list: v64-85-02.xx.xxx.com/10.20.14.50:60020
Thu Nov 20 15:38:02 EST 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@54930080, java.net.ConnectException:
Connection refused
Thu Nov 20 15:38:04 EST 2014, org.apache.hadoop.hbase.client.RpcRetryingCaller@54930080, java.net.ConnectException:
Connection refused
                at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:134)
                at org.apache.hadoop.hbase.client.HTable.getRowOrBefore(HTable.java:705)
                at org.apache.hadoop.hbase.client.MetaScanner.metaScan(MetaScanner.java:144)
                at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.prefetchRegionCache(HConnectionManager.java:1102)
                at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegionInMeta(HConnectionManager.java:1162)
                at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:1054)
                at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.locateRegion(HConnectionManager.java:1011)
                at org.apache.hadoop.hbase.client.HTable.finishSetup(HTable.java:326)
                at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:192)
                at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:150)
                at org.apache.hadoop.hbase.mapreduce.TableOutputFormat.setConf(TableOutputFormat.java:206)
                at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:73)
                at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:133)
                at org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:455)
                at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:343)
                at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1295)
                at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1292)
                at java.security.AccessController.doPrivileged(Native Method)
                at javax.security.auth.Subject.doAs(Subject.java:415)
                at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
                at org.apache.hadoop.mapreduce.Job.submit(Job.java:1292)
                at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:1313)
                at org.apache.hadoop.hbase.mapreduce.ImportTsv.run(ImportTsv.java:481)
                at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
                at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
                at org.apache.hadoop.hbase.mapreduce.ImportTsv.main(ImportTsv.java:485)
Caused by: java.net.ConnectException: Connection refused
                at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
                at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:735)
                at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
                at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:529)
                at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:493)
                at org.apache.hadoop.hbase.ipc.RpcClient$Connection.setupConnection(RpcClient.java:575)
                at org.apache.hadoop.hbase.ipc.RpcClient$Connection.setupIOstreams(RpcClient.java:860)
                at org.apache.hadoop.hbase.ipc.RpcClient.getConnection(RpcClient.java:1535)
                at org.apache.hadoop.hbase.ipc.RpcClient.call(RpcClient.java:1424)
                at org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1653)
                at org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1711)
                at org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.get(ClientProtos.java:27308)
                at org.apache.hadoop.hbase.protobuf.ProtobufUtil.getRowOrBefore(ProtobufUtil.java:1406)
                at org.apache.hadoop.hbase.client.HTable$2.call(HTable.java:701)
                at org.apache.hadoop.hbase.client.HTable$2.call(HTable.java:699)
                at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:120)
                ... 25 more
2014-11-20 15:47:12,475 DEBUG [main] client.HConnectionManager$HConnectionImplementation:
locateRegionInMeta failed; parentTable=hbase:meta, metaLocation={region=hbase:meta,,1.1588230740,
hostname=v64-85-02.xx.xxx.com,60020,1415509554582, seqNum=0}, attempt=3/35; retrying after=503ms;
because: This server is in the failed servers list: v64-85-02.xx.xxx.com/10.20.14.50:60020

Carlotta


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message