hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Akmal Abbasov <akmal.abba...@icloud.com>
Subject Re: HBase master WebUI is not responding
Date Fri, 04 Sep 2015 15:20:33 GMT
Hi Ted, 

No, I’ve restarted master node and now there is no exceptions, but still webui is not responding
2015-09-04 14:54:57,926 DEBUG [FifoRpcScheduler.handler1-thread-26] master.AssignmentManager:
based on AM, current region=hbase:meta,,1.1588230740 is on server=test-rs3,60020,1441378286999
server being checked: test-rs5,60020,1441261824016
What else could be wrong?

Thanks.


> On 03 Sep 2015, at 16:20, Ted Yu <yuzhihong@gmail.com> wrote:
> 
> Did the exception lead to hbase:meta assignment failure ?
> 
> Please check region server log on 10.10.8.53 <http://10.10.8.53:60000/>
> 
> On Thu, Sep 3, 2015 at 7:15 AM, Akmal Abbasov <akmal.abbasov@icloud.com <mailto:akmal.abbasov@icloud.com>>
wrote:
> Hi Ted,
> I’m getting a lot of
> 2015-09-03 14:08:20,651 WARN  [1865504699@qtp-838397579-13] client.HConnectionManager$HConnectionImplementation:
Checking master connection
> com.google.protobuf.ServiceException: java.net.SocketTimeoutException: Call to test-m/10.10.8.53:60000
<http://10.10.8.53:60000/> failed because java.net.SocketTimeoutException: 60000 millis
timeout while waiting for channel to be ready for read. ch : java.nio.channels.SocketChannel[connected
local=/10.10.8.53:41243 <http://10.10.8.53:41243/> remote=test-m/10.10.8.53:60000 <http://10.10.8.53:60000/>]
>         at org.apache.hadoop.hbase.ipc.RpcClient.callBlockingMethod(RpcClient.java:1678)
>         at org.apache.hadoop.hbase.ipc.RpcClient$BlockingRpcChannelImplementation.callBlockingMethod(RpcClient.java:1719)
>         at org.apache.hadoop.hbase.protobuf.generated.MasterProtos$MasterService$BlockingStub.isMasterRunning(MasterProtos.java:42561)
>         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation$MasterServiceState.isMasterRunning(HConnectionManager.java:1508)
>         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.isKeepAliveMasterConnectedAndRunning(HConnectionManager.java:2146)
>         at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getKeepAliveMasterService(HConnectionManager.java:1859)
>         at org.apache.hadoop.hbase.client.HBaseAdmin$MasterCallable.prepare(HBaseAdmin.java:3320)
>         at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:113)
>         at org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:90)
>         at org.apache.hadoop.hbase.client.HBaseAdmin.executeCallable(HBaseAdmin.java:3347)
>         at org.apache.hadoop.hbase.client.HBaseAdmin.listTableDescriptorsByNamespace(HBaseAdmin.java:2281)
>         at org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.__jamon_innerUnit__catalogTables(MasterStatusTmplImpl.java:465)
>         at org.apache.hadoop.hbase.tmpl.master.MasterStatusTmplImpl.renderNoFlush(MasterStatusTmplImpl.java:274)
>         at org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.renderNoFlush(MasterStatusTmpl.java:386)
>         at org.apache.hadoop.hbase.tmpl.master.MasterStatusTmpl.render(MasterStatusTmpl.java:376)
>         at org.apache.hadoop.hbase.master.MasterStatusServlet.doGet(MasterStatusServlet.java:95)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:735)
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:848)
>         at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
>         at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
>         at org.apache.hadoop.http.lib.StaticUserWebFilter$StaticUserFilter.doFilter(StaticUserWebFilter.java:109)
>         at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>         at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:1122)
>         at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>         at org.apache.hadoop.http.NoCacheFilter.doFilter(NoCacheFilter.java:45)
>         at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
>         at java.io.FilterInputStream.read(FilterInputStream.java:133)
>         at org.apache.hadoop.hbase.ipc.RpcClient$Connection$PingInputStream.read(RpcClient.java:558)
>         at java.io.BufferedInputStream.fill(BufferedInputStream.java:235)
>         at java.io.BufferedInputStream.read(BufferedInputStream.java:254)
>         at java.io.DataInputStream.readInt(DataInputStream.java:387)
>         at org.apache.hadoop.hbase.ipc.RpcClient$Connection.readResponse(RpcClient.java:1076)
>         at org.apache.hadoop.hbase.ipc.RpcClient$Connection.run(RpcClient.java:727)
> 
> Thanks
> 
> > On 03 Sep 2015, at 15:50, Ted Yu <yuzhihong@gmail.com <mailto:yuzhihong@gmail.com>>
wrote:
> >
> > Can you log into the master node and check master log ?
> >
> > Pastebin snippet of master log if you need more help.
> >
> > Cheers
> >
> >
> >> On Sep 3, 2015, at 6:24 AM, Akmal Abbasov <akmal.abbasov@icloud.com <mailto:akmal.abbasov@icloud.com>>
wrote:
> >>
> >> Hi,
> >> I’m having problems with accessing HBase master webui. when I try to access
it, it returns redirect page /master-status, and then it is freezes.
> >> I’ve tried to wait for a long time, but still it didn’t work.
> >> I am using hbase hbase-0.98.7-hadoop2, hadoop hadoop-2.5.1
> >> Any suggestions?
> >> Thanks.
> 
> 


Mime
View raw message