hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Dignan <pdig...@hubspot.com>
Subject Re: No IPC Handlers showing up in Web UI
Date Fri, 31 Oct 2014 17:39:06 GMT
Hi Esteban,

We are using the AggregateImplementation coprocessors, but we have been
using them without issue for a while.  Nothing else uncommon- running on
Java 7, no SELinux.  We don't see any exceptions related to OS resources,
however we have had periods of very high CPU load due to clients
effectively DDoSing the region servers.  We actually noticed this issue
while debugging the load issues.

I'm guessing you meant a jstack from the regionserver, so I created a gist
of the jstack output from an affected regionserver:
https://gist.github.com/dignan/f22cedfbad36d6420035

Thanks!

On Fri, Oct 31, 2014 at 1:03 AM, Esteban Gutierrez <esteban@cloudera.com>
wrote:

> Hi Patrick,
>
> That is very interesting. Are you using coprocessors? is there anything
> else un-common in your environment, e.g. SELinux on, a  Java 8 or limited
> OS resources? If you experience this again is there any chance for you to
> upload a jstack of the HBase master process to pastebin or gist?
>
> cheers,
> esteban.
>
>
> --
> Cloudera, Inc.
>
>
> On Thu, Oct 30, 2014 at 3:14 PM, Patrick Dignan <pdignan@hubspot.com>
> wrote:
>
> > There don't seem to be any exceptions related to this, except maybe this:
> > 2014-10-29 21:05:59,664 WARN org.apache.hadoop.ipc.HBaseServer: IPC
> Server
> > listener on 60020: readAndProcess threw exception java.io.IOException:
> > Connection reset by peer. Count of bytes read: 0
> >
> > Accessing from the URLs in the Tasks section does not work either.  No
> > errors show up in the log if I reload the page a few times.  There's no
> > proxy causing issues.
> >
> > On Thu, Oct 30, 2014 at 6:04 PM, Esteban Gutierrez <esteban@cloudera.com
> >
> > wrote:
> >
> > > Hi Patrick,
> > >
> > > Have you tried to access directly via any of the URLs from the Tasks
> > > section? e.g. http://<regionserver>:<port>/rs-status?filter=all
or
> > > http://<regionserver>:<port>/rs-status?filter=handler
> > > do you see any error in the logs if you reload the page a couple of
> > times?
> > > Another possibility is that you are behind a proxy and that might be
> > > causing an odd behavior.
> > >
> > > esteban.
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message