hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ramkrishna vasudevan <ramkrishna.s.vasude...@gmail.com>
Subject Re: HConnection thread waiting on blocking queue indefinitely
Date Fri, 05 Jun 2015 06:07:24 GMT
When you started  your client server was the META table assigned.  May be
some thing happened around that time and the client app was just waiting on
the meta table to be assigned.  It would have retried - Can you check the
logs.?

So the best part here is the stand alone client was able to be successful -
which means the new clients were able to talk successfully with the
server.  And hence the restart of your client has solved  your problem.  It
may be difficult to trouble shoot the exact issue with the limited info -
but see if your client app regularly gets stalled and then it is better to
trouble shoot your app and the way it accesses the server.

On Fri, Jun 5, 2015 at 11:21 AM, PRANEESH KUMAR <praneesh.sankar@gmail.com>
wrote:

> The client connection was in stalled state. But there was only one
> hconnection thread found in our thread dump, which was waiting indefinitely
> in BoundedCompletionService.take call. Meanwhile we ran a standalone test
> program which was successful.
>
> Once we restarted the client server, the problem got resolved.
>
> The basic doubt is, when the hconnection thread stalled, why the HBase
> client failed to create any more hconnections(max pool size was 10). In
> case of problem with table/meta regions how come the test program
> succeeded.
>
> Regards,
> Praneesh
>
> On Fri, Jun 5, 2015 at 10:21 AM, ramkrishna vasudevan <
> ramkrishna.s.vasudevan@gmail.com> wrote:
>
> > Can you tell us more. Is your client not working at all and it is
> stalled ?
> > Are you seeing some results but you find it slow than you expected?
> >
> > What type of workload are you running?  All the tables are healthy?  Are
> > you able to read or write to them individually using the hbase shell?
> >
> > On Fri, Jun 5, 2015 at 10:18 AM, PRANEESH KUMAR <
> praneesh.sankar@gmail.com
> > >
> > wrote:
> >
> > > Hi Ram,
> > >
> > > The cluster ran without any problem for about 2 to 3 days with low
> load,
> > > once we enabled it for high load we immediately faced this issue.
> > >
> > >
> > > Regards,
> > > Praneesh.
> > >
> > > On Thursday 4 June 2015, ramkrishna vasudevan <
> > > ramkrishna.s.vasudevan@gmail.com> wrote:
> > >
> > > > Is your cluster in working condition.  Can you see if the META has
> been
> > > > assigned properly?  If the META table is not initialized and opened
> > then
> > > > your client thread will hang.
> > > >
> > > > Regards
> > > > Ram
> > > >
> > > > On Thu, Jun 4, 2015 at 9:05 PM, PRANEESH KUMAR <
> > > praneesh.sankar@gmail.com
> > > > <javascript:;>>
> > > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > We are using Hbase-1.0.0. We also facing the same issue that client
> > > > > connection thread is waiting at
> > > > >
> > > > >
> > > >
> > >
> >
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.locateRegionInMeta(ConnectionManager.java:1200).
> > > > >
> > > > > Any help is appreciated.
> > > > >
> > > > > Regards,
> > > > > Praneesh
> > > > >
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message