hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From lohit <lohit.vijayar...@gmail.com>
Subject Re: HConnectionManager max cached instances
Date Fri, 29 Apr 2011 03:59:43 GMT
2011/4/28 Ted Yu <yuzhihong@gmail.com>

> I would be happy to know that solution.
> If that solution exists, Karthick and I wouldn't have spent 2 weeks on
> HBASE-3777 :-)
>
I was about to suggest that the size be set to same as maxClientConnections,
but I see that it has already been done in HBASE-3777

>
> On Thu, Apr 28, 2011 at 7:51 PM, lohit <lohit.vijayarenu@gmail.com> wrote:
>
> > Thanks for the pointer. Even if we increase
> > hbase.zookeeper.property.maxClientCnxns
> > wouldn't the cache size still be 31?
> > I was wondering if there was way to avoid cache miss so that it would not
> > create new ZK connection.
> >
> > 2011/4/28 Ted Yu <yuzhihong@gmail.com>
> >
> > > Increase value for the following parameter:
> > >  <property>
> > >    <name>hbase.zookeeper.property.maxClientCnxns</name>
> > >    <value>30</value>
> > >
> > > Please read my blog for long-term fix which should be commited soon:
> > >
> > >
> >
> http://zhihongyu.blogspot.com/2011/04/managing-connections-in-hbase-090-and.html
> > >
> > > On Thu, Apr 28, 2011 at 7:26 PM, lohit <lohit.vijayarenu@gmail.com>
> > wrote:
> > >
> > > > Hi,
> > > >
> > > > By looking at HConnectionManager it looks like from a single node a
> max
> > > of
> > > > 31 connections can be cached.
> > > >
> > > > >  static final int MAX_CACHED_HBASE_INSTANCES = 31;
> > > >
> > > > I read the comment that this is based on the assumption that max
> client
> > > > connections to zookeeper is 30 from a single node.
> > > > ZooKeeper has an option to change that value, but we seem to be doing
> > lot
> > > > of
> > > > cache misses if we made more than 30 connections from a client.
> > > > This cache miss end up making new ZK connection and if number of
> HTable
> > > > operations are more, we see spike in ZooKeeper connections from
> single
> > > > node.
> > > >
> > > > This is seen in hbase.0.90.2. Is there any work around for this apart
> > > from
> > > > change the code?
> > > >
> > > > --
> > > > Thanks
> > > > Lohit
> > > >
> > >
> >
> >
> >
> > --
> > Have a Nice Day!
> > Lohit
> >
>



-- 
Have a Nice Day!
Lohit

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message