Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: pass (athena.apache.org: domain of dlieu.7@gmail.com designates
 209.85.220.169 as permitted sender)
DomainKey-Signature: a=rsa-sha1; c=nofws;
        d=gmail.com; s=gamma;
        h=mime-version:in-reply-to:references:date:message-id:subject:from:to
         :cc:content-type:content-transfer-encoding;
        b=hKkaxewF5g76wUB3wXDt5aVBJ7dmxGRTBKLBAcP+PSwUF81JIav6JcoGORmha9moBQ
         26R8eD7z1og4gwSbS0fp4n3hYVGGXIfwUoXt1/ElNklPru6w690oFmJosQCLXG2RX6eE
         yJW4284kVas8ACsqEicQhdiw6B4XzUMjiXkco=
MIME-Version: 1.0
In-Reply-To: <AANLkTimT2mtjAMHj8G5o6u=w_En9U0_VSBXek=CaUS4G@mail.gmail.com>
References: <AANLkTik0-JAYiiWrNZG0Phz=2SYoAiCZkqGfhbr7ZPVH@mail.gmail.com>
	<AANLkTin58Dm4436j35=Qj8HQVvbPJ+Ohy7pBkdVB8E7A@mail.gmail.com>
	<AANLkTimT2mtjAMHj8G5o6u=w_En9U0_VSBXek=CaUS4G@mail.gmail.com>
Date: Sat, 26 Mar 2011 02:04:26 -0700
Message-ID: <AANLkTimJhRkGSheQ2rMSEpLrOGgCN9iOD2abymVe4Pbb@mail.gmail.com>
Subject: Re: zookeeper-3.3.2 has default maxClientCnxns set to 10?
From: Dmitriy Lyubimov <dlieu.7@gmail.com>
To: user@hbase.apache.org
Cc: Alex Baranau <alex.baranov.v@gmail.com>, Stack <stack@duboce.net>,
	zookeeper-user@hadoop.apache.org
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: quoted-printable

yes i had a very similar  issue although i prefer to think about it in
terms of hbase and by extension zk connection leak in TableInputFormat
rather than adjusting max zk connection to 30 'cause sooner or later
you will run out of it too.

The problem is that HConnectionManager now identifies hbase connection
by identity comparison of the Configuration object. Which is kind of
unfortunate because cloning configuration is a common practice and in
reality it means every MR client would open its own connection to
hbase.

To exacerbate situation, TableInput/OutputFormats do not relinquish
unshared connection and there's very little you can do work around
this. I posted workaround to this i used with cdh3b4 just a day or two
ago. The issue turned out to be a pain as it really quite hard to
figure where and whether configuration objects get cloned or not.

Also, if you use HTable, be mindful that you may want to release that
connection explicitly too by calling
HConnectionManager.deleteConnection (conf, true).

On Fri, Mar 25, 2011 at 1:57 PM, Alex Baranau <alex.baranov.v@gmail.com> wr=
ote:
> I see what you are asking. I'm using stand-alone Zookeeper, not "internal=
"
> one of HBase. So it reads configuration only form zoo.cfg. And it seems t=
hat
> by default (when maxClientCnxns is absent in it) it acts like
> maxClientCnxns=3D10. I'd expect it to be unlimited when this property is
> omitted. At least it was like this..
>
> Aha! I see they changed default to 10 *in 3.3.x*. I think HBase users won=
't
> like it. What do you guys think?
>
> Alex Baranau
> ----
> Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch - Hadoop - HBas=
e
> Hadoop ecosystem search :: http://search-hadoop.com/
>
> On Fri, Mar 25, 2011 at 4:00 PM, Stack <stack@duboce.net> wrote:
>
>> On Fri, Mar 25, 2011 at 12:36 PM, Alex Baranau <alex.baranov.v@gmail.com=
>
>> wrote:
>> > As far as I know HBase configured to initiate up to 30 connections by
>> > default, and maxClientCnxns for Zookeeper was meant to be 30 as well.
>>
>> Yes
>>
>> I'm not sure how it'd go from 30 to 10 (Is 10 the default connections
>> for zk?). =A0Is it possible its not reading hbase-site.xml when the task
>> runs?
>>
>> St.Ack
>>
>