hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: how to check whether zookeeper quorum is working fine or not ?
Date Fri, 08 Jul 2011 09:57:25 GMT
Can you check zookeeper log ?

Cheers



On Jul 8, 2011, at 2:31 AM, praveenesh kumar <praveenesh@gmail.com> wrote:

> Guys.. I have increased the zookeeper connections to 2000, still I am
> getting the same zookeeper exception while running the pig scripts.
> Can anyone tell me what can be the reason ?
> 
> Thanks,
> Praveenesh
> 
> On Fri, Jul 8, 2011 at 12:16 AM, Barney Frank <barneyfranks1@gmail.com>wrote:
> 
>> Why not just set the maxClientCnxcns to 0?  That would make it unlimited,
>> right?
>> 
>> On Thu, Jul 7, 2011 at 12:55 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>> 
>>> To 1000.
>>> 
>>> Click on 'zk dump' link from Master UI.
>>> You would know roughly how many connections were established.
>>> If 1000 is exhausted, increase more.
>>> 
>>> See
>>> 
>>> 
>> http://zhihongyu.blogspot.com/2011/04/managing-connections-in-hbase-090-and.html
>>> 
>>> On Thu, Jul 7, 2011 at 10:52 AM, praveenesh kumar <praveenesh@gmail.com
>>>> wrote:
>>> 
>>>> I did that.. I increased it to 200.. still facing the same error.. what
>>>> should be the ideal value ?
>>>> 
>>>> On Thu, Jul 7, 2011 at 11:20 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>>>> 
>>>>> Increase hbase.zookeeper.property.maxClientCnxns to much bigger
>> value.
>>>>> 
>>>>> 
>>>>> On Thu, Jul 7, 2011 at 10:46 AM, praveenesh kumar <
>>> praveenesh@gmail.com
>>>>>> wrote:
>>>>> 
>>>>>> My HBase and pig both are running fine for me..
>>>>>> Only problem is coming whenever I am trying to access big hbase
>> table
>>>>>> containing too many columns and rows..
>>>>>> whenever I am trying to access hbase table from pig... I am getting
>>>> this
>>>>>> following error in the middle of the map process:
>>>>>> 
>>>>>> org.apache.pig.tools.pigstats.SimplePigStats - ERROR 2997: Unable
>> to
>>>>>> recreate exception from backed error:
>>>>>> org.apache.hadoop.hbase.ZooKeeperConnectionException: HBase is able
>>> to
>>>>>> connect to ZooKeeper but the connection closes immediately. This
>>> could
>>>> be
>>>>> a
>>>>>> sign that the server has too many connections (30 is the default).
>>>>> Consider
>>>>>> inspecting your ZK server logs for that error and then make sure
>> you
>>>> are
>>>>>> reusing HBaseConfiguration as often as you can. See HTable's
>> javadoc
>>>> for
>>>>>> more information.
>>>>>> But if I access a small HBase table from pig.. its working fine..
>> no
>>>>>> zookeeper exception is coming. I am able to do all other things
>>>> perfectly
>>>>>> fine.
>>>>>> This error made me think that there is some problem in my zookeeper
>>>> that
>>>>> is
>>>>>> why I was asking is there anyway to check whether the zookeeper are
>>>>> working
>>>>>> fine or not ?
>>>>>> 
>>>>>> My zookeeper log -- http://pastebin.com/RRdzZYui
>>>>>> My hbase-site.xml
>>>>>> 
>>>>>> <configuration>
>>>>>> <property>
>>>>>> <name>hbase.rootdir</name>
>>>>>> <value>hdfs://ub13:54310/hbase</value>
>>>>>> <final>true</final>
>>>>>> </property>
>>>>>> 
>>>>>> 
>>>>>> <property>
>>>>>> <name>hbase.zookeeper.quorum</name>
>>>>>> <value>ub13,ub12,ub11</value>
>>>>>> <final>true</final>
>>>>>> </property>
>>>>>> 
>>>>>> <property>
>>>>>> <name>hbase.zookeeper.property.maxClientCnxns</name>
>>>>>> <value>200</value>
>>>>>> <final>true</final>
>>>>>> </property>
>>>>>> 
>>>>>> <property>
>>>>>> <name>hbase.cluster.distributed</name>
>>>>>> <value>true</value>
>>>>>> <description>The mode the cluster will be in. Possible values
are
>>>>>> false: standalone and pseudo-distributed setups with managed
>>> Zookeeper
>>>>>> true: fully-distributed with unmanaged Zookeeper Quorum
>>>>>> </description>
>>>>>> </property>
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> On Thu, Jul 7, 2011 at 10:47 PM, Stack <stack@duboce.net> wrote:
>>>>>> 
>>>>>>> On Thu, Jul 7, 2011 at 8:54 AM, praveenesh kumar <
>>>> praveenesh@gmail.com
>>>>>> 
>>>>>>> wrote:
>>>>>>>> Can you please go through this and tell me what can be the
>> issue
>>> ?
>>>>>>>> 
>>>>>>> 
>>>>>>> Can you summarize?  (Life is short)
>>>>>>> 
>>>>>>> If stuff works generally -- shell?  And can you do the hbase
>>>>>>> rowcounter mapreduce job successfully -- but then when you put
>> PIG
>>> in
>>>>>>> the picture it does not, it would seem to be an issue with your
>> PIG
>>>>>>> setup?
>>>>>>> 
>>>>>>> St.Ack
>>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>>> 
>> 

Mime
View raw message