Return-Path: X-Original-To: apmail-hbase-user-archive@www.apache.org Delivered-To: apmail-hbase-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 9652FD4F8 for ; Wed, 20 Jun 2012 05:21:23 +0000 (UTC) Received: (qmail 3536 invoked by uid 500); 20 Jun 2012 05:21:21 -0000 Delivered-To: apmail-hbase-user-archive@hbase.apache.org Received: (qmail 3486 invoked by uid 500); 20 Jun 2012 05:21:21 -0000 Mailing-List: contact user-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hbase.apache.org Delivered-To: mailing list user@hbase.apache.org Received: (qmail 3454 invoked by uid 99); 20 Jun 2012 05:21:20 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 20 Jun 2012 05:21:20 +0000 X-ASF-Spam-Status: No, hits=2.8 required=5.0 tests=FREEMAIL_ENVFROM_END_DIGIT,FREEMAIL_REPLY,FSL_RCVD_USER,HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of manupkd87@gmail.com designates 74.125.82.169 as permitted sender) Received: from [74.125.82.169] (HELO mail-we0-f169.google.com) (74.125.82.169) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 20 Jun 2012 05:21:11 +0000 Received: by wefh52 with SMTP id h52so6749094wef.14 for ; Tue, 19 Jun 2012 22:20:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=tdBSqSyCpBptku3luSuz0YCPj4A8dcTj/vyUcR555sU=; b=h4QVLre3KXqUcWk+fCYkzxkq+ojPDxc6yVuoxvKK7DL8bHjQLU2JYTXsPLwedQfgQi X1yXgCNMMwn/nopl8jwCUx7LjmqxXE3yIXeuc7pcStHEmn12qnwFbdQnsJ2dDYD9nu+F MOrQQH3SBc2CJYE0LgWVO1NZK8ALtHR1vdpCmuVuLaCItPQW9zps/f1F4UA+XslG+zjp gHcf/S4SZJaMu1Tk6ZsxyPz7gZxrKIBVZEBUlhxtOceFAsW/6YpDBXnVEcbVcIxFkvJJ zNAdJmRLBdpmUscGJoUx+m+qiim0Cl5y1JiFpAOqjdZA4AoExPPU2HeaNxHIHc5elzzn ZFnA== MIME-Version: 1.0 Received: by 10.216.206.168 with SMTP id l40mr11745234weo.135.1340169651038; Tue, 19 Jun 2012 22:20:51 -0700 (PDT) Received: by 10.194.15.231 with HTTP; Tue, 19 Jun 2012 22:20:50 -0700 (PDT) In-Reply-To: References: Date: Wed, 20 Jun 2012 10:50:50 +0530 Message-ID: Subject: Re: HBase is able to connect to ZooKeeper but the connection closes immediately From: Manu S To: user@hbase.apache.org Content-Type: multipart/alternative; boundary=0016e6dee847fd290804c2e08f5f --0016e6dee847fd290804c2e08f5f Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Hi All, I am still facing this issue in our fully distributed cluster, but its working fine in pseudo-distributed node. We have installed *ZooKeeper *on our *Master node* which contains *Hbase-Master, Namenode & Job tracker*. 4 slave nodes having *Hbase-region, Datanode & Task tracker.* Zookeeper can be installed with Hbase-master, right? I did all the configuration changes in zoo.cfg & hbase-site.xml, copied all the hadoop jar & common-configuration jar to Hbase/lib directory. But still I am getting the same error *Caused by: org.apache.hadoop.hbase.**ZooKeeperConnectionException: HBase is able to connect to ZooKeeper but the connection closes immediately= . This could be a sign that the server has too many connections (30 is the default). Consider inspecting your ZK server logs for that error and then make sure you are reusing HBaseConfiguration as often as you can. See HTable's javadoc for more information**.* Is there any specific code modifications for writing output into HDFS using mapreduce or configuration problem?? Because it is working fine in pseudo-distributed node. Appreciate your help on the same. Thanks, Manu S On Thu, Jun 7, 2012 at 10:57 PM, shashwat shriparv < dwivedishashwat@gmail.com> wrote: > If you have separate zookeeper running no need to specify in hbase > settings... > > On Thu, Jun 7, 2012 at 8:12 PM, Manu S wrote: > > > Hi Tariq, > > > > Version: HBase-0.90.4 > > I downloaded commons-configuration-1.6.jar and put it inside > HBASE_HOME/lib > > & HADOOP_HOME/lib(In pseudo distributed) and tested. > > > > *hbase(main):002:0> status > > 12/06/07 19:59:59 FATAL zookeeper.ZKConfig: The server in zoo.cfg canno= t > be > > set to localhost in a fully-distributed setup because it won't be > > reachable. See "Getting Started" for more information. > > > > ERROR: org.apache.hadoop.hbase.ZooKeeperConnectionException: HBase is > able > > to connect to ZooKeeper but the connection closes immediately. This cou= ld > > be a sign that the server has too many connections (30 is the default). > > Consider inspecting your ZK server logs for that error and then make su= re > > you are reusing HBaseConfiguration as often as you can. See HTable's > > javadoc for more information.* > > > > Is this common-configuration version is compatible for hbase-0.90.4? > > > > > > > > @Shashwat: > > Thank you!! > > Some parameters are missing in my hbase configuration, I will add these > and > > test. The zookeeper parameters I already added in zoo.cfg. Is that enou= gh > > or do I need to add once again in hbase configuration? > > > > Thanks, > > Manu S > > > > > > On Thu, Jun 7, 2012 at 7:11 PM, shashwat shriparv < > > dwivedishashwat@gmail.com > > > wrote: > > > > > Try this settings check what you have and what you dont have in the > > > configuration : > > > > > > > > > ** > > > ** > > > *hbase.rootdir* > > > *hdfs://{your machine name} or {localhost}:9000/hbase* > > > ** > > > ** > > > *hbase.master* > > > *{your machine name} or {localhost}:60000* > > > *The host and port that the HBase master runs > > > at. > > > * > > > ** > > > ** > > > *hbase.regionserver.port* > > > *60020* > > > *The host and port that the HBase master runs > > > at. > > > * > > > ** > > > ** > > > ** > > > *hbase.cluster.distributed* > > > *true* > > > ** > > > ** > > > *hbase.tmp.dir* > > > */home/shashwat/Hadoop/hbase-0.90.4/temp* > > > ** > > > ** > > > *hbase.zookeeper.quorum* > > > *{your machine name} or {localhost}* > > > ** > > > ** > > > *dfs.replication* > > > *1* > > > ** > > > ** > > > *hbase.zookeeper.property.clientPort* > > > *2181* > > > *Property from ZooKeeper's config zoo.cfg.* > > > *The port at which the clients will connect.* > > > ** > > > ** > > > ** > > > *hbase.zookeeper.property.dataDir* > > > */home/shashwat/zookeeper* > > > *Property from ZooKeeper's config zoo.cfg.* > > > *The directory where the snapshot is stored.* > > > ** > > > ** > > > * > > > * > > > * > > > * > > > ** > > > * zookeeper.session.timeout* > > > * 18000000* > > > * Session Time out.* > > > * * > > > * * > > > * hbase.client.scanner.caching* > > > * 5000* > > > * * > > > ** > > > *hbase.regionserver.lease.period* > > > *2400000* > > > ** > > > > > > > > > > > > > > > Thanx and regards > > > > > > =E2=88=9E > > > Shashwat Shriparv > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Jun 7, 2012 at 3:16 PM, Mohammad Tariq > > wrote: > > > > > > > which distribution are you using??actually this is not possible..it > > > > must be there..download it put it there > > > > > > > > Regards, > > > > Mohammad Tariq > > > > > > > > > > > > On Thu, Jun 7, 2012 at 2:41 PM, Manu S wrote: > > > > > Hi Tariq, > > > > > > > > > > Thank you!! > > > > > I already changed the maxClientCnxns to 1000. > > > > > Also we have set CLASSPATH that includes all the Hadoop,HBase & > > > Zookeper > > > > > path's. I think copying hadoop .jar files to Hbase lib folder is > the > > > same > > > > > affect of setting CLASSPATH with all the folders. > > > > > There is no commons-configuration-*.jar inside hadoop/lib folder. > > > > > > > > > > Any other options? > > > > > > > > > > Thanks, > > > > > Manu S > > > > > > > > > > On Thu, Jun 7, 2012 at 1:31 PM, Mohammad Tariq > > > > > wrote: > > > > > > > > > >> Actually zookeeper servers have an active connections limit, whi= ch > > by > > > > >> default is 30. You can increase this limit by setting > maxClientCnxns > > > > >> property accordingly in your zookeeper config file, zoo.cfg. For > > > > >> example - maxClientCnxns=3D100....but before that copy the > > > > >> hadoop-core-*.jar present inside hadoop folder to the hbase/lib > > > > >> folder.Also copy commons-configuration-1.6.jar from hadoop/lib > > folder > > > > >> to hbase/lib folder and check it once and see if it works for yo= u. > > > > >> > > > > >> Regards, > > > > >> Mohammad Tariq > > > > >> > > > > >> > > > > >> On Thu, Jun 7, 2012 at 1:13 PM, Manu S > wrote: > > > > >> > Hi All, > > > > >> > > > > > >> > Thank you for your reply. > > > > >> > > > > > >> > I tried all these options but still I am facing this issue. > > > > >> > > > > > >> > @Mayank: I tried the same, but still getting error. > > > > >> > export > > > > >> > > > > > >> > > > > > > > > > > HADOOP_CLASSPATH=3D"/usr/lib/hadoop/:/usr/lib/hadoop/lib/:/usr/lib/hadoop= /conf/" > > > > >> > export > > > > >> > > > > > >> > > > > > > > > > > HBASE_CLASSPATH=3D"/usr/lib/hbase/:/usr/lib/hbase/lib/:/usr/lib/hbase/con= f/:/usr/lib/zookeeper/:/usr/lib/zookeeper/conf/:/usr/lib/zookeeper/lib/" > > > > >> > export CLASSPATH=3D"${HADOOP_CLASSPATH}:${HBASE_CLASSPATH}" > > > > >> > > > > > >> > @Marcos & Tariq: > > > > >> > We are using Hbase version 0.90.4 > > > > >> > Job creating single HBaseConfiguration object only > > > > >> > > > > > >> > @Kevin: > > > > >> > No luck, same error > > > > >> > > > > > >> > > > > > >> > Thanks, > > > > >> > Manu S > > > > >> > > > > > >> > On Thu, Jun 7, 2012 at 3:50 AM, Mayank Bansal < > mayank@apache.org> > > > > wrote: > > > > >> > > > > > >> >> > > > > >> >> zookeeper conf is not on the class path for the mapreduce job= . > > Add > > > > conf > > > > >> >>> file to class path for the job. > > > > >> >>> > > > > >> >>> Thanks, > > > > >> >>> Mayank > > > > >> >>> > > > > >> >>> > > > > >> >>> On Wed, Jun 6, 2012 at 7:25 AM, Manu S > > > wrote: > > > > >> >>> > > > > >> >>>> Hi All, > > > > >> >>>> > > > > >> >>>> We are running a mapreduce job in a fully distributed > > cluster.The > > > > >> output > > > > >> >>>> of the job is writing to HBase. > > > > >> >>>> > > > > >> >>>> While running this job we are getting an error: > > > > >> >>>> > > > > >> >>>> *Caused by: > > org.apache.hadoop.hbase.ZooKeeperConnectionException: > > > > >> HBase is able to connect to ZooKeeper but the connection closes > > > > >> immediately. This could be a sign that the server has too many > > > > connections > > > > >> (30 is the default). Consider inspecting your ZK server logs for > > that > > > > error > > > > >> and then make sure you are reusing HBaseConfiguration as often a= s > > you > > > > can. > > > > >> See HTable's javadoc for more information.* > > > > >> >>>> at > > > > >> > > > > > > > > > > org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.(ZooKeeperWatche= r.java:155) > > > > >> >>>> at > > > > >> > > > > > > > > > > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementati= on.getZooKeeperWatcher(HConnectionManager.java:1002) > > > > >> >>>> at > > > > >> > > > > > > > > > > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementati= on.setupZookeeperTrackers(HConnectionManager.java:304) > > > > >> >>>> at > > > > >> > > > > > > > > > > org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementati= on.(HConnectionManager.java:295) > > > > >> >>>> at > > > > >> > > > > > > > > > > org.apache.hadoop.hbase.client.HConnectionManager.getConnection(HConnecti= onManager.java:157) > > > > >> >>>> at > > > > org.apache.hadoop.hbase.client.HTable.(HTable.java:169) > > > > >> >>>> at > > > > >> > > > > > > > > > > org.apache.hadoop.hbase.client.HTableFactory.createHTableInterface(HTable= Factory.java:36) > > > > >> >>>> > > > > >> >>>> > > > > >> >>>> I had gone through some threads related to this issue and I > > > > modified > > > > >> the > > > > >> >>>> *zoo.cfg* accordingly. These configurations are same in all > the > > > > nodes. > > > > >> >>>> Please find the configuration of HBase & ZooKeeper: > > > > >> >>>> > > > > >> >>>> Hbase-site.xml: > > > > >> >>>> > > > > >> >>>> > > > > >> >>>> > > > > >> >>>> > > > > >> >>>> hbase.cluster.distributed > > > > >> >>>> true > > > > >> >>>> > > > > >> >>>> > > > > >> >>>> > > > > >> >>>> hbase.rootdir > > > > >> >>>> hdfs://namenode/hbase > > > > >> >>>> > > > > >> >>>> > > > > >> >>>> > > > > >> >>>> hbase.zookeeper.quorum > > > > >> >>>> namenode > > > > >> >>>> > > > > >> >>>> > > > > >> >>>> > > > > >> >>>> > > > > >> >>>> > > > > >> >>>> Zoo.cfg: > > > > >> >>>> > > > > >> >>>> # The number of milliseconds of each tick > > > > >> >>>> tickTime=3D2000 > > > > >> >>>> # The number of ticks that the initial > > > > >> >>>> # synchronization phase can take > > > > >> >>>> initLimit=3D10 > > > > >> >>>> # The number of ticks that can pass between > > > > >> >>>> # sending a request and getting an acknowledgement > > > > >> >>>> syncLimit=3D5 > > > > >> >>>> # the directory where the snapshot is stored. > > > > >> >>>> dataDir=3D/var/zookeeper > > > > >> >>>> # the port at which the clients will connect > > > > >> >>>> clientPort=3D2181 > > > > >> >>>> #server.0=3Dlocalhost:2888:3888 > > > > >> >>>> server.0=3Dnamenode:2888:3888 > > > > >> >>>> > > > > >> >>>> ################# Max Client connections ##################= # > > > > >> >>>> *maxClientCnxns=3D1000 > > > > >> >>>> minSessionTimeout=3D4000 > > > > >> >>>> maxSessionTimeout=3D40000* > > > > >> >>>> > > > > >> >>>> > > > > >> >>>> It would be really great if anyone can help me to resolve > this > > > > issue > > > > >> by > > > > >> >>>> giving your thoughts/suggestions. > > > > >> >>>> > > > > >> >>>> Thanks, > > > > >> >>>> Manu S > > > > >> >>>> > > > > >> >>> > > > > >> >>> > > > > >> >> > > > > >> > > > > > > > > > > > > > > > > -- > > > > > > > > > =E2=88=9E > > > Shashwat Shriparv > > > > > > > > > -- > > > =E2=88=9E > Shashwat Shriparv > --0016e6dee847fd290804c2e08f5f--