Return-Path: X-Original-To: apmail-hbase-user-archive@www.apache.org Delivered-To: apmail-hbase-user-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id F0894110FA for ; Sat, 20 Sep 2014 05:45:17 +0000 (UTC) Received: (qmail 45110 invoked by uid 500); 20 Sep 2014 05:45:15 -0000 Delivered-To: apmail-hbase-user-archive@hbase.apache.org Received: (qmail 45042 invoked by uid 500); 20 Sep 2014 05:45:15 -0000 Mailing-List: contact user-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@hbase.apache.org Delivered-To: mailing list user@hbase.apache.org Received: (qmail 45024 invoked by uid 99); 20 Sep 2014 05:45:15 -0000 Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 20 Sep 2014 05:45:15 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (athena.apache.org: domain of lhofhansl@yahoo.com designates 98.139.212.163 as permitted sender) Received: from [98.139.212.163] (HELO nm4.bullet.mail.bf1.yahoo.com) (98.139.212.163) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 20 Sep 2014 05:45:08 +0000 Received: from [98.139.212.150] by nm4.bullet.mail.bf1.yahoo.com with NNFMP; 20 Sep 2014 05:44:47 -0000 Received: from [98.139.212.208] by tm7.bullet.mail.bf1.yahoo.com with NNFMP; 20 Sep 2014 05:44:47 -0000 Received: from [127.0.0.1] by omp1017.mail.bf1.yahoo.com with NNFMP; 20 Sep 2014 05:44:47 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 411591.96829.bm@omp1017.mail.bf1.yahoo.com Received: (qmail 6355 invoked by uid 60001); 20 Sep 2014 05:44:47 -0000 X-YMail-OSG: JucckzsVM1kgdOz7gPUhJE8qnPDSdyU1IKC76XBednILXjC lsZoNZnxuZveD9yuzfkSkUEA7vbL6u5.COGP_4qdAzdTkfigrYRXUBw43ZxH nAL7QUFlSzcvAmkDEviXhb3WtWNFOPyXQ6_8QqBITcTTebyiMEbuHqwCNjef AJZK1vJbpFr4T0kLO4Ax7gv7fSQjZ1OzdoLp.0up7NVXbAyyVrbBz8GSkw1Q QY4Vlk7xWm2wlHkEx_cq7A5aRHmdXBFmLgRQZXM9e91cNFpfw6tXQV7cHG2P FOOU6NcJFJqeWQHhS86S1BLseRZhpAeqZCrqU2nmwG8tZngTHwOYJ4f7Om44 lEEC6yvfu1y.yyMuMrPF7ikhbM8rkHKYte5Lkrb6BkMm7QRUaTzJzWqo771K mboSzDdsgYZB3aRjnOmTd71cHQzMn8iJQHGoOAxCRtepRgVDZAM0ngslKewG axAZEULKfnoy1OGuV0.d5WjhJoDbomJqkBhgZbiIjG_ysHPB4iuGEa16R_rg 5UF30jPHrKxY7LchMoTaxN5hRDfyVX0M95jBsM.NBk6sOKb55E4IZHP.jOyD YxpliU5xrQo4COEmxKpGgT35UB51kCccsrXGp6HV_tfnFsuQEImL3rqY0qHi plerV7QTtFk.FdXR2lLNx2yDQM3LEivpJhgIPSbeVw86j5tttkLerpYGEoXs kbAUgjZqhEyzYP04bJFInxrN74xEoE4ZZHkn747g2So9.rviIDBnjFyd1Ygu DiDDZ7YwjxUdWr_bakwluFPWQf42rD8GmWnZuDP0I3WZnBaOXl2dj896j03L W5o8WVb5AdSTT806DmwDPDSffQQ5kWpALTiOPqOoRSMA3pd4gNo12EiHGSTn kMQJH9nAYtYRXA_1_abYP3A-- Received: from [24.4.160.78] by web140602.mail.bf1.yahoo.com via HTTP; Fri, 19 Sep 2014 22:44:47 PDT X-Rocket-MIMEInfo: 002.001,SGksCgpjYW4geW91IGRlZmluZSAiZnJlcXVlbnRseSI_CkkuZS4gc2VuZCBhIGxhcmdlciBzbmlwcGV0IG9mIHRoZSBsb2cuIENvbm5lY3RpbmcgZXZlcnkgZmV3IG1pbnV0ZXMgd291bGQgT0ssIE11bHRpcGxlIHRpbWVzIHBlciBzZWNvbmQgd291bGQgYmUgc3RyYW5nZS4KCi0tIExhcnMKCgoKX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18KIEZyb206IHRvYmUgPHRvYmVnM29vZ2xlQGdtYWlsLmNvbT4KVG86ICJ1c2VyQGhiYXNlLmFwYWNoZS5vcmciIDx1c2VyQGhiYXNlLmFwYWNoZS5vcmc.IAoBMAEBAQE- X-RocketYMMF: lhofhansl X-Mailer: YahooMailWebService/0.8.203.696 References: Message-ID: <1411191887.72472.YahooMailNeo@web140602.mail.bf1.yahoo.com> Date: Fri, 19 Sep 2014 22:44:47 -0700 From: lars hofhansl Reply-To: lars hofhansl Subject: Re: HBase establishes session with ZooKeeper and close the session immediately To: "user@hbase.apache.org" In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="-118416272-1378206587-1411191887=:72472" X-Virus-Checked: Checked by ClamAV on apache.org ---118416272-1378206587-1411191887=:72472 Content-Type: text/plain; charset=us-ascii Hi, can you define "frequently"? I.e. send a larger snippet of the log. Connecting every few minutes would OK, Multiple times per second would be strange. -- Lars ________________________________ From: tobe To: "user@hbase.apache.org" Sent: Thursday, September 18, 2014 1:50 AM Subject: HBase establishes session with ZooKeeper and close the session immediately I have found that our RegionServers connect to the ZooKeeper frequently. They seems to constantly establish the session, close it and reconnect the ZooKeeper. Here is the log for both server and client sides. I have no idea why this happens and how to deal with it? We're using HBase 0.94.11 and ZooKeeper 3.4.4. The log from HBase RegionServer: 2014-09-18,16:38:17,867 INFO org.apache.zookeeper.ZooKeeper: Initiating client connection, connectString=10.2.201.74:11000,10.2.201.73:11000, 10.101.10.67:11000,10.101.10.66:11000,10.2.201.75:11000 sessionTimeout=30000 watcher=catalogtracker-on-org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@69d892a1 2014-09-18,16:38:17,868 INFO org.apache.zookeeper.client.ZooKeeperSaslClient: Client will use GSSAPI as SASL mechanism. 2014-09-18,16:38:17,868 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server lg-hadoop-srv-ct01.bj/10.2.201.73:11000. Will attempt to SASL-authenticate using Login Context section 'Client' 2014-09-18,16:38:17,868 INFO org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: The identifier of this process is 11787@lg-hadoop-srv-st05.bj 2014-09-18,16:38:17,868 INFO org.apache.zookeeper.ClientCnxn: Socket connection established to lg-hadoop-srv-ct01.bj/10.2.201.73:11000, initiating session 2014-09-18,16:38:17,870 INFO org.apache.zookeeper.ClientCnxn: Session establishment complete on server lg-hadoop-srv-ct01.bj/10.2.201.73:11000, sessionid = 0x248782700e52b3c, negotiated timeout = 30000 2014-09-18,16:38:17,876 INFO org.apache.zookeeper.ZooKeeper: Session: 0x248782700e52b3c closed 2014-09-18,16:38:17,876 INFO org.apache.zookeeper.ClientCnxn: EventThread shut down 2014-09-18,16:38:17,878 INFO org.apache.hadoop.hbase.replication.regionserver.ReplicationSink: Total replicated: 24 The log from its ZooKeeper server: 2014-09-18,16:38:17,869 INFO org.apache.zookeeper.server.NIOServerCnxnFactory: [myid:2] Accepted socket connection from /10.2.201.76:55621 2014-09-18,16:38:17,869 INFO org.apache.zookeeper.server.ZooKeeperServer: [myid:2] Client attempting to establish new session at /10.2.201.76:55621 2014-09-18,16:38:17,870 INFO org.apache.zookeeper.server.ZooKeeperServer: [myid:2] Established session 0x248782700e52b3c with negotiated timeout 30000 for client /10.2.201.76:55621 2014-09-18,16:38:17,872 INFO org.apache.zookeeper.server.auth.SaslServerCallbackHandler: [myid:2] Successfully authenticated client: authenticationID=hbase_srv/hadoop@XIAOMI.HADOOP; authorizationID=hbase_srv/hadoop@XIAOMI.HADOOP. 2014-09-18,16:38:17,872 INFO org.apache.zookeeper.server.auth.SaslServerCallbackHandler: [myid:2] Setting authorizedID: hbase_srv 2014-09-18,16:38:17,872 INFO org.apache.zookeeper.server.ZooKeeperServer: [myid:2] adding SASL authorization for authorizationID: hbase_srv 2014-09-18,16:38:17,877 INFO org.apache.zookeeper.server.NIOServerCnxn: [myid:2] Closed socket connection for client /10.2.201.76:55621 which had sessionid 0x248782700e52b3c ---118416272-1378206587-1411191887=:72472--