Return-Path: X-Original-To: apmail-hbase-dev-archive@www.apache.org Delivered-To: apmail-hbase-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id BE3F810953 for ; Wed, 4 Mar 2015 10:11:43 +0000 (UTC) Received: (qmail 20368 invoked by uid 500); 4 Mar 2015 10:11:40 -0000 Delivered-To: apmail-hbase-dev-archive@hbase.apache.org Received: (qmail 20277 invoked by uid 500); 4 Mar 2015 10:11:40 -0000 Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@hbase.apache.org Delivered-To: mailing list dev@hbase.apache.org Received: (qmail 20266 invoked by uid 99); 4 Mar 2015 10:11:39 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 04 Mar 2015 10:11:39 +0000 X-ASF-Spam-Status: No, hits=2.8 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS,URI_HEX X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of esteban@cloudera.com designates 209.85.215.47 as permitted sender) Received: from [209.85.215.47] (HELO mail-la0-f47.google.com) (209.85.215.47) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 04 Mar 2015 10:11:15 +0000 Received: by labgf13 with SMTP id gf13so18839528lab.5 for ; Wed, 04 Mar 2015 02:10:28 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:content-type; bh=HEbMHzdywe3iDMetMvW9m/Cbey6xPx0b9TwqRNTsWok=; b=diOMizc0eP7xhZ+aZIz5BtOxeIvHmTTnvltgAw7mM6g0Hq6MQUSqQFMML1PuAQOd3l xfNA8bf8/dcHFGFc94/biLXao/8rQ6cOf7pEs/7BivQaGIU7RV5wK7OGDTsRS5oev7eI MlND1GtyCkY5zYWLK9e0fa7d1TdwzS/mJCdHDHLc8VUkU+6QuVbEb0tAuT45gazv2GOO u494G+nLQWiV2sZf5zHw+EQTiBgPp40Jn/hw2nXcuVHdfopJ168lVvjjDzlxWM16oVYe 2PTba2poDjh1ogCoursIds3wVeflxqaQ/9OpPnbA+E4ABjLAGXYV8aOR/pgbH4qohbHk GU9A== X-Gm-Message-State: ALoCoQkZKvvgSw9zT8A2D2297uWU0aOwmJJOOpyepBMrXP5lDuArKWa6IOL3m0dRweX+G+6cJGcb X-Received: by 10.112.150.131 with SMTP id ui3mr2765270lbb.62.1425463828659; Wed, 04 Mar 2015 02:10:28 -0800 (PST) MIME-Version: 1.0 Received: by 10.25.25.19 with HTTP; Wed, 4 Mar 2015 02:10:13 -0800 (PST) In-Reply-To: <1425452560942-4068834.post@n3.nabble.com> References: <1425452560942-4068834.post@n3.nabble.com> From: Esteban Gutierrez Date: Wed, 4 Mar 2015 02:10:13 -0800 Message-ID: Subject: Re: Hbase Regionserver shutting down To: "user@hbase.apache.org" Content-Type: multipart/alternative; boundary=047d7b3a810825cc2d051073a955 X-Virus-Checked: Checked by ClamAV on apache.org --047d7b3a810825cc2d051073a955 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable -dev +user Hi, Please try upload the logs from this RegionServer to pastebin.com or gist.github.com so we can look into that. Also which version of HBase are you using? cheers, esteban. -- Cloudera, Inc. On Tue, Mar 3, 2015 at 11:02 PM, shivanandpawar wrote: > We have installed a small Hadoop cluster with 3 machines. > > We use the following code to connect to Hbase from java client. > Configuration config =3D HBaseConfiguration.create() > config.set(=E2=80=9Chbase.zookeeper.quorum=E2=80=9D, =E2=80=9Cxxxxxx=E2= =80=9D); > config.set(=E2=80=9Chbase.zookeeper.property.clientPort=E2=80=9D, =E2=80= =9C2181=E2=80=9D); > config.set(=E2=80=9Chbase.client.operation.timeout=E2=80=9D, =E2=80=9C100= 0=E2=80=9D); > HConnection connection =3D HConnectionManager.createConnection(config); > HTableInterface hTable =3D connection.getTable("tableName"); > > But somehow the Hbase regionservers automatically shuts down after some > time > giving the following error. > 2015-02-23 03:38:49,472 WARN [regionserver60020] > zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper, > > quorum=3Dip-10-174-18-85.ap-southeast-1.compute.internal:2181,ip-10-149-1= 30-178.ap-southeast-1.compute.internal:2181,ip-10-174-28-180.ap-southeast-1= .compute.internal:2181, > exception=3Dorg.apache.zookeeper.KeeperException$SessionExpiredException: > KeeperErrorCode =3D Session expired for > > /hbase/rs/ip-10-174-28-180.ap-southeast-1.compute.internal,60020,14246800= 62858 > 2015-02-23 03:38:49,472 ERROR [regionserver60020] > zookeeper.RecoverableZooKeeper: ZooKeeper delete failed after 4 attempts > 2015-02-23 03:38:49,472 WARN [regionserver60020] > regionserver.HRegionServer: > Failed deleting my ephemeral node > org.apache.zookeeper.KeeperException$SessionExpiredException: > KeeperErrorCode =3D Session expired for > > /hbase/rs/ip-10-174-28-180.ap-southeast-1.compute.internal,60020,14246800= 62858 > at org.apache.zookeeper.KeeperException.create(KeeperException.java:127) > at org.apache.zookeeper.KeeperException.create(KeeperException.java:51) > at org.apache.zookeeper.ZooKeeper.delete(ZooKeeper.java:873) > at > > org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.delete(Recoverable= ZooKeeper.java:156) > at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1270) > at org.apache.hadoop.hbase.zookeeper.ZKUtil.deleteNode(ZKUtil.java:1259) > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.deleteMyEphemeralNode(= HRegionServer.java:1286) > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java= :1016) > at java.lang.Thread.run(Thread.java:744) > 2015-02-23 03:39:10,233 INFO [regionserver60020] > regionserver.HRegionServer: > stopping server > ip-10-174-28-180.ap-southeast-1.compute.internal,60020,1424680062858; > zookeeper connection closed. > 2015-02-23 03:39:10,233 INFO [regionserver60020] > regionserver.HRegionServer: > regionserver60020 exiting > 2015-02-23 03:39:10,236 ERROR [main] regionserver.HRegionServerCommandLin= e: > Region server exiting > java.lang.RuntimeException: HRegionServer Aborted > at > > org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegi= onServerCommandLine.java:66) > at > > org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegion= ServerCommandLine.java:85) > at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) > at > > org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.j= ava:126) > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.jav= a:2403) > 2015-02-23 03:39:10,240 INFO [Thread-11] regionserver.ShutdownHook: > Shutdown > hook starting; hbase.shutdown.hook=3Dtrue; > > fsShutdownHook=3Dorg.apache.hadoop.fs.FileSystem$Cache$ClientFinalizer@ab= de0e7 > > I tried increasing the JVM Heap size of regioserver to 16GB and session > time > out to 120s. But nothing seems to help. > > > Please help to get the issue resolved?? > > > > -- > View this message in context: > http://apache-hbase.679495.n3.nabble.com/Hbase-Regionserver-shutting-down= -tp4068834.html > Sent from the HBase Developer mailing list archive at Nabble.com. > --047d7b3a810825cc2d051073a955--