Mailing-List: contact user-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: user@hbase.apache.org
Received-SPF: error (nike.apache.org: local policy)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
  s=s1024; d=yahoo.co.in;
  h=X-YMail-OSG:Received:X-Rocket-MIMEInfo:X-Mailer:References:Message-ID:Date:From:Reply-To:Subject:To:In-Reply-To:MIME-Version:Content-Type:Content-Transfer-Encoding;
  b=EGI0zxNZhJB6dXwznwUyIbqjjoU4YdW2z+nUw65tD/HZgU5+nb132R6hkw0qIj8ETJFj/b251NrjVMS4JaVvxS80aSmy5AvUWFf34IDQpgSqHSgtVmTEP++Ln50AgL7oGbrLV6rm9RCS7yQGxZf53syzXrdOkN3bbBx0pcH8ewk=;
References: 
 <CAPVFMeXUOB-xAOZdA-2LjDdz0xLZRLS4MszG0FSpKh=WJBL+VA@mail.gmail.com>
	<201308232129388403141@gmail.com>
	<CAPVFMeVTPW7pUg4wgWwKX=_j4GetK20-6ip0ZkTDn_8ixnTtbA@mail.gmail.com>
	<1377267054.52012.YahooMailNeo@web190106.mail.sg3.yahoo.com>
 <CAPVFMeW4HGHgTdXt+f+yN=pe6LWVLAwH=XqYhbz_ZU7nd34e8w@mail.gmail.com>
Message-ID: <1377269358.57623.YahooMailNeo@web190106.mail.sg3.yahoo.com>
Date: Fri, 23 Aug 2013 22:49:18 +0800 (SGT)
From: Dhaval Shah <prince_mithibai@yahoo.co.in>
Reply-To: Dhaval Shah <prince_mithibai@yahoo.co.in>
Subject: Re: Will hbase automatically distribute the data across region
 servers or NOT..??
To: "user@hbase.apache.org" <user@hbase.apache.org>
In-Reply-To: 
 <CAPVFMeW4HGHgTdXt+f+yN=pe6LWVLAwH=XqYhbz_ZU7nd34e8w@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: quoted-printable

Ok. The balancer runs as a separate thread (there is a config to set how of=
ten the thread wakes up but can't remember off the top of my head). Maybe i=
f you wait long enough, it will balance eventually. Another thing you can t=
ry is run the balancer from hbase shell and see what you get back. If you g=
et back a true, it means it should balance. If you get back a false, look a=
t hbase master logs to see whats happening. I once had a scenario where my =
Unix accounts were messed up (2 users - hbase and another user mapped to th=
e same unix ID and HDFS thought the user did not have the permissions to wr=
ite to the HBase files on HDFS) and balancer did not run due to this except=
ion.=A0=0A=0AAnother thing is (I think!) balancer generally does not run wh=
en regions are splitting. So its possible in your case that your regions ar=
e splitting so often (due to 10MB limit) that the balancer cannot be run si=
nce your regions are not stationary=0A=0A=0ARegards,=0ADhaval=0A=0A=0A_____=
___________________________=0AFrom: Vamshi Krishna <vamshi2105@gmail.com>=
=0ATo: user@hbase.apache.org; Dhaval Shah <prince_mithibai@yahoo.co.in> =0A=
Sent: Friday, 23 August 2013 10:21 AM=0ASubject: Re: Will hbase automatical=
ly distribute the data across region servers or NOT..??=0A=0A=0ANo that is =
10MB itself. Just to observe the region splitting with respect=0Ato the amo=
unt of data i am inserting in to hbase.=0ASo, here i am inserting 40-50mb d=
ata and fixing that property to 10mb and=0Achecking the region splitting ha=
ppening.=0ABut the intersting thing is regions got split BUT they are not b=
eing=0Adistributed across other servers.=0AWhatever regions formed from the=
 created tables on machine-1, all of them=0Aare residing on the same machin=
e-1 not being moved to other machine.=0A=0A=0A=0A=0AOn Fri, Aug 23, 2013 at=
 7:40 PM, Dhaval Shah <prince_mithibai@yahoo.co.in>wrote:=0A=0A> Vamshi, ma=
x value for hbase.hregion.max.filesize to 10MB seems too small.=0A> Did you=
 mean 10GB?=0A>=0A>=0A> Regards,=0A> Dhaval=0A>=0A>=0A> ___________________=
_____________=0A> From: Vamshi Krishna <vamshi2105@gmail.com>=0A> To: user@=
hbase.apache.org; zhoushuaifeng <zhoushuaifeng@gmail.com>=0A> Sent: Friday,=
 23 August 2013 9:38 AM=0A> Subject: Re: Will hbase automatically distribut=
e the data across region=0A> servers or NOT..??=0A>=0A>=0A> Thanks for the =
clarifications.=0A> I am using hbase-0.94.10 and zookeepr-3.4.5=0A> But I a=
m running into different issues .=0A> I set=A0 hbase.hregion.max.filesize t=
o 10Mb and i am inserting 10 million=0A> rows in to hbase table. During the=
 insertion after some time, suddenly=0A> master is going down. I don't know=
 what is the reason for such peculiar=0A> behavior.=0A> I found in master l=
og below content and not able to make out what exactly=0A> the mistake. Ple=
ase somebody help.=0A>=0A> master-log:=0A>=0A> 2013-08-23 18:56:36,865 FATA=
L org.apache.hadoop.hbase.master.HMaster:=0A> Master server abort: loaded c=
oprocessors are: []=0A> 2013-08-23 18:56:36,866 FATAL org.apache.hadoop.hba=
se.master.HMaster:=0A> Unexpected state :=0A>=0A> scores,\x00\x00\x00\x00\x=
00\x02\xC8t,1377264003140.a564f31795091b6513880c5db49ec90f.=0A> state=3DPEN=
DING_OPEN, ts=3D1377264396861, server=3Dvamshi,60020,1377263789273 ..=0A> C=
annot transit it to OFFLINE.=0A> java.lang.IllegalStateException: Unexpecte=
d state :=0A>=0A> scores,\x00\x00\x00\x00\x00\x02\xC8t,1377264003140.a564f3=
1795091b6513880c5db49ec90f.=0A> state=3DPENDING_OPEN, ts=3D1377264396861, s=
erver=3Dvamshi,60020,1377263789273 ..=0A> Cannot transit it to OFFLINE.=0A>=
=A0 =A0=A0=A0at=0A>=0A> org.apache.hadoop.hbase.master.AssignmentManager.se=
tOfflineInZooKeeper(AssignmentManager.java:1879)=0A>=A0 =A0=A0=A0at=0A>=0A>=
 org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.=
java:1688)=0A>=A0 =A0=A0=A0at=0A>=0A> org.apache.hadoop.hbase.master.Assign=
mentManager.assign(AssignmentManager.java:1424)=0A>=A0 =A0=A0=A0at=0A>=0A> =
org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.j=
ava:1399)=0A>=A0 =A0=A0=A0at=0A>=0A> org.apache.hadoop.hbase.master.Assignm=
entManager.assign(AssignmentManager.java:1394)=0A>=A0 =A0=A0=A0at=0A>=0A> o=
rg.apache.hadoop.hbase.master.handler.ClosedRegionHandler.process(ClosedReg=
ionHandler.java:105)=0A>=A0 =A0=A0=A0at=0A> org.apache.hadoop.hbase.executo=
r.EventHandler.run(EventHandler.java:175)=0A>=A0 =A0=A0=A0at=0A>=0A> java.u=
til.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:89=
5)=0A>=A0 =A0=A0=A0at=0A>=0A> java.util.concurrent.ThreadPoolExecutor$Worke=
r.run(ThreadPoolExecutor.java:918)=0A>=A0 =A0=A0=A0at java.lang.Thread.run(=
Thread.java:662)=0A> 2013-08-23 18:56:36,867 INFO org.apache.hadoop.hbase.m=
aster.HMaster:=0A> Aborting=0A> 2013-08-23 18:56:36,867 DEBUG org.apache.ha=
doop.hbase.master.HMaster:=0A> Stopping service threads=0A> 2013-08-23 18:5=
6:36,867 INFO org.apache.hadoop.ipc.HBaseServer: Stopping=0A> server on 600=
00=0A> 2013-08-23 18:56:36,867 INFO org.apache.hadoop.ipc.HBaseServer: IPC =
Server=0A> handler 0 on 60000: exiting=0A> 2013-08-23 18:56:36,867 INFO org=
.apache.hadoop.ipc.HBaseServer: IPC Server=0A> handler 5 on 60000: exiting=
=0A> 2013-08-23 18:56:36,867 INFO org.apache.hadoop.ipc.HBaseServer: IPC Se=
rver=0A> handler 3 on 60000: exiting=0A> 2013-08-23 18:56:36,873 INFO org.a=
pache.hadoop.ipc.HBaseServer: Stopping=0A> IPC Server listener on 60000=0A>=
 2013-08-23 18:56:36,873 INFO org.apache.hadoop.ipc.HBaseServer: REPL IPC=
=0A> Server handler 2 on 60000: exiting=0A> 2013-08-23 18:56:36,873 INFO or=
g.apache.hadoop.ipc.HBaseServer: REPL IPC=0A> Server handler 1 on 60000: ex=
iting=0A> 2013-08-23 18:56:36,873 INFO org.apache.hadoop.hbase.master.HMast=
er$2:=0A> vamshi,60000,1377263788019-BalancerChore exiting=0A> 2013-08-23 1=
8:56:36,873 INFO org.apache.hadoop.hbase.master.HMaster:=0A> Stopping infoS=
erver=0A> 2013-08-23 18:56:36,873 INFO=0A> org.apache.hadoop.hbase.master.c=
leaner.HFileCleaner:=0A> master-vamshi,60000,1377263788019.archivedHFileCle=
aner exiting=0A> 2013-08-23 18:56:36,873 INFO org.apache.hadoop.hbase.maste=
r.CatalogJanitor:=0A> vamshi,60000,1377263788019-CatalogJanitor exiting=0A>=
 2013-08-23 18:56:36,873 INFO org.apache.hadoop.ipc.HBaseServer: REPL IPC=
=0A> Server handler 0 on 60000: exiting=0A> 2013-08-23 18:56:36,873 INFO or=
g.apache.hadoop.ipc.HBaseServer: IPC Server=0A> handler 9 on 60000: exiting=
=0A> 2013-08-23 18:56:36,874 INFO org.mortbay.log: Stopped=0A> SelectChanne=
lConnector@0.0.0.0:60010=0A> 2013-08-23 18:56:36,874 INFO=0A> org.apache.ha=
doop.hbase.master.cleaner.LogCleaner:=0A> master-vamshi,60000,1377263788019=
.oldLogCleaner exiting=0A> 2013-08-23 18:56:36,874 INFO org.apache.hadoop.i=
pc.HBaseServer: IPC Server=0A> handler 1 on 60000: exiting=0A> 2013-08-23 1=
8:56:36,874 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server=0A> handler =
7 on 60000: exiting=0A> 2013-08-23 18:56:36,874 INFO org.apache.hadoop.ipc.=
HBaseServer: IPC Server=0A> handler 6 on 60000: exiting=0A> 2013-08-23 18:5=
6:36,874 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server=0A> handler 8 o=
n 60000: exiting=0A> 2013-08-23 18:56:36,874 INFO org.apache.hadoop.ipc.HBa=
seServer: Stopping=0A> IPC Server Responder=0A> 2013-08-23 18:56:36,876 INF=
O org.apache.hadoop.ipc.HBaseServer: Stopping=0A> IPC Server Responder=0A> =
2013-08-23 18:56:36,874 INFO org.apache.hadoop.ipc.HBaseServer: IPC Server=
=0A> handler 2 on 60000: exiting=0A> 2013-08-23 18:56:36,873 INFO org.apach=
e.hadoop.ipc.HBaseServer: IPC Server=0A> handler 4 on 60000: exiting=0A> 20=
13-08-23 18:56:36,877 WARN org.apache.hadoop.hbase.zookeeper.ZKUtil:=0A> ma=
ster:60000-0x140ab519b0f0000 Unable to set watcher on znode=0A> (/hbase/una=
ssigned/05e30711673614f6b41a364c76f3f05f)=0A> java.lang.InterruptedExceptio=
n=0A>=A0 =A0=A0=A0at java.lang.Object.wait(Native Method)=0A>=A0 =A0=A0=A0a=
t java.lang.Object.wait(Object.java:485)=0A>=A0 =A0=A0=A0at org.apache.zook=
eeper.ClientCnxn.submitRequest(ClientCnxn.java:1309)=0A>=A0 =A0=A0=A0at org=
.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1036)=0A>=A0 =A0=A0=A0at=
=0A>=0A> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(Reco=
verableZooKeeper.java:172)=0A>=A0 =A0=A0=A0at=0A> org.apache.hadoop.hbase.z=
ookeeper.ZKUtil.checkExists(ZKUtil.java:450)=0A>=A0 =A0=A0=A0at=0A>=0A> org=
.apache.hadoop.hbase.zookeeper.ZKAssign.createOrForceNodeOffline(ZKAssign.j=
ava:271)=0A>=A0 =A0=A0=A0at=0A>=0A> org.apache.hadoop.hbase.master.Assignme=
ntManager.setOfflineInZooKeeper(AssignmentManager.java:1905)=0A>=A0 =A0=A0=
=A0at=0A>=0A> org.apache.hadoop.hbase.master.AssignmentManager.assign(Assig=
nmentManager.java:1688)=0A>=A0 =A0=A0=A0at=0A>=0A> org.apache.hadoop.hbase.=
master.AssignmentManager.assign(AssignmentManager.java:1424)=0A>=A0 =A0=A0=
=A0at=0A>=0A> org.apache.hadoop.hbase.master.AssignmentManager.assign(Assig=
nmentManager.java:1399)=0A>=A0 =A0=A0=A0at=0A>=0A> org.apache.hadoop.hbase.=
master.AssignmentManager.assign(AssignmentManager.java:1394)=0A>=A0 =A0=A0=
=A0at=0A>=0A> org.apache.hadoop.hbase.master.handler.ClosedRegionHandler.pr=
ocess(ClosedRegionHandler.java:105)=0A>=A0 =A0=A0=A0at=0A> org.apache.hadoo=
p.hbase.executor.EventHandler.run(EventHandler.java:175)=0A>=A0 =A0=A0=A0at=
=0A>=0A> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolE=
xecutor.java:895)=0A>=A0 =A0=A0=A0at=0A>=0A> java.util.concurrent.ThreadPoo=
lExecutor$Worker.run(ThreadPoolExecutor.java:918)=0A>=A0 =A0=A0=A0at java.l=
ang.Thread.run(Thread.java:662)=0A> 2013-08-23 18:56:36,876 WARN=0A> org.ap=
ache.hadoop.hbase.master.AssignmentManager: Attempted to create/force=0A> n=
ode into OFFLINE state before completing assignment but failed to do so=0A>=
 for=0A>=0A> scores,\x00\x00\x00\x00\x00\x08b8,1377264147374.39794b7deea320=
3fc260756f5038d6f8.=0A> state=3DOFFLINE, ts=3D1377264396802, server=3Dnull=
=0A> 2013-08-23 18:56:36,876 WARN org.apache.hadoop.hbase.zookeeper.ZKUtil:=
=0A> master:60000-0x140ab519b0f0000 Unable to get data of znode=0A> /hbase/=
unassigned/d476f8442ce31de90b60080b74daf47f=0A> java.lang.InterruptedExcept=
ion=0A>=A0 =A0=A0=A0at java.lang.Object.wait(Native Method)=0A>=A0 =A0=A0=
=A0at java.lang.Object.wait(Object.java:485)=0A>=A0 =A0=A0=A0at org.apache.=
zookeeper.ClientCnxn.submitRequest(ClientCnxn.java:1309)=0A>=A0 =A0=A0=A0at=
 org.apache.zookeeper.ZooKeeper.getData(ZooKeeper.java:1149)=0A>=A0 =A0=A0=
=A0at=0A>=0A> org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.getDat=
a(RecoverableZooKeeper.java:290)=0A>=A0 =A0=A0=A0at=0A> org.apache.hadoop.h=
base.zookeeper.ZKUtil.getDataNoWatch(ZKUtil.java:746)=0A>=A0 =A0=A0=A0at=0A=
>=0A> org.apache.hadoop.hbase.zookeeper.ZKAssign.getDataNoWatch(ZKAssign.ja=
va:904)=0A>=A0 =A0=A0=A0at=0A>=0A> org.apache.hadoop.hbase.zookeeper.ZKAssi=
gn.createOrForceNodeOffline(ZKAssign.java:283)=0A>=A0 =A0=A0=A0at=0A>=0A> o=
rg.apache.hadoop.hbase.master.AssignmentManager.setOfflineInZooKeeper(Assig=
nmentManager.java:1905)=0A>=A0 =A0=A0=A0at=0A>=0A> org.apache.hadoop.hbase.=
master.AssignmentManager.assign(AssignmentManager.java:1688)=0A>=A0 =A0=A0=
=A0at=0A>=0A> org.apache.hadoop.hbase.master.AssignmentManager.assign(Assig=
nmentManager.java:1424)=0A>=A0 =A0=A0=A0at=0A>=0A> org.apache.hadoop.hbase.=
master.AssignmentManager.assign(AssignmentManager.java:1399)=0A>=A0 =A0=A0=
=A0at=0A>=0A> org.apache.hadoop.hbase.master.AssignmentManager.assign(Assig=
nmentManager.java:1394)=0A>=A0 =A0=A0=A0at=0A>=0A> org.apache.hadoop.hbase.=
master.handler.ClosedRegionHandler.process(ClosedRegionHandler.java:105)=0A=
>=A0 =A0=A0=A0at=0A> org.apache.hadoop.hbase.executor.EventHandler.run(Even=
tHandler.java:175)=0A>=A0 =A0=A0=A0at=0A>=0A> java.util.concurrent.ThreadPo=
olExecutor$Worker.runTask(ThreadPoolExecutor.java:895)=0A>=A0 =A0=A0=A0at=
=0A>=0A> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecu=
tor.java:918)=0A>=A0 =A0=A0=A0at java.lang.Thread.run(Thread.java:662)=0A> =
2013-08-23 18:56:36,877 WARN=0A> org.apache.hadoop.hbase.master.AssignmentM=
anager: Attempted to create/force=0A> node into OFFLINE state before comple=
ting assignment but failed to do so=0A> for=0A>=0A> scores,\x00\x00\x00\x00=
\x00\x10\xC1\xF4,1377264146360.05e30711673614f6b41a364c76f3f05f.=0A> state=
=3DOFFLINE, ts=3D1377264396862, server=3Dnull=0A> 2013-08-23 18:56:36,877 W=
ARN=0A> org.apache.hadoop.hbase.master.AssignmentManager: Attempted to crea=
te/force=0A> node into OFFLINE state before completing assignment but faile=
d to do so=0A> for=0A>=0A> scores,\x00\x00\x00\x00\x00\x17\xC0i,13772643023=
91.d476f8442ce31de90b60080b74daf47f.=0A> state=3DOFFLINE, ts=3D137726439681=
3, server=3Dnull=0A> 2013-08-23 18:56:36,882 DEBUG=0A> org.apache.hadoop.hb=
ase.master.AssignmentManager: Handling=0A> transition=3DRS_ZK_REGION_FAILED=
_OPEN, server=3Dvamshi_RS,60020,1377263792053,=0A> region=3Dd476f8442ce31de=
90b60080b74daf47f=0A> 2013-08-23 18:56:36,882 DEBUG=0A> org.apache.hadoop.h=
base.master.AssignmentManager: Found an existing plan=0A> for=0A>=0A> score=
s,\x00\x00\x00\x00\x00\x17\xC0i,1377264302391.d476f8442ce31de90b60080b74daf=
47f.=0A> destination server is vamshi,60020,1377263789273=0A> 2013-08-23 18=
:56:36,882 DEBUG=0A> org.apache.hadoop.hbase.master.AssignmentManager: No p=
revious transition=0A> plan was found (or we are ignoring an existing plan)=
 for=0A>=0A> scores,\x00\x00\x00\x00\x00\x17\xC0i,1377264302391.d476f8442ce=
31de90b60080b74daf47f.=0A> so generated a random one;=0A>=0A> hri=3Dscores,=
\x00\x00\x00\x00\x00\x17\xC0i,1377264302391.d476f8442ce31de90b60080b74daf47=
f.,=0A> src=3D, dest=3Dvamshi,60020,1377263789273; 2 (online=3D2, available=
=3D1) available=0A> servers=0A> 2013-08-23 18:56:36,882 ERROR=0A> org.apach=
e.hadoop.hbase.executor.ExecutorService: Cannot submit=0A> [ClosedRegionHan=
dler-vamshi,60000,1377263788019-38] because the executor is=0A> missing. Is=
 this process shutting down?=0A> 2013-08-23 18:56:36,906 DEBUG=0A> org.apac=
he.hadoop.hbase.catalog.CatalogTracker: Stopping catalog tracker=0A> org.ap=
ache.hadoop.hbase.catalog.CatalogTracker@451415c8=0A> 2013-08-23 18:56:36,9=
06 INFO=0A> org.apache.hadoop.hbase.master.AssignmentManager$TimeoutMonitor=
:=0A> vamshi,60000,1377263788019.timeoutMonitor exiting=0A> 2013-08-23 18:5=
6:36,906 INFO=0A> org.apache.hadoop.hbase.master.AssignmentManager$TimerUpd=
ater:=0A> vamshi,60000,1377263788019.timerUpdater exiting=0A> 2013-08-23 18=
:56:36,907 INFO=0A> org.apache.hadoop.hbase.master.SplitLogManager$TimeoutM=
onitor:=0A> vamshi,60000,1377263788019.splitLogManagerTimeoutMonitor exitin=
g=0A> 2013-08-23 18:56:36,910 DEBUG=0A> org.apache.hadoop.hbase.master.Assi=
gnmentManager: Handling=0A> transition=3DRS_ZK_REGION_FAILED_OPEN, server=
=3Dvamshi_RS,60020,1377263792053,=0A> region=3D05e30711673614f6b41a364c76f3=
f05f=0A> 2013-08-23 18:56:36,911 DEBUG=0A> org.apache.hadoop.hbase.master.A=
ssignmentManager: Found an existing plan=0A> for=0A>=0A> scores,\x00\x00\x0=
0\x00\x00\x10\xC1\xF4,1377264146360.05e30711673614f6b41a364c76f3f05f.=0A> d=
estination server is vamshi,60020,1377263789273=0A> 2013-08-23 18:56:36,911=
 DEBUG=0A> org.apache.hadoop.hbase.master.AssignmentManager: No previous tr=
ansition=0A> plan was found (or we are ignoring an existing plan) for=0A>=
=0A> scores,\x00\x00\x00\x00\x00\x10\xC1\xF4,1377264146360.05e30711673614f6=
b41a364c76f3f05f.=0A> so generated a random one;=0A>=0A> hri=3Dscores,\x00\=
x00\x00\x00\x00\x10\xC1\xF4,1377264146360.05e30711673614f6b41a364c76f3f05f.=
,=0A> src=3D, dest=3Dvamshi,60020,1377263789273; 2 (online=3D2, available=
=3D1) available=0A> servers=0A> 2013-08-23 18:56:36,911 ERROR=0A> org.apach=
e.hadoop.hbase.executor.ExecutorService: Cannot submit=0A> [ClosedRegionHan=
dler-vamshi,60000,1377263788019-39] because the executor is=0A> missing. Is=
 this process shutting down?=0A> 2013-08-23 18:56:36,912 WARN=0A> org.apach=
e.hadoop.hbase.zookeeper.RecoverableZooKeeper: Possibly transient=0A> ZooKe=
eper exception:=0A> org.apache.zookeeper.KeeperException$ConnectionLossExce=
ption:=0A> KeeperErrorCode =3D ConnectionLoss for=0A> /hbase/unassigned/d47=
6f8442ce31de90b60080b74daf47f=0A> 2013-08-23 18:56:36,912 INFO org.apache.h=
adoop.hbase.util.RetryCounter:=0A> Sleeping 2000ms before retry #1...=0A> 2=
013-08-23 18:56:36,914 INFO org.apache.zookeeper.ZooKeeper: Session:=0A> 0x=
140ab519b0f0000 closed=0A> 2013-08-23 18:56:36,914 INFO org.apache.hadoop.h=
base.master.HMaster:=0A> HMaster main thread exiting=0A> 2013-08-23 18:56:3=
6,914 ERROR=0A> org.apache.hadoop.hbase.master.HMasterCommandLine: Failed t=
o start master=0A> java.lang.RuntimeException: HMaster Aborted=0A>=A0 =A0=
=A0=A0at=0A>=0A> org.apache.hadoop.hbase.master.HMasterCommandLine.startMas=
ter(HMasterCommandLine.java:160)=0A>=A0 =A0=A0=A0at=0A>=0A> org.apache.hado=
op.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:104)=0A>=A0 =
=A0=A0=A0at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)=0A>=
=A0 =A0=A0=A0at=0A>=0A> org.apache.hadoop.hbase.util.ServerCommandLine.doMa=
in(ServerCommandLine.java:76)=0A>=A0 =A0=A0=A0at org.apache.hadoop.hbase.ma=
ster.HMaster.main(HMaster.java:2100)=0A>=0A>=0A>=0A> My hbase-site.xml :=0A=
>=0A> <configuration>=0A>=A0 =A0=A0=A0<property>=0A>=A0 =A0 =A0 =A0=A0=A0<n=
ame>hbase.rootdir</name>=0A>=0A>=A0 =A0=A0=A0<value>/home/biginfolabs/BILSf=
twrs/hbase-0.94.10/hbstmp/</value>=0A>=A0 =A0=A0=A0</property>=0A>=0A>=A0 =
=A0=A0=A0<property>=0A>=A0 =A0 =A0 =A0=A0=A0<name>hbase.cluster.distributed=
</name>=0A>=A0 =A0 =A0 =A0=A0=A0<value>true</value>=0A>=A0 =A0=A0=A0</prope=
rty>=0A>=A0 =A0=A0=A0<property>=0A>=A0 =A0 =A0 =A0=A0=A0<name>hbase.master<=
/name>=0A>=A0 =A0 =A0 =A0=A0=A0<value>vamshi</value>=0A>=A0 =A0=A0=A0</prop=
erty>=0A>=A0 =A0=A0=A0<property>=0A>=A0 =A0 =A0 =A0=A0=A0<name>hbase.zookee=
per.property.clientPort</name>=0A>=A0 =A0 =A0 =A0=A0=A0<value>2181</value>=
=0A>=A0 =A0=A0=A0</property>=0A>=0A>=0A>=A0 =A0 <property>=0A>=A0 =A0 =A0 =
=A0=A0=A0<name>hbase.hregion.max.filesize</name>=0A>=A0 =A0 =A0 =A0=A0=A0<v=
alue>10485760</value>=0A>=A0 =A0=A0=A0</property>=0A>=0A>=0A>=0A>=A0 =A0=A0=
=A0<property>=0A>=A0 =A0 =A0 =A0=A0=A0<name>hbase.zookeeper.quorum</name>=
=0A>=A0 =A0 =A0 =A0=A0=A0<value>vamshi</value>=0A>=A0 =A0=A0=A0</property>=
=0A>=A0 =A0=A0=A0<property>=0A>=A0 =A0 =A0 =A0=A0=A0<name>hbase.zookeeper.p=
roperty.dataDir</name>=0A>=A0 =A0 =A0 =A0=A0=A0<value>/home/biginfolabs/BIL=
Sftwrs/hbase-0.94.10/zkptmp</value>=0A>=A0 =A0=A0=A0</property>=0A>=0A> <pr=
operty>=0A>=A0 =A0=A0=A0<name>hbase.zookeeper.property.maxClientCnxns</name=
>=0A>=A0 =A0=A0=A0<value>1024</value>=0A>=A0=A0=A0</property>=0A>=0A> <prop=
erty>=0A>=A0 =A0=A0=A0<name>hbase.coprocessor.user.region.classes</name>=0A=
>=A0 =A0=A0=A0<value>com.bil.coproc.ColumnAggregationEndpoint</value>=0A>=
=A0=A0=A0</property>=0A> </configuration>=0A>=0A>=0A>=0A>=0A> On Fri, Aug 2=
3, 2013 at 7:00 PM, Frank Chow <zhoushuaifeng@gmail.com>=0A> wrote:=0A>=0A>=
 > Hi,=0A> > You may should check if the compact is on. If data size in a r=
egion is=0A> max=0A> > than the limition, region will split and balance aft=
er a major=0A> > compaction(Usually occur automatically).=0A> > You can man=
ually by run the compact operaction by the shell commond:=0A> > compact <ta=
bleName>, or major_compact <tableName>=0A> >=0A> >=0A> >=0A> >=0A> > Frank =
Chow=0A>=0A>=0A>=0A>=0A> --=0A> *Regards*=0A> *=0A> Vamshi Krishna=0A> *=0A=
>=0A=0A=0A=0A-- =0A*Regards*=0A*=0AVamshi Krishna=0A*=A0