hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ch huang <justlo...@gmail.com>
Subject Re: unbalance number fo region in cluster
Date Fri, 05 Jul 2013 02:16:31 GMT
i see lot's of skip balance info,is balancer refuse to work?why?

2013-07-05 09:56:25,121 DEBUG org.apache.hadoop.hbase.regionserver.HRegion:
DELETING region
hdfs://CH22:9000/hbaseroot/demoK/febcd37c57994604c8e218d8eb9c75c2
2013-07-05 09:56:25,130 INFO org.apache.hadoop.hbase.catalog.MetaEditor:
Deleted region
demoK,rows720752,1372989269181.febcd37c57994604c8e218d8eb9c75c2. from META
2013-07-05 09:56:25,130 INFO org.apache.hadoop.hbase.master.CatalogJanitor:
Scanned 151 catalog row(s) and gc'd 8 unreferenced parent region(s)
2013-07-05 10:01:24,894 DEBUG org.apache.hadoop.hbase.master.LoadBalancer:
Server information: CH34,60020,1372899976453=72, CH35,60020,1372899981567=73
2013-07-05 10:01:24,894 INFO org.apache.hadoop.hbase.master.LoadBalancer:
Skipping load balancing.  servers=2 regions=145 average=72.5 mostloaded=73
leastloaded=72
2013-07-05 10:01:24,962 DEBUG
org.apache.hadoop.hbase.master.CatalogJanitor: Scanned 143 catalog row(s)
and gc'd 0 unreferenced parent region(s)
2013-07-05 10:06:24,895 DEBUG org.apache.hadoop.hbase.master.LoadBalancer:
Server information: CH34,60020,1372899976453=72, CH35,60020,1372899981567=73
2013-07-05 10:06:24,895 INFO org.apache.hadoop.hbase.master.LoadBalancer:
Skipping load balancing.  servers=2 regions=145 average=72.5 mostloaded=73
leastloaded=72
2013-07-05 10:06:24,964 DEBUG
org.apache.hadoop.hbase.master.CatalogJanitor: Scanned 143 catalog row(s)
and gc'd 0 unreferenced parent region(s)
2013-07-05 10:10:49,441 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
Scanning .META. starting at row=jerry_cm,,00000000000000 for max=2147483647
rows
2013-07-05 10:10:49,453 DEBUG
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation:
The connection to hconnection-0x3fa2814501005d has been closed.
2013-07-05 10:11:07,211 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
Scanning .META. starting at row=dmp_pageview,,00000000000000 for max=10 rows
2013-07-05 10:11:07,213 DEBUG
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation:
Cached location for
dmp_pageview,,1372758866561.364a36024f18bc6047a0bf7158b4c744. is CH34:60020
2013-07-05 10:11:07,214 DEBUG
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation:
The connection to hconnection-0x3fa2814501005d has been closed.
2013-07-05 10:11:07,215 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
Scanning .META. starting at row=dmp_pageview,,00000000000000 for
max=2147483647 rows
2013-07-05 10:11:07,219 DEBUG
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation:
The connection to hconnection-0x3fa2814501005d has been closed.
2013-07-05 10:11:13,176 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
Scanning .META. starting at row=usertable,,00000000000000 for max=10 rows
2013-07-05 10:11:13,179 DEBUG
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation:
Cached location for
usertable,,1372923221432.e2c013858f08c420c9a6c86b99068f75. is CH35:60020
2013-07-05 10:11:13,179 DEBUG
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation:
Cached location for
usertable,user2035056063890669682,1372923221432.833e4e71534caac4e34371819e42ca20.
is CH34:60020
2013-07-05 10:11:13,179 DEBUG
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation:
Cached location for
usertable,user3071366660001926980,1372923194708.da128be082d51ec5ab1233df63aabb46.
is CH34:60020
2013-07-05 10:11:13,180 DEBUG
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation:
Cached location for
usertable,user4104227499385169775,1372923194708.97d6138333c72839ce22adae4940e3d5.
is CH34:60020
2013-07-05 10:11:13,180 DEBUG
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation:
Cached location for
usertable,user5139370204013525946,1372923209834.e788429ec0b1e032f4b309337c590732.
is CH34:60020
2013-07-05 10:11:13,180 DEBUG
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation:
Cached location for
usertable,user6178888326785369277,1372923209834.ed841132313f30d51c82ad9f8e3e71c5.
is CH34:60020
2013-07-05 10:11:13,180 DEBUG
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation:
Cached location for
usertable,user7219194014980283263,1372923217190.7a4f56a2e754a707b80dc4d4689f54e7.
is CH34:60020
2013-07-05 10:11:13,181 DEBUG
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation:
Cached location for
usertable,user825931901641799999,1372923217190.b6275359a308360d59931b974d096c45.
is CH34:60020
2013-07-05 10:11:13,182 DEBUG
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation:
The connection to hconnection-0x3fa2814501005d has been closed.
2013-07-05 10:11:13,183 DEBUG org.apache.hadoop.hbase.client.MetaScanner:
Scanning .META. starting at row=usertable,,00000000000000 for
max=2147483647 rows
2013-07-05 10:11:13,188 DEBUG
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation:
The connection to hconnection-0x3fa2814501005d has been closed.
2013-07-05 10:11:24,894 DEBUG org.apache.hadoop.hbase.master.LoadBalancer:
Server information: CH34,60020,1372899976453=72, CH35,60020,1372899981567=73
2013-07-05 10:11:24,894 INFO org.apache.hadoop.hbase.master.LoadBalancer:
Skipping load balancing.  servers=2 regions=145 average=72.5 mostloaded=73
leastloaded=72
2013-07-05 10:11:24,962 DEBUG
org.apache.hadoop.hbase.master.CatalogJanitor: Scanned 143 catalog row(s)
and gc'd 0 unreferenced parent region(s)


On Thu, Jul 4, 2013 at 5:20 PM, Ted Yu <yuzhihong@gmail.com> wrote:

> Can you check load balancer related log lines in master log and put them
> on pastebin so that root cause can be diagnosed ?
>
> Which HBase version are you using ?
>
> Thanks
>
> On Jul 3, 2013, at 11:44 PM, ch huang <justlooks@gmail.com> wrote:
>
> > the load balancer process as i see is very slow i have a big table  ,it's
> > region distrabute is  node1: 5 ,node2: 22 ,node3: 5 ,after 4 hours ,it's
> > became
> > node1: 6  ,node2: 20 ,node1: 6, change is very very slow!
> >
> > On Thu, Jul 4, 2013 at 1:54 PM, Azuryy Yu <azuryyyu@gmail.com> wrote:
> >
> >> I supposed load balancer is enabled and there is no pre-split.
> >>
> >> LB run periodicly, (5 mininutes by default), so after insert bulk data
> into
> >> HBase, the table is unbalance but there will be balanced eventually
> after
> >> round-robin load balancer running.
> >>
> >>
> >> On Thu, Jul 4, 2013 at 1:25 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> >>
> >>> Did you presplit your table ?
> >>>
> >>> Was load balancer enabled ?
> >>>
> >>> What HBase version do you use ?
> >>>
> >>> Thanks
> >>>
> >>> On Jul 3, 2013, at 10:21 PM, ch huang <justlooks@gmail.com> wrote:
> >>>
> >>>> i insert 2GB data into my hbase cluster ,but i find the region is not
> >>>> balance,is insert operation always cause the region number different
> on
> >>>> region server?
> >>>>
> >>>> here is my 60010 output
> >>>>
> >>>> Table Regions  Name Region Server Start Key End Key
> >>>> demo2,,1372914631698.f42281ec44d113df3e644d5ecdb35366.
> >>>> CH36:60030<http://ch36:60030/>
> >>>> rows23703
> >> demo2,rows23703,1372914684582.c3373791daec95b849ba2908c517e045.
> >>>> CH35:60030 <http://ch35:60030/> rows23703 rows347161
> >>>> demo2,rows347161,1372914714248.dd9a6c39f65ad4a879638e1bfec76016.
> >>>> CH34:60030<http://ch34:60030/>
> >>>> rows347161 rows428378
> >>>> demo2,rows428378,1372914745946.41a4669f36bc99ef1592b2df13c0a948.
> >>>> CH36:60030<http://ch36:60030/>
> >>>> rows428378 rows512589
> >>>> demo2,rows512589,1372914776151.a4363bdd7403633c5775931ab32a7da8.
> >>>> CH36:60030<http://ch36:60030/>
> >>>> rows512589 rows598644
> >>>> demo2,rows598644,1372914806469.729f52018aac909e8080f00b8d698e2d.
> >>>> CH36:60030<http://ch36:60030/>
> >>>> rows598644 rows684871
> >>>> demo2,rows684871,1372914836812.d9cb1760fe3302500034913ff813e0f8.
> >>>> CH35:60030<http://ch35:60030/>
> >>>> rows684871 rows771559
> >>>> demo2,rows771559,1372914836812.ac16ff1404cff5c0db3d2e3b0661ffcd.
> >>>> CH36:60030<http://ch36:60030/>
> >>>> rows771559 Regions by Region Server  Region Server Region Count
> >>>> http://CH35:60030/ <http://ch35:60030/> <http://ch35:60030/>
2
> http://CH36:60030/ <http://ch36:60030/><http://ch36:60030/>5
> >> http://CH34:60030/ <http://ch34:60030/> <http://ch34:60030/> 1
> >>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message