hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Nichole Treadway <kntread...@gmail.com>
Subject Re: HRegionInfo was null or empty in -ROOT-
Date Wed, 30 Mar 2011 17:36:09 GMT
In the master logs, I see messages like I've pasted below. I get the same
behavior even when I stop master and restart it. I see the root and meta
table regions listed in the region server ui's

2011-03-30 13:15:00,675 DEBUG
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation:
locateRegionInMeta parentTable=-ROOT-, metaLocation=address:
node1.abc123.com:60020, regioninfo: -ROOT-,,0.70236052, attempt=7 of 10
failed; retrying after sleep of 8000 because: HRegionInfo was null or empty
in -ROOT-, row=keyvalues={.META.,,1/info:server/1301428760725/Put/vlen=33,
.META.,,1/info:serverstartcode/1301428760725/Put/vlen=8}
2011-03-30 13:15:00,675 DEBUG
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation:
Lookedup root region location,
connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@475a07bf;
hsa=node1.abc123.com:60020
2011-03-30 13:15:08,679 DEBUG
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation:
Lookedup root region location,
connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@475a07bf;
hsa=node1.abc123.com:60020
2011-03-30 13:15:08,681 DEBUG
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation:
locateRegionInMeta parentTable=-ROOT-, metaLocation=address:
node1.abc123.com:60020, regioninfo: -ROOT-,,0.70236052, attempt=8 of 10
failed; retrying after sleep of 16000 because: HRegionInfo was null or empty
in -ROOT-, row=keyvalues={.META.,,1/info:server/1301428760725/Put/vlen=33,
.META.,,1/info:serverstartcode/1301428760725/Put/vlen=8}
2011-03-30 13:15:08,681 DEBUG
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation:
Lookedup root region location,
connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@475a07bf;
hsa=node1.abc123.com:60020
2011-03-30 13:15:24,685 DEBUG
org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation:
Lookedup root region location,
connection=org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation@475a07bf;
hsa=node1.abc123.com:60020
2011-03-30 13:15:24,688 WARN org.mortbay.log: /master.jsp:
org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to contact
region server null for region , row '', but failed after 10 attempts.
Exceptions:
java.io.IOException: HRegionInfo was null or empty in -ROOT-,
row=keyvalues={.META.,,1/info:server/1301428760725/Put/vlen=33,
.META.,,1/info:serverstartcode/1301428760725/Put/vlen=8}
...
2011-03-30 13:19:32,344 DEBUG org.apache.hadoop.hbase.master.CatalogJanitor:
Scanned 0 catalog row(s) and gc'd 0 unreferenced parent region(s)
2011-03-30 13:19:32,346 INFO org.apache.hadoop.hbase.master.LoadBalancer:
Skipping load balancing.  servers=4 regions=2 average=0.5 mostloaded=1
leastloaded=1





On Wed, Mar 30, 2011 at 12:19 PM, Stack <stack@duboce.net> wrote:

> Can you check master log and see whats happening around -ROOT-? Why is
> it not being assigned out?  What if you kill master process and
> restart it?  Does -ROOT- get assigned then?
>
> St.Ack
>
> On Wed, Mar 30, 2011 at 7:21 AM, Nichole Treadway <kntreadway@gmail.com>
> wrote:
> > Hi all,
> >
> > This morning I was restarting my HBase cluster, and I'm finding that the
> > region servers start up fine, but master always fails with the following
> > error shown in the web ui:
> >
> > Problem accessing /master.jsp. Reason:
> >
> >    Trying to contact region server null for region , row '', but
> > failed after 10 attempts.
> > Exceptions:
> > java.io.IOException: HRegionInfo was null or empty in -ROOT-,
> > row=keyvalues={.META.,,1/info:server/1301428760725/Put/vlen=33,
> > .META.,,1/info:serverstartcode/1301428760725/Put/vlen=8}
> > java.io.IOException: HRegionInfo was null or empty in -ROOT-,
> > row=keyvalues={.META.,,1/info:server/1301428760725/Put/vlen=33,
> > .META.,,1/info:serverstartcode/1301428760725/Put/vlen=8}
> > ...
> >
> > Caused by:
> >
> >
> > org.apache.hadoop.hbase.client.RetriesExhaustedException: Trying to
> > contact region server null for region , row '', but failed after 10
> > attempts.
> > Exceptions:
> > java.io.IOException: HRegionInfo was null or empty in -ROOT-,
> > row=keyvalues={.META.,,1/info:server/1301428760725/Put/vlen=33,
> > .META.,,1/info:serverstartcode/1301428760725/Put/vlen=8}
> > java.io.IOException: HRegionInfo was null or empty in -ROOT-,
> > row=keyvalues={.META.,,1/info:server/1301428760725/Put/vlen=33,
> > .META.,,1/info:serverstartcode/1301428760725/Put/vlen=8}
> >
> >
> > ...
> >
> > Here is what I get when I scan -ROOT-:
> >
> >
> > hbase(main):001:0> scan '-ROOT-'
> > ROW                                     COLUMN+CELL
> >  .META.,,1                              column=info:server,
> > timestamp=1301428760725, value=sspv2rndhadoop1.gce2000.com:60020
> >  .META.,,1                              column=info:serverstartcode,
> > timestamp=1301428760725, value=1301428748891
> >
> >
> >
> > Any ideas what's going on / how I can fix this?
> >
> >
> > Cluster info: 4 nodes, hadoop-0.20-append, hbase-0.90.3, red hat 5.5,
> > 64-bit, jdk1.6; 64 gb ram, 2.8ghz.
> >
> >
> > Thanks guys
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message