hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: Cannot open filename exception in 0.20.5
Date Fri, 25 Jun 2010 21:49:01 GMT
10.32.56.159 and sjc9-flash-grid05 are the same server:

2010-06-25 09:56:52,308 INFO org.apache.hadoop.hbase.master.ServerManager: 2
region servers, 0 dead, average load 22.52010-06-25 09:56:52,490 INFO
org.apache.hadoop.hbase.master.BaseScanner: RegionManager.rootScanner
scanning meta region {server: 10.32.56.159:60020, regionname: -ROOT-,,0,
startKey: <>}
2010-06-25 09:56:52,498 DEBUG org.apache.hadoop.hbase.master.BaseScanner:
Current assignment of .META.,,1 is not valid;  serverAddress=
10.32.56.160:60020, startCode=1277484838693 unknown.
2010-06-25 09:56:52,499 INFO org.apache.hadoop.hbase.master.BaseScanner:
RegionManager.rootScanner scan of 1 row(s) of meta region {server:
10.32.56.159:60020, regionname: -ROOT-,,0, startKey: <>} complete
2010-06-25 09:56:52,847 DEBUG org.apache.hadoop.hbase.master.RegionManager:
Assigning for address: 10.32.56.159:60020, startcode: 1277484792423, load:
(requests=5, regions=23, usedHeap=30, maxHeap=3991): total nregions to
assign=0, regions to give other servers than this=1, isMetaAssign=true
2010-06-25 09:56:52,847 DEBUG org.apache.hadoop.hbase.master.RegionManager:
Assigning address: 10.32.56.159:60020, startcode: 1277484792423, load:
(requests=5, regions=23, usedHeap=30, maxHeap=3991) 0 regions
2010-06-25 09:56:52,847 INFO org.apache.hadoop.hbase.master.RegionManager:
Assigning region .META.,,1 to sjc9-flash-grid05.carrieriq.com
,60020,1277484792423
2010-06-25 09:56:52,849 INFO org.apache.hadoop.hbase.master.ServerManager:
Processing MSG_REPORT_OPEN: .META.,,1 from
sjc9-flash-grid05.carrieriq.com,60020,1277484792423;
1 of 1
2010-06-25 09:56:52,850 DEBUG org.apache.hadoop.hbase.master.HMaster:
Processing todo: PendingOpenOperation from sjc9-flash-grid05.carrieriq.com
,60020,1277484792423
2010-06-25 09:56:52,850 INFO
org.apache.hadoop.hbase.master.RegionServerOperation: .META.,,1 open on
10.32.56.159:60020
2010-06-25 09:56:52,852 INFO
org.apache.hadoop.hbase.master.RegionServerOperation: Updated row .META.,,1
in region -ROOT-,,0 with startcode=1277484792423, server=10.32.56.159:60020
2010-06-25 09:56:52,852 DEBUG
org.apache.hadoop.hbase.master.RegionServerOperation: Adding to
onlineMetaRegions: {server: 10.32.56.159:60020, regionname: .META.,,1,
startKey: <>}
2010-06-25 09:56:52,853 INFO org.apache.hadoop.hbase.master.BaseScanner:
RegionManager.metaScanner scanning meta region {server: 10.32.56.159:60020,
regionname: .META.,,1, startKey: <>}

On Fri, Jun 25, 2010 at 2:33 PM, Jean-Daniel Cryans <jdcryans@apache.org>wrote:

> While taking a second look, I see that actually your region server is
> the one who is opening it over and over again:
>
> 2010-06-25 09:56:03,918 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Worker:
> MSG_REGION_OPEN: .META.,,1
> 2010-06-25 09:56:03,919 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion: Creating region
> .META.,,1, encoded=1028785192
> 2010-06-25 09:56:03,933 DEBUG
> org.apache.hadoop.hbase.regionserver.Store: loaded
> /hbase/.META./1028785192/info/7628076653206028272, isReference=false,
> sequence id=2991, length=51941, majorCompaction=false
> ...
> 2010-06-25 10:00:03,447 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: MSG_REGION_OPEN:
> .META.,,1
> 2010-06-25 10:00:03,448 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Worker:
> MSG_REGION_OPEN: .META.,,1
> ...
> 2010-06-25 10:02:03,130 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: MSG_REGION_OPEN:
> .META.,,1
> 2010-06-25 10:02:03,130 INFO
> org.apache.hadoop.hbase.regionserver.HRegionServer: Worker:
> MSG_REGION_OPEN: .META.,,1
>
> These messages come from the master, look for "Assigning region
> .META.,,1" not the encoded name.
>
> J-D
>
>
> On Fri, Jun 25, 2010 at 2:22 PM, Ted Yu <yuzhihong@gmail.com> wrote:
> > Hi,
> > 2010-06-25 09:56:03,936 INFO
> org.apache.hadoop.hbase.regionserver.HRegion:
> > region .META.,,1/1028785192 available; sequence id is 2992
> >
> > I searched for 1028785192 in master log but didn't find any occurrence.
> > The only log about compaction I see in master log is something like this:
> >
> > 2010-06-25 09:59:39,898 DEBUG
> org.apache.hadoop.hbase.regionserver.HRegion:
> > Closing TED-PACKAGESUMMARY-1277485167424-0,,1277485173774: disabling
> > compactions & flushes
> >
> > Any insight ?
> >
> > Thanks
> >
> > On Fri, Jun 25, 2010 at 11:31 AM, Jean-Daniel Cryans <
> jdcryans@apache.org>
> > wrote:
> >>
> >> At first glance it looks like a double assignment of .META., and the
> >> file it's trying to get was probably already compacted by another
> >> region server. Check the master log to see why/how it happened.
> >>
> >> J-D
> >>
> >> On Fri, Jun 25, 2010 at 11:13 AM, Ted Yu <yuzhihong@gmail.com> wrote:
> >> > Hi,
> >> > I upgraded a 3 node (1 master and 2 RS) cluster to 0.20.5 this
> morning.
> >> > hadoop version is Cloudera 0.20.2+228
> >> > I saw the following in
> >> > hbase-hadoop-regionserver-sjc9-flash-grid05.ciq.com.log
> >> > (/hbase/.META./1028785192/info/7628076653206028272 was not on hdfs):
> >> >
> >> > http://pastebin.com/0rH56Vzb
> >> >
> >> > This is from hadoop-hadoop-datanode-sjc9-flash-grid05.ciq.com.log:
> >> > http://pastebin.com/eYvq6FEV
> >> >
> >> > Has anyone seen similar issue ?
> >> >
> >> > Thanks
> >> >
> >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message