hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sebastian Bauer <ad...@ugame.net.pl>
Subject Re: HBase exception after upgrading from hbase-0.90-branch to trunk
Date Thu, 18 Aug 2011 15:27:34 GMT
Hi, comments inline:

On 16.08.2011 06:42, Stack wrote:
> Oh, are you running on hbase trunk?  If so, did you write all your
> data with TRUNK or did you start up 0.92 on a data that was written w/
> 0.90 (It should work but you may have hit an issue).
I had branch-0.90 and after proper shutdown i had updated hbase to 
trunk, it looks like -ROOT- table was migrated but .META. had problem 
with saving information about migration in logs i had informations:

2011-08-18 17:16:09,334 INFO org.apache.hadoop.hbase.master.HMaster: 
.META. assigned=2, rit=false, location=lala.golden,60020,1313680556932
2011-08-18 17:16:09,340 INFO org.apache.hadoop.hbase.master.HMaster: 
metaUpdated = NULL.
2011-08-18 17:16:09,340 INFO org.apache.hadoop.hbase.master.HMaster: 
Meta updated status = false
2011-08-18 17:16:09,340 INFO org.apache.hadoop.hbase.master.HMaster: 
Meta has HRI with HTDs. Updating meta now.
2011-08-18 17:16:17,535 INFO org.apache.hadoop.hbase.catalog.MetaEditor: 
Updated -ROOT- row with metaMigrated status = true
2011-08-18 17:16:17,536 INFO org.apache.hadoop.hbase.master.HMaster: 
Meta updated with new HRI.

and even after restarting i saw this in logs.

./hbase shell show all one regions, but any operation on tables show 
information that .META. table doesn't exist.
Full logs from first and second run after upgrade are here:  
http://s2.e-back.net/hbase_logs.tar.gz

-ROOT- have this rows:
hbase(main):005:0> scan '-ROOT-'
ROW                                                                     
COLUMN+CELL
  -ROOT-,,0                                                              
column=info:metamigrated, timestamp=1313680577531, value=\xFF
  .META.,,1                                                              
column=info:regioninfo, timestamp=1278560826938, value=REGION => {NAME 
=> '.META.,,1 TableName => ', STARTKEY => '', ENDKEY => '', ENCODED => 
1028785192,}
  .META.,,1                                                              
column=info:server, timestamp=1313680569288, value=lala.golden:60020
  .META.,,1                                                              
column=info:serverstartcode, timestamp=1313680569288, value=1313680556932
2 row(s) in 0.0500 seconds


On 16.08.2011 06:40, Stack wrote:
> What version of hbase is this Sebastian?
>
> What you mean by remove -ROOT- and .META. (that doesn't sound good but
> you probably mean something else though looking at the exception
> below, maybe this is what you did).
Long time ago when i smashed my hbase instalation when i removed .META. 
and -ROOT- (./bin/hadoop fs -rmr /hbase/.META.) and run add_table.rb 
script all tables were online again.
>
> St.Ack
>
> On Thu, Aug 11, 2011 at 11:54 AM, Sebastian Bauer<admin@ugame.net.pl>  wrote:
>> 2011-08-11 20:53:55,889 WARN
>> org.apache.hadoop.hbase.master.AssignmentManager: Failed assignment of
>> Golden_ATU,m_ef06511a814155022686d50cebb0dba5_2010_9_40CC7C8170915ABC43432E524F0E6D2B,1301066006080.f656ca602b8638ad8329c36af85ae43e.
>> to db2a.goldenline.pl,60020,1313088827048, trying to assign elsewhere
>> instead; retry=
>> org.apache.hadoop.ipc.RemoteException: java.io.IOException:
>> java.lang.IllegalArgumentException: Can not create a Path from an empty
>> string
>>         at org.apache.hadoop.fs.Path.checkPathArg(Path.java:82)
>>         at org.apache.hadoop.fs.Path.<init>(Path.java:90)
>>         at org.apache.hadoop.fs.Path.<init>(Path.java:50)
>>         at
>> org.apache.hadoop.hbase.util.FSUtils.getTablePath(FSUtils.java:845)
>>         at
>> org.apache.hadoop.hbase.util.FSUtils.getTableInfoPath(FSUtils.java:818)
>>         at
>> org.apache.hadoop.hbase.util.FSUtils.getTableInfoModtime(FSUtils.java:832)
>>         at
>> org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:127)
>>         at
>> org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:99)
>>         at
>> org.apache.hadoop.hbase.regionserver.HRegionServer.openRegion(HRegionServer.java:2329)
>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>         at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>         at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>         at java.lang.reflect.Method.invoke(Method.java:597)
>>         at
>> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:312)
>>         at
>> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1222)
>>
>>         at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:871)
>>         at
>> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:141)
>>         at $Proxy7.openRegion(Unknown Source)
>>         at
>> org.apache.hadoop.hbase.master.ServerManager.sendRegionOpen(ServerManager.java:404)
>>         at
>> org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1239)
>>         at
>> org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1034)
>>         at
>> org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1014)
>>         at
>> org.apache.hadoop.hbase.master.handler.ClosedRegionHandler.process(ClosedRegionHandler.java:101)
>>         at
>> org.apache.hadoop.hbase.master.AssignmentManager.addToRITandCallClose(AssignmentManager.java:458)
>>         at
>> org.apache.hadoop.hbase.master.AssignmentManager.processRegionsInTransition(AssignmentManager.java:389)
>>         at
>> org.apache.hadoop.hbase.master.AssignmentManager.processRegionInTransition(AssignmentManager.java:355)
>>         at
>> org.apache.hadoop.hbase.master.AssignmentManager.processRegionsInTransition(AssignmentManager.java:296)
>>         at
>> org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:243)
>>         at
>> org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:467)
>>         at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:300)
>>
>> 2011-08-11 20:53:55,895 WARN
>> org.apache.hadoop.hbase.master.AssignmentManager: Failed assignment of
>> Golden_ATU,m_ef06511a814155022686d50cebb0dba5_2010_9_40CC7C8170915ABC43432E524F0E6D2B,1301066006080.f656ca602b8638ad8329c36af85ae43e.
>> to db2b.goldenline.pl,60020,1313088826553, trying to assign elsewhere
>> instead; retry=
>> org.apache.hadoop.ipc.RemoteException: java.io.IOException:
>> java.lang.IllegalArgumentException: Can not create a Path from an empty
>> string
>>         at org.apache.hadoop.fs.Path.checkPathArg(Path.java:82)
>>         at org.apache.hadoop.fs.Path.<init>(Path.java:90)
>>         at org.apache.hadoop.fs.Path.<init>(Path.java:50)
>>         at
>> org.apache.hadoop.hbase.util.FSUtils.getTablePath(FSUtils.java:845)
>>         at
>> org.apache.hadoop.hbase.util.FSUtils.getTableInfoPath(FSUtils.java:818)
>>         at
>> org.apache.hadoop.hbase.util.FSUtils.getTableInfoModtime(FSUtils.java:832)
>>         at
>> org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:127)
>>         at
>> org.apache.hadoop.hbase.util.FSTableDescriptors.get(FSTableDescriptors.java:99)
>>         at
>> org.apache.hadoop.hbase.regionserver.HRegionServer.openRegion(HRegionServer.java:2329)
>>         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>>         at
>> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>         at
>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>>         at java.lang.reflect.Method.invoke(Method.java:597)
>>         at
>> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:312)
>>         at
>> org.apache.hadoop.hbase.ipc.HBaseServer$Handler.run(HBaseServer.java:1222)
>>
>>         at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:871)
>>         at
>> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:141)
>>         at $Proxy7.openRegion(Unknown Source)
>>         at
>> org.apache.hadoop.hbase.master.ServerManager.sendRegionOpen(ServerManager.java:404)
>>         at
>> org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1239)
>>         at
>> org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1034)
>>         at
>> org.apache.hadoop.hbase.master.AssignmentManager.assign(AssignmentManager.java:1014)
>>         at
>> org.apache.hadoop.hbase.master.handler.ClosedRegionHandler.process(ClosedRegionHandler.java:101)
>>         at
>> org.apache.hadoop.hbase.master.AssignmentManager.addToRITandCallClose(AssignmentManager.java:458)
>>         at
>> org.apache.hadoop.hbase.master.AssignmentManager.processRegionsInTransition(AssignmentManager.java:389)
>>         at
>> org.apache.hadoop.hbase.master.AssignmentManager.processRegionInTransition(AssignmentManager.java:355)
>>         at
>> org.apache.hadoop.hbase.master.AssignmentManager.processRegionsInTransition(AssignmentManager.java:296)
>>         at
>> org.apache.hadoop.hbase.master.AssignmentManager.joinCluster(AssignmentManager.java:243)
>>         at
>> org.apache.hadoop.hbase.master.HMaster.finishInitialization(HMaster.java:467)
>>         at org.apache.hadoop.hbase.master.HMaster.run(HMaster.java:300)
>>
>> --
>>
>> Pozdrawiam
>> Sebastian Bauer
>> -----------------------------------------------------
>> http://tikecik.pl
>>
>>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message