hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dru Jensen <drujen...@gmail.com>
Subject Re: corrupt hbase meta
Date Fri, 09 Jan 2009 17:15:22 GMT
Andy and St.Ack,

Thanks for you help.  The directory didn't exist and the MR process  
failed in the same spot after restarting the cluster.  I decided to  
shutdown/restart and try again.  This time it fixed itself so I guess  
I was lucky.

Dru

On Jan 8, 2009, at 10:11 PM, stack wrote:

> Andrew Purtell wrote:
>> Hi Dru,
>>
>> You might try
>>  % hadoop fs -rmr hdfs://dev-hadoop0.brandaffinity.net:9000/hbase/ 
>> talentmaps/2022550034/base/mapfiles/7007481172739780749/
>>
>
> Dru, per Andrew's note above, does the parent directory exist?  If  
> so, and you list it, what do you see?  No data but is there an index?
>
> St.Ack
>
>> to at least partially recover the region. Don't try this until
>> someone else weighs in on this approach.
>>
>> Since you've already suffered data loss, possibly as a result of
>> HBASE-1104, I do not think there is any harm to try it.
>>   - Andy
>>
>>
>>> From: Dru Jensen <drujensen@gmail.com>
>>> Subject: corrupt hbase meta
>>> To: hbase-user@hadoop.apache.org
>>> Date: Thursday, January 8, 2009, 4:48 PM
>>> I have a MR process that is failing at the same point
>>> everytime I try to run it.  I have restarted hadoop and
>>> hbase but that didn't fix it (unlike last time).
>>>
>>> I checked dfs and the file that hbase is looking for does
>>> not exist.  Is there a tool I can run to fix this?
>>>
>>> Thanks,
>>> Dru
>>>
>>> org.apache.hadoop.hbase.client.RetriesExhaustedException:
>>> Trying to contact region server 10.10.10.52:60020 for region
>>> talentmaps,http://reporternews.com/news/2008/feb/13/sampson-in-trouble-over-major-violations/,1226767718516

>>> ,
>>> row
>>> 'http://reporternews.com/news/2008/feb/13/sampson-in-trouble-over-major-violations/'

>>> ,
>>> but failed after 10 attempts.
>>> Exceptions:
>>> java.io.IOException: java.io.IOException: HStoreScanner
>>> failed construction
>>> 	at
>>> org.apache.hadoop.hbase.regionserver.StoreFileScanner. 
>>> (StoreFileScanner.java:70)
>>> 	at
>>> org.apache.hadoop.hbase.regionserver.HStoreScanner. 
>>> (HStoreScanner.java:88)
>>> 	at
>>> org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java: 
>>> 2123)
>>> 	at
>>> org.apache.hadoop.hbase.regionserver.HRegion$HScanner. 
>>> (HRegion.java:1984)
>>> 	at
>>> org 
>>> .apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java: 
>>> 1178)
>>> 	at
>>> org 
>>> .apache 
>>> .hadoop 
>>> .hbase.regionserver.HRegionServer.openScanner(HRegionServer.java: 
>>> 1697)
>>> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native
>>> Method)
>>> 	at
>>> sun 
>>> .reflect 
>>> .NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
>>> 	at
>>> sun 
>>> .reflect 
>>> .DelegatingMethodAccessorImpl 
>>> .invoke(DelegatingMethodAccessorImpl.java:25)
>>> 	at java.lang.reflect.Method.invoke(Method.java:597)
>>> 	at
>>> org.apache.hadoop.hbase.ipc.HBaseRPC$Server.call(HBaseRPC.java:632)
>>> 	at
>>> org.apache.hadoop.hbase.ipc.HBaseServer 
>>> $Handler.run(HBaseServer.java:894)
>>> Caused by: java.io.FileNotFoundException: File does not
>>> exist:
>>> hdfs://dev-hadoop0.brandaffinity.net:9000/hbase/talentmaps/ 
>>> 2022550034/base/mapfiles/7007481172739780749/data
>>> 	at
>>> org 
>>> .apache 
>>> .hadoop 
>>> .hdfs 
>>> .DistributedFileSystem.getFileStatus(DistributedFileSystem.java:394)
>>> 	at
>>> org.apache.hadoop.fs.FileSystem.getLength(FileSystem.java:679)
>>> 	at
>>> org.apache.hadoop.hbase.io.SequenceFile$Reader.(SequenceFile.java: 
>>> 1431)
>>> 	at
>>> org.apache.hadoop.hbase.io.SequenceFile$Reader.(SequenceFile.java: 
>>> 1426)
>>> 	at
>>> org.apache.hadoop.hbase.io.MapFile 
>>> $Reader.createDataFileReader(MapFile.java:310)
>>> 	at
>>> org.apache.hadoop.hbase.io.HBaseMapFile 
>>> $HBaseReader.createDataFileReader(HBaseMapFile.java:96)
>>> 	at
>>> org.apache.hadoop.hbase.io.MapFile$Reader.open(MapFile.java:292)
>>> 	at
>>> org.apache.hadoop.hbase.io.HBaseMapFile$HBaseReader. 
>>> (HBaseMapFile.java:79)
>>> 	at
>>> org.apache.hadoop.hbase.io.BloomFilterMapFile$Reader. 
>>> (BloomFilterMapFile.java:65)
>>> 	at
>>> org 
>>> .apache 
>>> .hadoop.hbase.regionserver.HStoreFile.getReader(HStoreFile.java:443)
>>> 	at
>>> org 
>>> .apache 
>>> .hadoop 
>>> .hbase 
>>> .regionserver.StoreFileScanner.openReaders(StoreFileScanner.java:96)
>>> 	at
>>> org.apache.hadoop.hbase.regionserver.StoreFileScanner. 
>>> (StoreFileScanner.java:67)
>>> 	... 11 more
>>>
>>
>>
>>
>


Mime
View raw message