hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michael Dagaev <michael.dag...@gmail.com>
Subject Yet another Hbase failure
Date Thu, 19 Mar 2009 11:06:16 GMT
Hi, all

    We are running a small hbase 0.18 cluster. It stopped working now.
The master log contains "java.io.IOException: HStoreScanner failed
construction" errors.
Restart (stop/start-hbase) does not help. Hadoop fsck says that
everything is fine.

Has anybody run into such a problem ?

Thank you for your cooperation,
M.

2009-03-19 10:57:01,306 WARN org.apache.hadoop.hbase.master.BaseScanner:
Scan one META region: {regionname: .META.,,1, startKey: <>, server:
10.251.142.47:60020}
java.io.IOException: HStoreScanner failed construction
	at org.apache.hadoop.hbase.regionserver.StoreFileScanner.<init>(StoreFileScanner.java:70)
	at org.apache.hadoop.hbase.regionserver.HStoreScanner.<init>(HStoreScanner.java:70)
	at org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:1916)
	at org.apache.hadoop.hbase.regionserver.HRegion$HScanner.<init>(HRegion.java:1954)
	at org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1345)
	at org.apache.hadoop.hbase.regionserver.HRegionServer.openScanner(HRegionServer.java:1175)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:554)
	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:888)
Caused by: java.io.IOException: Could not obtain block:
blk_-2893342489105927361_2846854
file=/hbase/.META./1028785192/info/mapfiles/3882153324238090640/index
	at org.apache.hadoop.dfs.DFSClient$DFSInputStream.chooseDataNode(DFSClient.java:1462)
	at org.apache.hadoop.dfs.DFSClient$DFSInputStream.blockSeekTo(DFSClient.java:1312)
	at org.apache.hadoop.dfs.DFSClient$DFSInputStream.read(DFSClient.java:1417)
	at java.io.DataInputStream.readFully(DataInputStream.java:178)
	at java.io.DataInputStream.readFully(DataInputStream.java:152)
	at org.apache.hadoop.io.SequenceFile$Reader.init(SequenceFile.java:1453)
	at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1431)
	at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1420)
	at org.apache.hadoop.io.SequenceFile$Reader.<init>(SequenceFile.java:1415)
	at org.apache.hadoop.io.MapFile$Reader.open(MapFile.java:292)
	at org.apache.hadoop.hbase.regionserver.HStoreFile$HbaseMapFile$HbaseReader.<init>(HStoreFile.java:632)
	at org.apache.hadoop.hbase.regionserver.HStoreFile$BloomFilterMapFile$Reader.<init>(HStoreFile.java:714)
	at org.apache.hadoop.hbase.regionserver.HStoreFile.getReader(HStoreFile.java:413)
	at org.apache.hadoop.hbase.regionserver.StoreFileScanner.openReaders(StoreFileScanner.java:96)
	at org.apache.hadoop.hbase.regionserver.StoreFileScanner.<init>(StoreFileScanner.java:67)
	... 11 more

	at sun.reflect.GeneratedConstructorAccessor11.newInstance(Unknown Source)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
	at org.apache.hadoop.hbase.RemoteExceptionHandler.decodeRemoteException(RemoteExceptionHandler.java:82)
	at org.apache.hadoop.hbase.master.BaseScanner.scanRegion(BaseScanner.java:191)
	at org.apache.hadoop.hbase.master.MetaScanner.scanOneMetaRegion(MetaScanner.java:74)
	at org.apache.hadoop.hbase.master.MetaScanner.maintenanceScan(MetaScanner.java:129)
	at org.apache.hadoop.hbase.master.BaseScanner.chore(BaseScanner.java:139)
	at org.apache.hadoop.hbase.Chore.run(Chore.java:62)

Mime
View raw message