hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matteo Bertozzi <theo.berto...@gmail.com>
Subject Re: Corrupt HBase table
Date Tue, 18 Mar 2014 17:33:41 GMT
can you try to read the file by using:
org.apache.hadoop.hbase.io.hfile.HFile -f FILE_PATH -s -e

to verify if the problem is reading the block from cache or the data on
disk is corrupted in some way?
The stacktrace is showing a file size > 255GB and a Key of 16M which I
doubt being correct.



Matteo



On Tue, Mar 18, 2014 at 5:26 PM, Ted Yu <yuzhihong@gmail.com> wrote:

> The stack trace doesn't reveal too much other than the error occurred at
> the beginning of a scan.
>
> Other experts would have more insights than I do.
>
> Cheers
>
>
> On Tue, Mar 18, 2014 at 10:10 AM, Sudarshan Kadambi (BLOOMBERG/ 731 LEXIN)
> <
> skadambi@bloomberg.net> wrote:
>
> > Thanks Ted. No, we don't use data block encoding. This is 0.96. It
> appears
> > multiple HFiles are affected. Does the stack trace give some context on
> > what is that it's attempting to do at the lowest level?
> >
> >
> > ----- Original Message -----
> > From: yuzhihong@gmail.com
> > To: Sudarshan Kadambi (BLOOMBERG/ 731 LEXIN) <skadambi@bloomberg.net>,
> > user@hbase.apache.org
> > At: Mar 18 2014 13:03:20
> >
> > Are you using Data Block Encoding for this table ?
> >
> > Can you sideline the HFile and see if other HFiles were affected ?
> >
> > I assume you use 0.94.x
> >
> > Cheers
> >
> >
> > On Tue, Mar 18, 2014 at 7:14 AM, Sudarshan Kadambi (BLOOMBERG/ 731 LEXIN)
> > <skadambi@bloomberg.net> wrote:
> >
> >> Hi Folks:
> >>
> >> We've a table that we're unable to make online. A scan on this table
> >> yields the following stack trace. hbck doesn't report any
> inconsistencies.
> >> I went ahead and did a repair, which didn't make a difference. Any tips?
> >>
> >> ERROR: java.io.IOException: Invalid currKeyLen 16777280 or currValueLen
> >> 64. Block offset: 274609471488, block length: 16415, position: 0
> (without
> >> header).
> >>         at
> org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2213)
> >>         at
> >> org.apache.hadoop.hbase.ipc.RpcServer$Handler.run(RpcServer.java:1879)
> >> Caused by: java.lang.IllegalStateException: Invalid currKeyLen 16777280
> >> or currValueLen 64. Block offset: 274609471488, block length: 16415,
> >> position: 0 (without header).
> >>         at
> >>
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.readKeyValueLen(HFileReaderV2.java:820)
> >>         at
> >>
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.updateCurrBlock(HFileReaderV2.java:787)
> >>         at
> >>
> org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.seekTo(HFileReaderV2.java:749)
> >>         at
> >>
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.seekAtOrAfter(StoreFileScanner.java:232)
> >>         at
> >>
> org.apache.hadoop.hbase.regionserver.StoreFileScanner.seek(StoreFileScanner.java:145)
> >>         at
> >>
> org.apache.hadoop.hbase.regionserver.StoreScanner.<init>(StoreScanner.java:168)
> >>         at
> >> org.apache.hadoop.hbase.regionserver.HStore.getScanner(HStore.java:1642)
> >>         at
> >>
> org.apache.hadoop.hbase.regionserver.HRegion$RegionScannerImpl.<init>(HRegion.java:3460)
> >>         at
> >>
> org.apache.hadoop.hbase.regionserver.HRegion.instantiateRegionScanner(HRegion.java:1768)
> >>         at
> >>
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1760)
> >>         at
> >>
> org.apache.hadoop.hbase.regionserver.HRegion.getScanner(HRegion.java:1737)
> >>         at
> >>
> org.apache.hadoop.hbase.regionserver.HRegionServer.scan(HRegionServer.java:2996)
> >>         at
> >>
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:26929)
> >>         at
> org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2175)
> >>         ... 1 more
> >>
> >>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message