hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From stack <st...@duboce.net>
Subject Re: Probably lack-of-HADOOP-1700 causing DATA LOSS
Date Thu, 22 Jan 2009 05:43:22 GMT
Any luck with this Genady?
St.Ack

On Tue, Jan 20, 2009 at 11:01 AM, stack <stack@duboce.net> wrote:

> My guess is that those files are empty.  Can you check them using
> './bin/hadoop fs -ls
> hdfs://master:54310/hbase/step0/980646640/oldlogfile.log'.  They may have
> been removed though.
>
> They are written by master splitting logs of a crashed regionserver so can
> you back up and look see why regionserver crashed in first place?
>
> HADOOP-1700 has been closed but hbase still needs some HDFS loving
> minimizing dataloss.  See HADOOP-4379 (Hopefully fixed in hadoop 0.19.1).
>
> St.Ack
>
>
>
>
> Genady Gillin wrote:
>
>> Hi,
>>
>> We're trying to use Hadoop0.19/Hbase 0.19 RC with four nodes, region
>> server
>> already has about 150 regions, each few successful runs one of region
>> servers keep to close all regions and fail the running task with the log
>> errors are listed below, complaining about Hadoop-1700 issue, which
>> already
>> fixed in Hadoop 19, is there is some workaround to prevent this behaviour
>> ?
>>
>> Thanks,
>> Gennady
>>
>>
>> *2009-01-20 10:46:53,049 WARN org.apache.hadoop.hbase.regionserver.HStore:
>> Exception processing reconstruction log
>> hdfs://master:54310/hbase/step0/980646640/oldlogfile.log opening
>> [B@940592-- continuing.  Probably lack-of-HADOOP-1700 causing DATA
>> LOSS!
>> java.io.EOFException
>>        at java.io.DataInputStream.readFully(DataInputStream.java:180)
>>        at java.io.DataInputStream.readFully(DataInputStream.java:152)
>>        at
>>
>> org.apache.hadoop.hbase.io.SequenceFile$Reader.init(SequenceFile.java:1464)
>>        at
>>
>> org.apache.hadoop.hbase.io.SequenceFile$Reader.<init>(SequenceFile.java:1442)
>>        at
>>
>> org.apache.hadoop.hbase.io.SequenceFile$Reader.<init>(SequenceFile.java:1431)
>>        at
>>
>> org.apache.hadoop.hbase.io.SequenceFile$Reader.<init>(SequenceFile.java:1426)
>>        at
>>
>> org.apache.hadoop.hbase.regionserver.HStore.doReconstructionLog(HStore.java:342)
>>        at
>>
>> org.apache.hadoop.hbase.regionserver.HStore.runReconstructionLog(HStore.java:297)
>>        at
>> org.apache.hadoop.hbase.regionserver.HStore.<init>(HStore.java:237)
>>        at
>>
>> org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:1764)
>>        at
>> org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:276)
>>        at
>>
>> org.apache.hadoop.hbase.regionserver.HRegionServer.instantiateRegion(HRegionServer.java:1367)
>>        at
>>
>> org.apache.hadoop.hbase.regionserver.HRegionServer.openRegion(HRegionServer.java:1338)
>>        at
>>
>> org.apache.hadoop.hbase.regionserver.HRegionServer$Worker.run(HRegionServer.java:1253)
>>        at java.lang.Thread.run(Thread.java:619)
>> 2009-01-20 10:46:53,303 WARN org.apache.hadoop.hbase.regionserver.HStore:
>> Exception processing reconstruction log
>> hdfs://master:54310/hbase/step0/980646640/oldlogfile.log opening
>> [B@980af4-- continuing.  Probably lack-of-HADOOP-1700 causing DATA
>> LOSS!
>> java.io.EOFException
>>        at java.io.DataInputStream.readFully(DataInputStream.java:180)
>>        at java.io.DataInputStream.readFully(DataInputStream.java:152)
>>        at
>>
>> org.apache.hadoop.hbase.io.SequenceFile$Reader.init(SequenceFile.java:1464)
>>        at
>>
>> org.apache.hadoop.hbase.io.SequenceFile$Reader.<init>(SequenceFile.java:1442)
>>        at
>>
>> org.apache.hadoop.hbase.io.SequenceFile$Reader.<init>(SequenceFile.java:1431)
>>        at
>>
>> org.apache.hadoop.hbase.io.SequenceFile$Reader.<init>(SequenceFile.java:1426)
>>        at
>>
>> org.apache.hadoop.hbase.regionserver.HStore.doReconstructionLog(HStore.java:342)
>>        at
>>
>> org.apache.hadoop.hbase.regionserver.HStore.runReconstructionLog(HStore.java:297)
>>        at
>> org.apache.hadoop.hbase.regionserver.HStore.<init>(HStore.java:237)
>>        at
>>
>> org.apache.hadoop.hbase.regionserver.HRegion.instantiateHStore(HRegion.java:1764)
>>        at
>> org.apache.hadoop.hbase.regionserver.HRegion.initialize(HRegion.java:276)
>>        at
>>
>> org.apache.hadoop.hbase.regionserver.HRegionServer.instantiateRegion(HRegionServer.java:1367)
>>        at
>>
>> org.apache.hadoop.hbase.regionserver.HRegionServer.openRegion(HRegionServer.java:1338)
>>        at
>>
>> org.apache.hadoop.hbase.regionserver.HRegionServer$Worker.run(HRegionServer.java:1253)
>>        at java.lang.Thread.run(Thread.java:619)
>> *
>>
>>
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message