hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <apurt...@apache.org>
Subject Re: OOME hell
Date Tue, 02 Dec 2008 08:54:02 GMT
> Is it possible to disable compression on a column family
> after enabling it? Wouldn't that cause subsequent reads
> to return the compressed (therefore, scrambled) data? Or
> would this work as intended: Old data is uncompressed 
> transparently then written in uncompressed form into new
> storefiles as compactions happen going forward. 

I reinitialized with compression *not* enabled and ran the
cluster for a while. Around the time the region count got 
above 180, after a split one of the regions went offline.
The crawler then got NSREs on store attempts. Scanners got
this:

java.io.IOException: java.io.IOException: java.lang.IndexOutOfBoundsException
        at java.io.DataInputStream.readFully(DataInputStream.java:175)
        at org.apache.hadoop.io.DataOutputBuffer$Buffer.write(DataOutputBuffer.java:64)
        at org.apache.hadoop.io.DataOutputBuffer.write(DataOutputBuffer.java:102)
        at org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1933)
        at org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1833)
        at org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1879)
        at org.apache.hadoop.io.MapFile$Reader.next(MapFile.java:516)
        at org.apache.hadoop.hbase.regionserver.StoreFileScanner.getNext(StoreFileScanner.java:312)
        at org.apache.hadoop.hbase.regionserver.StoreFileScanner.next(StoreFileScanner.java:183)
        at org.apache.hadoop.hbase.regionserver.HStoreScanner.next(HStoreScanner.java:196)
        at org.apache.hadoop.hbase.regionserver.HRegion$HScanner.next(HRegion.java:2027)
        at org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1087)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        at java.lang.reflect.Method.invoke(Method.java:597)
        at org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:554)
        at org.apache.hadoop.ipc.Server$Handler.run(Server.java:888)

After a restart all seems well. But that restart was a painful
manual intervention. Have you seen this before? I can file a JIRA
or comment on an existing one.

Please advise,

   - Andy






      

Mime
View raw message