hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-1039) Compaction fails if bloomfilters are enabled
Date Mon, 01 Dec 2008 14:44:46 GMT

    [ https://issues.apache.org/jira/browse/HBASE-1039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12652014#action_12652014
] 

Andrew Purtell commented on HBASE-1039:
---------------------------------------

One crucial detail it seems is that the bloomfilter related exception happens even when no
bloomfilters are enabled in the schema.  There are also DFS related exceptions. 

>From Thibaut:

I created all the tables from scratch and didn't change them at run time. The schema for all
the tables right now is as followed. (data is a bytearray of a serialized google buffer object)

    {NAME => 'entries', IS_ROOT => 'false', IS_META => 'false', FAMILIES => [{NAME
=> 'data', BLOOMFILTER => 'false', COMPRESSION => 'NONE', VERSIONS => '3', LENGTH
=> '2147483647', TTL => '-1', IN_MEMORY => 'false', BLOCKCACHE => 'false'}]}

I reran everything from scratch with the new table scheme and got the same exception again,
just on a different table this time: (Disabling the bloomfilter, compression and the blockcache
doesn't seem to have any effect)

2008-11-30 23:22:20,774 ERROR org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction
failed for region entries,,1228075277421
java.lang.IllegalArgumentException: maxValue must be > 0
    at org.onelab.filter.HashFunction.<init>(HashFunction.java:84)
    at org.onelab.filter.Filter.<init>(Filter.java:97)
    at org.onelab.filter.BloomFilter.<init>(BloomFilter.java:102)
    at org.apache.hadoop.hbase.regionserver.HStoreFile$BloomFilterMapFile$Writer.<init>(HStoreFile.java:829)
    at org.apache.hadoop.hbase.regionserver.HStoreFile.getWriter(HStoreFile.java:436)
    at org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:889)
    at org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:902)
    at org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:860)
    at org.apache.hadoop.hbase.regionserver.CompactSplitThread.run(CompactSplitThread.java:83)

The log file is also full of these kind of errors: (before and after)

2008-11-30 23:22:44,500 INFO org.apache.hadoop.ipc.Server: IPC Server handler 16 on 60020,
call next(8976385860586379110) from x.x.x.203:52747: error: org.apache.hadoop.hbase.UnknownScannerException:
Name: 8976385860586379110
    at org.apache.hadoop.hbase.regionserver.HRegionServer.next(HRegionServer.java:1077)
    at sun.reflect.GeneratedMethodAccessor4.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:585)
    at org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:554)
    at org.apache.hadoop.ipc.Server$Handler.run(Server.java:888)

Shortly afterwards I got the dfs error on the regionserver again, on a different table though
(and might be completely unreleated and not important?):

2008-11-30 23:26:55,885 WARN org.apache.hadoop.dfs.DFSClient: Exception while reading from
blk_-9066140877711029349_706715 of /hbase/webrequestscache/2091560474/data/mapfiles/1510543474646532027/data
from x.x.x.204:50010: java.io.IOException: Premeture EOF from inputStream
    at org.apache.hadoop.io.IOUtils.readFully(IOUtils.java:102)
    at org.apache.hadoop.dfs.DFSClient$BlockReader.readChunk(DFSClient.java:996)
    at org.apache.hadoop.fs.FSInputChecker.readChecksumChunk(FSInputChecker.java:236)
    at org.apache.hadoop.fs.FSInputChecker.read1(FSInputChecker.java:191)
    at org.apache.hadoop.fs.FSInputChecker.read(FSInputChecker.java:159)
    at org.apache.hadoop.dfs.DFSClient$BlockReader.read(DFSClient.java:858)
    at org.apache.hadoop.dfs.DFSClient$DFSInputStream.readBuffer(DFSClient.java:1384)
    at org.apache.hadoop.dfs.DFSClient$DFSInputStream.read(DFSClient.java:1420)
    at java.io.DataInputStream.readFully(DataInputStream.java:176)
    at org.apache.hadoop.io.DataOutputBuffer$Buffer.write(DataOutputBuffer.java:64)
    at org.apache.hadoop.io.DataOutputBuffer.write(DataOutputBuffer.java:102)
    at org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1933)
    at org.apache.hadoop.io.SequenceFile$Reader.next(SequenceFile.java:1833)
    at org.apache.hadoop.io.MapFile$Reader.seekInternal(MapFile.java:463)
    at org.apache.hadoop.io.MapFile$Reader.getClosest(MapFile.java:558)
    at org.apache.hadoop.io.MapFile$Reader.getClosest(MapFile.java:541)
    at org.apache.hadoop.hbase.regionserver.HStoreFile$BloomFilterMapFile$Reader.getClosest(HStoreFile.java:761)
    at org.apache.hadoop.hbase.regionserver.HStore.get(HStore.java:1291)
    at org.apache.hadoop.hbase.regionserver.HRegion.get(HRegion.java:1154)
    at org.apache.hadoop.hbase.regionserver.HRegionServer.get(HRegionServer.java:1020)
    at sun.reflect.GeneratedMethodAccessor9.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
    at java.lang.reflect.Method.invoke(Method.java:585)
    at org.apache.hadoop.hbase.ipc.HbaseRPC$Server.call(HbaseRPC.java:554)
    at org.apache.hadoop.ipc.Server$Handler.run(Server.java:888)

Datnode entries related to that block:

08/11/30 22:44:19 INFO dfs.DataNode: Receiving block blk_-9066140877711029349_706715 src:
/x.x.x.204:44313 dest: /x.x.x.204:50010
08/11/30 22:44:41 INFO dfs.DataNode: Received block blk_-9066140877711029349_706715 of size
33554432 from /x.x.x.204
08/11/30 22:44:41 INFO dfs.DataNode: PacketResponder 3 for block blk_-9066140877711029349_706715
terminating
08/11/30 22:53:18 WARN dfs.DataNode: DatanodeRegistration(x.x.x.204:50010, storageID=DS-364968361-x.x.x.204-50010-1220223683238,
infoPort=50075, ipcPort=50020):Got exception while serving blk_-9066140877711029349_706715
to /x.x.x.204: java.net.SocketTimeoutException: 480000 millis timeout while waiting for channel
to be ready for write. ch :java.nio.channels.SocketChannel[connected local=/x.x.x.204:50010
remote=/x.x.x.204:46220]
        at org.apache.hadoop.net.SocketIOWithTimeout.waitForIO(SocketIOWithTimeout.java:185)
        at org.apache.hadoop.net.SocketOutputStream.waitForWritable(SocketOutputStream.java:159)
        at org.apache.hadoop.net.SocketOutputStream.transferToFully(SocketOutputStream.java:198)
        at org.apache.hadoop.dfs.DataNode$BlockSender.sendChunks(DataNode.java:1873)
        at org.apache.hadoop.dfs.DataNode$BlockSender.sendBlock(DataNode.java:1967)
        at org.apache.hadoop.dfs.DataNode$DataXceiver.readBlock(DataNode.java:1109)
        at org.apache.hadoop.dfs.DataNode$DataXceiver.run(DataNode.java:1037)
        at java.lang.Thread.run(Thread.java:595)


> Compaction fails if bloomfilters are enabled
> --------------------------------------------
>
>                 Key: HBASE-1039
>                 URL: https://issues.apache.org/jira/browse/HBASE-1039
>             Project: Hadoop HBase
>          Issue Type: Bug
>          Components: regionserver
>    Affects Versions: 0.18.1
>            Reporter: Andrew Purtell
>
> From Thibaut up on the list.
> As soon as hbase tries to compact the table, the following exception appears in the logfile:
(Other compactations also work fine without any errors)
> 2008-11-30 00:55:57,769 ERROR
> org.apache.hadoop.hbase.regionserver.CompactSplitThread: Compaction failed for region
mytable,,1228002541526
> java.lang.IllegalArgumentException: maxValue must be > 0
>     at org.onelab.filter.HashFunction.<init>(HashFunction.java:84)
>     at org.onelab.filter.Filter.<init>(Filter.java:97)
>     at org.onelab.filter.BloomFilter.<init>(BloomFilter.java:102)
>     at org.apache.hadoop.hbase.regionserver.HStoreFile$BloomFilterMapFile$Writer.<init>(HStoreFile.java:829)
>     at org.apache.hadoop.hbase.regionserver.HStoreFile.getWriter(HStoreFile.java:436)
>     at org.apache.hadoop.hbase.regionserver.HStore.compact(HStore.java:889)
>     at org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:902)
>     at org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:860)
>     at org.apache.hadoop.hbase.regionserver.CompactSplitThread.run(CompactSplitThread.java:83)
> Because the region cannot compact and/or split, it is soon dead after (re)assignment.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message