hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nicolas Spiegelberg (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HBASE-3155) HFile.appendMetaBlock() uses wrong comparator
Date Tue, 26 Oct 2010 18:49:19 GMT

    [ https://issues.apache.org/jira/browse/HBASE-3155?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12925085#action_12925085

Nicolas Spiegelberg commented on HBASE-3155:

Code prior to this patch is probably not sorted right, so the fix I give won't be backwards
compatible for correctly finding meta data without further effort.  The worst case should
be that you request a meta block and it isn't found.  In our case, this is fine because all
we store in META is bloom data.  Note that this bug was introduced by the addition of BloomFilters,
so only 0.89 + 0.90 users will experience any migration 'oddities', 0.20 users who upgrade
should be fine.

> HFile.appendMetaBlock() uses wrong comparator
> ---------------------------------------------
>                 Key: HBASE-3155
>                 URL: https://issues.apache.org/jira/browse/HBASE-3155
>             Project: HBase
>          Issue Type: Bug
>            Reporter: Nicolas Spiegelberg
>            Assignee: Nicolas Spiegelberg
>            Priority: Critical
>             Fix For: 0.90.0
> We hit this exception last night...
> 2010-10-26 01:20:20,056 INFO org.apache.hadoop.hbase.regionserver.StoreFile: Bloom added
to HFile (...): 18752B, 13012/13601 (96%)
> 2010-10-26 01:20:20,056 INFO org.apache.hadoop.hbase.regionserver.HRegion: aborted compaction
on region 04,04c84c80,1286302852528.77d461b19c7f410041f1d03f4823ef8b. after 20mins, 43sec
> 2010-10-26 01:20:20,056 ERROR org.apache.hadoop.hbase.regionserver.CompactSplitThread:
Compaction failed for region 04,04c84c80,1286302852528.77d461b19c7f410041f1d03f4823ef8b.
> java.lang.ArrayIndexOutOfBoundsException: 17
> at org.apache.hadoop.hbase.util.Bytes.compareTo(Bytes.java:860)
> at org.apache.hadoop.hbase.KeyValue$KeyComparator.compareRows(KeyValue.java:1888)
> at org.apache.hadoop.hbase.KeyValue$KeyComparator.compare(KeyValue.java:1822)
> at org.apache.hadoop.hbase.io.hfile.HFile$Writer.appendMetaBlock(HFile.java:476)
> at org.apache.hadoop.hbase.regionserver.StoreFile$Writer.close(StoreFile.java:862)
> at org.apache.hadoop.hbase.regionserver.Store.compact(Store.java:896)
> at org.apache.hadoop.hbase.regionserver.Store.compact(Store.java:687)
> at org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:858)
> at org.apache.hadoop.hbase.regionserver.HRegion.compactStores(HRegion.java:807)
> The problem is that appendMetaBlock() is using the wrong comparator. although the variable
is called 'rawComparator', it's actually a normal comparator (KeyComparator) that defaults
to RawComparator if not specified. All meta sorting needs to be done using the actual Bytes.RAW_COMPARATOR.
 This happened because >=2 things were inserted into meta. 

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message