hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "chunhui shen (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HBASE-11234) FastDiffDeltaEncoder#getFirstKeyInBlock returns wrong result
Date Thu, 22 May 2014 08:50:39 GMT

     [ https://issues.apache.org/jira/browse/HBASE-11234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

chunhui shen updated HBASE-11234:
---------------------------------

    Description: 
As Ted found, 

With this change:
{noformat}
Index: hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestReversibleScanners.java
===================================================================
--- hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestReversibleScanners.java
(revision 1596579)
+++ hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestReversibleScanners.java
(working copy)
@@ -51,6 +51,7 @@
 import org.apache.hadoop.hbase.filter.FilterList.Operator;
 import org.apache.hadoop.hbase.filter.PageFilter;
 import org.apache.hadoop.hbase.filter.SingleColumnValueFilter;
+import org.apache.hadoop.hbase.io.encoding.DataBlockEncoding;
 import org.apache.hadoop.hbase.io.hfile.CacheConfig;
 import org.apache.hadoop.hbase.io.hfile.HFileContext;
 import org.apache.hadoop.hbase.io.hfile.HFileContextBuilder;
@@ -90,6 +91,7 @@
     CacheConfig cacheConf = new CacheConfig(TEST_UTIL.getConfiguration());
     HFileContextBuilder hcBuilder = new HFileContextBuilder();
     hcBuilder.withBlockSize(2 * 1024);
+    hcBuilder.withDataBlockEncoding(DataBlockEncoding.FAST_DIFF);
     HFileContext hFileContext = hcBuilder.build();
     StoreFile.Writer writer = new StoreFile.WriterBuilder(
         TEST_UTIL.getConfiguration(), cacheConf, fs).withOutputDir(
{noformat}
I got:

java.lang.AssertionError: expected:<testRow0197/testCf:testQual0000/1400712260004/Put/vlen=13/mvcc=5>
but was:<testRow0198/testCf:testQual0000/1400712260004/   Put/vlen=13/mvcc=0>
  at org.junit.Assert.fail(Assert.java:88)
  at org.junit.Assert.failNotEquals(Assert.java:743)
  at org.junit.Assert.assertEquals(Assert.java:118)
  at org.junit.Assert.assertEquals(Assert.java:144)
  at org.apache.hadoop.hbase.regionserver.TestReversibleScanners.seekTestOfReversibleKeyValueScanner(TestReversibleScanners.java:533)
  at org.apache.hadoop.hbase.regionserver.TestReversibleScanners.testReversibleStoreFileScanner(TestReversibleScanners.java:108)



After debugging, it seems the method of FastDiffDeltaEncoder#getFirstKeyInBlock become broken.
And it will cause hfilescanner#seekBefore returns wrong result.


The solution is simple, see the patch.

  was:
As Ted found, 
{noformat}
With this change:

Index: hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestReversibleScanners.java
===================================================================
--- hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestReversibleScanners.java
(revision 1596579)
+++ hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestReversibleScanners.java
(working copy)
@@ -51,6 +51,7 @@
 import org.apache.hadoop.hbase.filter.FilterList.Operator;
 import org.apache.hadoop.hbase.filter.PageFilter;
 import org.apache.hadoop.hbase.filter.SingleColumnValueFilter;
+import org.apache.hadoop.hbase.io.encoding.DataBlockEncoding;
 import org.apache.hadoop.hbase.io.hfile.CacheConfig;
 import org.apache.hadoop.hbase.io.hfile.HFileContext;
 import org.apache.hadoop.hbase.io.hfile.HFileContextBuilder;
@@ -90,6 +91,7 @@
     CacheConfig cacheConf = new CacheConfig(TEST_UTIL.getConfiguration());
     HFileContextBuilder hcBuilder = new HFileContextBuilder();
     hcBuilder.withBlockSize(2 * 1024);
+    hcBuilder.withDataBlockEncoding(DataBlockEncoding.FAST_DIFF);
     HFileContext hFileContext = hcBuilder.build();
     StoreFile.Writer writer = new StoreFile.WriterBuilder(
         TEST_UTIL.getConfiguration(), cacheConf, fs).withOutputDir(

I got:

java.lang.AssertionError: expected:<testRow0197/testCf:testQual0000/1400712260004/Put/vlen=13/mvcc=5>
but was:<testRow0198/testCf:testQual0000/1400712260004/   Put/vlen=13/mvcc=0>
  at org.junit.Assert.fail(Assert.java:88)
  at org.junit.Assert.failNotEquals(Assert.java:743)
  at org.junit.Assert.assertEquals(Assert.java:118)
  at org.junit.Assert.assertEquals(Assert.java:144)
  at org.apache.hadoop.hbase.regionserver.TestReversibleScanners.seekTestOfReversibleKeyValueScanner(TestReversibleScanners.java:533)
  at org.apache.hadoop.hbase.regionserver.TestReversibleScanners.testReversibleStoreFileScanner(TestReversibleScanners.java:108)
{noformat}


After debugging, it seems the method of FastDiffDeltaEncoder#getFirstKeyInBlock become broken.
And it will cause hfilescanner#seekBefore returns wrong result.


The solution is simple, see the patch.


> FastDiffDeltaEncoder#getFirstKeyInBlock returns wrong result
> ------------------------------------------------------------
>
>                 Key: HBASE-11234
>                 URL: https://issues.apache.org/jira/browse/HBASE-11234
>             Project: HBase
>          Issue Type: Bug
>            Reporter: chunhui shen
>            Assignee: chunhui shen
>            Priority: Critical
>             Fix For: 0.99.0
>
>         Attachments: HBASE-11234.patch
>
>
> As Ted found, 
> With this change:
> {noformat}
> Index: hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestReversibleScanners.java
> ===================================================================
> --- hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestReversibleScanners.java
(revision 1596579)
> +++ hbase-server/src/test/java/org/apache/hadoop/hbase/regionserver/TestReversibleScanners.java
(working copy)
> @@ -51,6 +51,7 @@
>  import org.apache.hadoop.hbase.filter.FilterList.Operator;
>  import org.apache.hadoop.hbase.filter.PageFilter;
>  import org.apache.hadoop.hbase.filter.SingleColumnValueFilter;
> +import org.apache.hadoop.hbase.io.encoding.DataBlockEncoding;
>  import org.apache.hadoop.hbase.io.hfile.CacheConfig;
>  import org.apache.hadoop.hbase.io.hfile.HFileContext;
>  import org.apache.hadoop.hbase.io.hfile.HFileContextBuilder;
> @@ -90,6 +91,7 @@
>      CacheConfig cacheConf = new CacheConfig(TEST_UTIL.getConfiguration());
>      HFileContextBuilder hcBuilder = new HFileContextBuilder();
>      hcBuilder.withBlockSize(2 * 1024);
> +    hcBuilder.withDataBlockEncoding(DataBlockEncoding.FAST_DIFF);
>      HFileContext hFileContext = hcBuilder.build();
>      StoreFile.Writer writer = new StoreFile.WriterBuilder(
>          TEST_UTIL.getConfiguration(), cacheConf, fs).withOutputDir(
> {noformat}
> I got:
> java.lang.AssertionError: expected:<testRow0197/testCf:testQual0000/1400712260004/Put/vlen=13/mvcc=5>
but was:<testRow0198/testCf:testQual0000/1400712260004/   Put/vlen=13/mvcc=0>
>   at org.junit.Assert.fail(Assert.java:88)
>   at org.junit.Assert.failNotEquals(Assert.java:743)
>   at org.junit.Assert.assertEquals(Assert.java:118)
>   at org.junit.Assert.assertEquals(Assert.java:144)
>   at org.apache.hadoop.hbase.regionserver.TestReversibleScanners.seekTestOfReversibleKeyValueScanner(TestReversibleScanners.java:533)
>   at org.apache.hadoop.hbase.regionserver.TestReversibleScanners.testReversibleStoreFileScanner(TestReversibleScanners.java:108)
> After debugging, it seems the method of FastDiffDeltaEncoder#getFirstKeyInBlock become
broken. And it will cause hfilescanner#seekBefore returns wrong result.
> The solution is simple, see the patch.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message