drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Barclay (Drill) (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (DRILL-3954) HBase tests use only 1 region, don't detect bug(s) in dummy-column NullableIntVector creation/resolution
Date Mon, 19 Oct 2015 23:51:27 GMT

     [ https://issues.apache.org/jira/browse/DRILL-3954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Daniel Barclay (Drill) updated DRILL-3954:
------------------------------------------
    Description: 
Currently, the HBase tests (e.g., {{TestHBaseFilterPushDown}}) use only one region.

That causes them to miss detecting a bug in creating and/or resolving dummy fields ({{NullableIntVectors}}
for referenced but non-existent fields) somewhere between reading from HBase and wherever
dummy fields are supposed to be get resolved  (or maybe two separate bugs).

Reproduction:

In HBaseTestsSuite, change the line:
{noformat}
            UTIL.startMiniHBaseCluster(1, 1);
{noformat}
to:
{noformat}
            UTIL.startMiniHBaseCluster(1, 3);
{noformat}
and change the line:
{noformat}
    TestTableGenerator.generateHBaseDataset1(admin, TEST_TABLE_1, 1);
{noformat}
to:
{noformat}
    TestTableGenerator.generateHBaseDataset1(admin, TEST_TABLE_1, 3);
{noformat}
.

Run unit test class {{TestHBaseFilterPushDown}}.

Depending on which region gets processed first (it's non-deteministic), test methods {{testFilterPushDownOrRowKeyEqualRangePred}}
and 
{{testFilterPushDownMultiColumns}} get exceptions like this:

{noformat}
java.lang.IndexOutOfBoundsException: index: 0, length: 1 (expected: range(0, 0))
	at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:189)
	at io.netty.buffer.DrillBuf.chk(DrillBuf.java:211)
	at io.netty.buffer.DrillBuf.getByte(DrillBuf.java:746)
	at org.apache.drill.exec.vector.UInt1Vector$Accessor.get(UInt1Vector.java:364)
	at org.apache.drill.exec.vector.NullableVarBinaryVector$Accessor.isSet(NullableVarBinaryVector.java:391)
	at org.apache.drill.exec.vector.NullableVarBinaryVector$Accessor.isNull(NullableVarBinaryVector.java:387)
	at org.apache.drill.exec.vector.NullableVarBinaryVector$Accessor.getObject(NullableVarBinaryVector.java:411)
	at org.apache.drill.exec.vector.NullableVarBinaryVector$Accessor.getObject(NullableVarBinaryVector.java:1)
	at org.apache.drill.exec.vector.complex.MapVector$Accessor.getObject(MapVector.java:313)
	at org.apache.drill.exec.util.VectorUtil.showVectorAccessibleContent(VectorUtil.java:166)
	at org.apache.drill.BaseTestQuery.printResult(BaseTestQuery.java:487)
	at org.apache.drill.hbase.BaseHBaseTest.printResultAndVerifyRowCount(BaseHBaseTest.java:95)
	at org.apache.drill.hbase.BaseHBaseTest.runHBaseSQLVerifyCount(BaseHBaseTest.java:91)
	at org.apache.drill.hbase.TestHBaseFilterPushDown.testFilterPushDownMultiColumns(TestHBaseFilterPushDown.java:592)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at java.lang.reflect.Method.invoke(Method.java:606)
{noformat}

See DRILL-TBD.

  was:
Currently, the HBase tests (e.g., {{TestHBaseFilterPushDown}}) use only one region.

That causes them to miss detecting a bug in creating and/or resolving dummy fields ({{NullableIntVectors}}
for referenced but non-existent fields) somewhere between reading from HBase and {{ProjectRecordBatch.setupNewSchema}}
(or maybe two separate bugs).

Reproduction:

In HBaseTestsSuite, change the line:
{noformat}
            UTIL.startMiniHBaseCluster(1, 1);
{noformat}
to:
{noformat}
            UTIL.startMiniHBaseCluster(1, 3);
{noformat}
and change the line:
{noformat}
    TestTableGenerator.generateHBaseDataset1(admin, TEST_TABLE_1, 1);
{noformat}
to:
{noformat}
    TestTableGenerator.generateHBaseDataset1(admin, TEST_TABLE_1, 3);
{noformat}
.

Run unit test class {{TestHBaseFilterPushDown}}.

Depending on which region gets processed first (it's non-deteministic), test methods {{testFilterPushDownOrRowKeyEqualRangePred}}
and 
{{testFilterPushDownMultiColumns}} get exceptions like this:

{noformat}
java.lang.IndexOutOfBoundsException: index: 0, length: 1 (expected: range(0, 0))
	at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:189)
	at io.netty.buffer.DrillBuf.chk(DrillBuf.java:211)
	at io.netty.buffer.DrillBuf.getByte(DrillBuf.java:746)
	at org.apache.drill.exec.vector.UInt1Vector$Accessor.get(UInt1Vector.java:364)
	at org.apache.drill.exec.vector.NullableVarBinaryVector$Accessor.isSet(NullableVarBinaryVector.java:391)
	at org.apache.drill.exec.vector.NullableVarBinaryVector$Accessor.isNull(NullableVarBinaryVector.java:387)
	at org.apache.drill.exec.vector.NullableVarBinaryVector$Accessor.getObject(NullableVarBinaryVector.java:411)
	at org.apache.drill.exec.vector.NullableVarBinaryVector$Accessor.getObject(NullableVarBinaryVector.java:1)
	at org.apache.drill.exec.vector.complex.MapVector$Accessor.getObject(MapVector.java:313)
	at org.apache.drill.exec.util.VectorUtil.showVectorAccessibleContent(VectorUtil.java:166)
	at org.apache.drill.BaseTestQuery.printResult(BaseTestQuery.java:487)
	at org.apache.drill.hbase.BaseHBaseTest.printResultAndVerifyRowCount(BaseHBaseTest.java:95)
	at org.apache.drill.hbase.BaseHBaseTest.runHBaseSQLVerifyCount(BaseHBaseTest.java:91)
	at org.apache.drill.hbase.TestHBaseFilterPushDown.testFilterPushDownMultiColumns(TestHBaseFilterPushDown.java:592)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.lang.reflect.Method.invoke(Method.java:606)
	at java.lang.reflect.Method.invoke(Method.java:606)
{noformat}

See DRILL-TBD.


> HBase tests use only 1 region, don't detect bug(s) in dummy-column NullableIntVector
creation/resolution
> --------------------------------------------------------------------------------------------------------
>
>                 Key: DRILL-3954
>                 URL: https://issues.apache.org/jira/browse/DRILL-3954
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Storage - HBase
>            Reporter: Daniel Barclay (Drill)
>
> Currently, the HBase tests (e.g., {{TestHBaseFilterPushDown}}) use only one region.
> That causes them to miss detecting a bug in creating and/or resolving dummy fields ({{NullableIntVectors}}
for referenced but non-existent fields) somewhere between reading from HBase and wherever
dummy fields are supposed to be get resolved  (or maybe two separate bugs).
> Reproduction:
> In HBaseTestsSuite, change the line:
> {noformat}
>             UTIL.startMiniHBaseCluster(1, 1);
> {noformat}
> to:
> {noformat}
>             UTIL.startMiniHBaseCluster(1, 3);
> {noformat}
> and change the line:
> {noformat}
>     TestTableGenerator.generateHBaseDataset1(admin, TEST_TABLE_1, 1);
> {noformat}
> to:
> {noformat}
>     TestTableGenerator.generateHBaseDataset1(admin, TEST_TABLE_1, 3);
> {noformat}
> .
> Run unit test class {{TestHBaseFilterPushDown}}.
> Depending on which region gets processed first (it's non-deteministic), test methods
{{testFilterPushDownOrRowKeyEqualRangePred}} and 
> {{testFilterPushDownMultiColumns}} get exceptions like this:
> {noformat}
> java.lang.IndexOutOfBoundsException: index: 0, length: 1 (expected: range(0, 0))
> 	at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:189)
> 	at io.netty.buffer.DrillBuf.chk(DrillBuf.java:211)
> 	at io.netty.buffer.DrillBuf.getByte(DrillBuf.java:746)
> 	at org.apache.drill.exec.vector.UInt1Vector$Accessor.get(UInt1Vector.java:364)
> 	at org.apache.drill.exec.vector.NullableVarBinaryVector$Accessor.isSet(NullableVarBinaryVector.java:391)
> 	at org.apache.drill.exec.vector.NullableVarBinaryVector$Accessor.isNull(NullableVarBinaryVector.java:387)
> 	at org.apache.drill.exec.vector.NullableVarBinaryVector$Accessor.getObject(NullableVarBinaryVector.java:411)
> 	at org.apache.drill.exec.vector.NullableVarBinaryVector$Accessor.getObject(NullableVarBinaryVector.java:1)
> 	at org.apache.drill.exec.vector.complex.MapVector$Accessor.getObject(MapVector.java:313)
> 	at org.apache.drill.exec.util.VectorUtil.showVectorAccessibleContent(VectorUtil.java:166)
> 	at org.apache.drill.BaseTestQuery.printResult(BaseTestQuery.java:487)
> 	at org.apache.drill.hbase.BaseHBaseTest.printResultAndVerifyRowCount(BaseHBaseTest.java:95)
> 	at org.apache.drill.hbase.BaseHBaseTest.runHBaseSQLVerifyCount(BaseHBaseTest.java:91)
> 	at org.apache.drill.hbase.TestHBaseFilterPushDown.testFilterPushDownMultiColumns(TestHBaseFilterPushDown.java:592)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at java.lang.reflect.Method.invoke(Method.java:606)
> 	at java.lang.reflect.Method.invoke(Method.java:606)
> {noformat}
> See DRILL-TBD.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message