drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Barclay (Drill) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-3954) HBase tests use only 1 region, don't detect bug(s) in dummy-column NullableIntVector creation/resolution
Date Sun, 01 Nov 2015 03:31:27 GMT

    [ https://issues.apache.org/jira/browse/DRILL-3954?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14984247#comment-14984247
] 

Daniel Barclay (Drill) commented on DRILL-3954:
-----------------------------------------------

One problem revealed by increasing the numbers of regions beyond one is that one of the HBase
tests violates the condition whose check yields the error message saying that column names
"must be singular names" (which actually should say something like "simple names" (a standard
term for the intended concept)).

(Additionally, the fact that changing the number of regions and thereby (presumably) fragments
indicates that Drill's check for that column-name requirement is hacked-\-checking at the
point of serialization or fragment splitting or whatever code imposes that requirement--rather
than checking earlier when column names should be checked for all relevant requirements (so
that, for example, the number of regions a table has doesn't affect whether the column name
is checked).) 

> HBase tests use only 1 region, don't detect bug(s) in dummy-column NullableIntVector
creation/resolution
> --------------------------------------------------------------------------------------------------------
>
>                 Key: DRILL-3954
>                 URL: https://issues.apache.org/jira/browse/DRILL-3954
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Storage - HBase
>            Reporter: Daniel Barclay (Drill)
>
> Currently, the HBase tests (e.g., {{TestHBaseFilterPushDown}}) use only one region.
> That causes them to miss detecting a bug in creating and/or resolving dummy fields ({{NullableIntVectors}}
for referenced but non-existent fields) somewhere between reading from HBase and wherever
dummy fields are supposed to be get resolved  (or maybe two separate bugs).
> Reproduction:
> In HBaseTestsSuite, change the line:
> {noformat}
>             UTIL.startMiniHBaseCluster(1, 1);
> {noformat}
> to:
> {noformat}
>             UTIL.startMiniHBaseCluster(1, 3);
> {noformat}
> and change the line:
> {noformat}
>     TestTableGenerator.generateHBaseDataset1(admin, TEST_TABLE_1, 1);
> {noformat}
> to:
> {noformat}
>     TestTableGenerator.generateHBaseDataset1(admin, TEST_TABLE_1, 3);
> {noformat}
> .
> Run unit test class {{TestHBaseFilterPushDown}}.
> Depending on which region gets processed first (it's non-deteministic), test methods
{{testFilterPushDownOrRowKeyEqualRangePred}} and 
> {{testFilterPushDownMultiColumns}} get exceptions like this:
> {noformat}
> java.lang.IndexOutOfBoundsException: index: 0, length: 1 (expected: range(0, 0))
> 	at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:189)
> 	at io.netty.buffer.DrillBuf.chk(DrillBuf.java:211)
> 	at io.netty.buffer.DrillBuf.getByte(DrillBuf.java:746)
> 	at org.apache.drill.exec.vector.UInt1Vector$Accessor.get(UInt1Vector.java:364)
> 	at org.apache.drill.exec.vector.NullableVarBinaryVector$Accessor.isSet(NullableVarBinaryVector.java:391)
> 	at org.apache.drill.exec.vector.NullableVarBinaryVector$Accessor.isNull(NullableVarBinaryVector.java:387)
> 	at org.apache.drill.exec.vector.NullableVarBinaryVector$Accessor.getObject(NullableVarBinaryVector.java:411)
> 	at org.apache.drill.exec.vector.NullableVarBinaryVector$Accessor.getObject(NullableVarBinaryVector.java:1)
> 	at org.apache.drill.exec.vector.complex.MapVector$Accessor.getObject(MapVector.java:313)
> 	at org.apache.drill.exec.util.VectorUtil.showVectorAccessibleContent(VectorUtil.java:166)
> 	at org.apache.drill.BaseTestQuery.printResult(BaseTestQuery.java:487)
> 	at org.apache.drill.hbase.BaseHBaseTest.printResultAndVerifyRowCount(BaseHBaseTest.java:95)
> 	at org.apache.drill.hbase.BaseHBaseTest.runHBaseSQLVerifyCount(BaseHBaseTest.java:91)
> 	at org.apache.drill.hbase.TestHBaseFilterPushDown.testFilterPushDownMultiColumns(TestHBaseFilterPushDown.java:592)
> 	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> 	at java.lang.reflect.Method.invoke(Method.java:606)
> 	at java.lang.reflect.Method.invoke(Method.java:606)
> {noformat}
> See DRILL-3955.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message