Return-Path: X-Original-To: apmail-drill-issues-archive@minotaur.apache.org Delivered-To: apmail-drill-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 0DCEB181DB for ; Mon, 19 Oct 2015 23:51:28 +0000 (UTC) Received: (qmail 14674 invoked by uid 500); 19 Oct 2015 23:51:28 -0000 Delivered-To: apmail-drill-issues-archive@drill.apache.org Received: (qmail 14646 invoked by uid 500); 19 Oct 2015 23:51:27 -0000 Mailing-List: contact issues-help@drill.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@drill.apache.org Delivered-To: mailing list issues@drill.apache.org Received: (qmail 14637 invoked by uid 99); 19 Oct 2015 23:51:27 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 19 Oct 2015 23:51:27 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id B4AEE2C033A for ; Mon, 19 Oct 2015 23:51:27 +0000 (UTC) Date: Mon, 19 Oct 2015 23:51:27 +0000 (UTC) From: "Daniel Barclay (Drill) (JIRA)" To: issues@drill.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Updated] (DRILL-3954) HBase tests use only 1 region, don't detect bug(s) in dummy-column NullableIntVector creation/resolution MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/DRILL-3954?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Barclay (Drill) updated DRILL-3954: ------------------------------------------ Description: Currently, the HBase tests (e.g., {{TestHBaseFilterPushDown}}) use only one region. That causes them to miss detecting a bug in creating and/or resolving dummy fields ({{NullableIntVectors}} for referenced but non-existent fields) somewhere between reading from HBase and wherever dummy fields are supposed to be get resolved (or maybe two separate bugs). Reproduction: In HBaseTestsSuite, change the line: {noformat} UTIL.startMiniHBaseCluster(1, 1); {noformat} to: {noformat} UTIL.startMiniHBaseCluster(1, 3); {noformat} and change the line: {noformat} TestTableGenerator.generateHBaseDataset1(admin, TEST_TABLE_1, 1); {noformat} to: {noformat} TestTableGenerator.generateHBaseDataset1(admin, TEST_TABLE_1, 3); {noformat} . Run unit test class {{TestHBaseFilterPushDown}}. Depending on which region gets processed first (it's non-deteministic), test methods {{testFilterPushDownOrRowKeyEqualRangePred}} and {{testFilterPushDownMultiColumns}} get exceptions like this: {noformat} java.lang.IndexOutOfBoundsException: index: 0, length: 1 (expected: range(0, 0)) at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:189) at io.netty.buffer.DrillBuf.chk(DrillBuf.java:211) at io.netty.buffer.DrillBuf.getByte(DrillBuf.java:746) at org.apache.drill.exec.vector.UInt1Vector$Accessor.get(UInt1Vector.java:364) at org.apache.drill.exec.vector.NullableVarBinaryVector$Accessor.isSet(NullableVarBinaryVector.java:391) at org.apache.drill.exec.vector.NullableVarBinaryVector$Accessor.isNull(NullableVarBinaryVector.java:387) at org.apache.drill.exec.vector.NullableVarBinaryVector$Accessor.getObject(NullableVarBinaryVector.java:411) at org.apache.drill.exec.vector.NullableVarBinaryVector$Accessor.getObject(NullableVarBinaryVector.java:1) at org.apache.drill.exec.vector.complex.MapVector$Accessor.getObject(MapVector.java:313) at org.apache.drill.exec.util.VectorUtil.showVectorAccessibleContent(VectorUtil.java:166) at org.apache.drill.BaseTestQuery.printResult(BaseTestQuery.java:487) at org.apache.drill.hbase.BaseHBaseTest.printResultAndVerifyRowCount(BaseHBaseTest.java:95) at org.apache.drill.hbase.BaseHBaseTest.runHBaseSQLVerifyCount(BaseHBaseTest.java:91) at org.apache.drill.hbase.TestHBaseFilterPushDown.testFilterPushDownMultiColumns(TestHBaseFilterPushDown.java:592) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.lang.reflect.Method.invoke(Method.java:606) at java.lang.reflect.Method.invoke(Method.java:606) {noformat} See DRILL-TBD. was: Currently, the HBase tests (e.g., {{TestHBaseFilterPushDown}}) use only one region. That causes them to miss detecting a bug in creating and/or resolving dummy fields ({{NullableIntVectors}} for referenced but non-existent fields) somewhere between reading from HBase and {{ProjectRecordBatch.setupNewSchema}} (or maybe two separate bugs). Reproduction: In HBaseTestsSuite, change the line: {noformat} UTIL.startMiniHBaseCluster(1, 1); {noformat} to: {noformat} UTIL.startMiniHBaseCluster(1, 3); {noformat} and change the line: {noformat} TestTableGenerator.generateHBaseDataset1(admin, TEST_TABLE_1, 1); {noformat} to: {noformat} TestTableGenerator.generateHBaseDataset1(admin, TEST_TABLE_1, 3); {noformat} . Run unit test class {{TestHBaseFilterPushDown}}. Depending on which region gets processed first (it's non-deteministic), test methods {{testFilterPushDownOrRowKeyEqualRangePred}} and {{testFilterPushDownMultiColumns}} get exceptions like this: {noformat} java.lang.IndexOutOfBoundsException: index: 0, length: 1 (expected: range(0, 0)) at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:189) at io.netty.buffer.DrillBuf.chk(DrillBuf.java:211) at io.netty.buffer.DrillBuf.getByte(DrillBuf.java:746) at org.apache.drill.exec.vector.UInt1Vector$Accessor.get(UInt1Vector.java:364) at org.apache.drill.exec.vector.NullableVarBinaryVector$Accessor.isSet(NullableVarBinaryVector.java:391) at org.apache.drill.exec.vector.NullableVarBinaryVector$Accessor.isNull(NullableVarBinaryVector.java:387) at org.apache.drill.exec.vector.NullableVarBinaryVector$Accessor.getObject(NullableVarBinaryVector.java:411) at org.apache.drill.exec.vector.NullableVarBinaryVector$Accessor.getObject(NullableVarBinaryVector.java:1) at org.apache.drill.exec.vector.complex.MapVector$Accessor.getObject(MapVector.java:313) at org.apache.drill.exec.util.VectorUtil.showVectorAccessibleContent(VectorUtil.java:166) at org.apache.drill.BaseTestQuery.printResult(BaseTestQuery.java:487) at org.apache.drill.hbase.BaseHBaseTest.printResultAndVerifyRowCount(BaseHBaseTest.java:95) at org.apache.drill.hbase.BaseHBaseTest.runHBaseSQLVerifyCount(BaseHBaseTest.java:91) at org.apache.drill.hbase.TestHBaseFilterPushDown.testFilterPushDownMultiColumns(TestHBaseFilterPushDown.java:592) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.lang.reflect.Method.invoke(Method.java:606) at java.lang.reflect.Method.invoke(Method.java:606) {noformat} See DRILL-TBD. > HBase tests use only 1 region, don't detect bug(s) in dummy-column NullableIntVector creation/resolution > -------------------------------------------------------------------------------------------------------- > > Key: DRILL-3954 > URL: https://issues.apache.org/jira/browse/DRILL-3954 > Project: Apache Drill > Issue Type: Bug > Components: Storage - HBase > Reporter: Daniel Barclay (Drill) > > Currently, the HBase tests (e.g., {{TestHBaseFilterPushDown}}) use only one region. > That causes them to miss detecting a bug in creating and/or resolving dummy fields ({{NullableIntVectors}} for referenced but non-existent fields) somewhere between reading from HBase and wherever dummy fields are supposed to be get resolved (or maybe two separate bugs). > Reproduction: > In HBaseTestsSuite, change the line: > {noformat} > UTIL.startMiniHBaseCluster(1, 1); > {noformat} > to: > {noformat} > UTIL.startMiniHBaseCluster(1, 3); > {noformat} > and change the line: > {noformat} > TestTableGenerator.generateHBaseDataset1(admin, TEST_TABLE_1, 1); > {noformat} > to: > {noformat} > TestTableGenerator.generateHBaseDataset1(admin, TEST_TABLE_1, 3); > {noformat} > . > Run unit test class {{TestHBaseFilterPushDown}}. > Depending on which region gets processed first (it's non-deteministic), test methods {{testFilterPushDownOrRowKeyEqualRangePred}} and > {{testFilterPushDownMultiColumns}} get exceptions like this: > {noformat} > java.lang.IndexOutOfBoundsException: index: 0, length: 1 (expected: range(0, 0)) > at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:189) > at io.netty.buffer.DrillBuf.chk(DrillBuf.java:211) > at io.netty.buffer.DrillBuf.getByte(DrillBuf.java:746) > at org.apache.drill.exec.vector.UInt1Vector$Accessor.get(UInt1Vector.java:364) > at org.apache.drill.exec.vector.NullableVarBinaryVector$Accessor.isSet(NullableVarBinaryVector.java:391) > at org.apache.drill.exec.vector.NullableVarBinaryVector$Accessor.isNull(NullableVarBinaryVector.java:387) > at org.apache.drill.exec.vector.NullableVarBinaryVector$Accessor.getObject(NullableVarBinaryVector.java:411) > at org.apache.drill.exec.vector.NullableVarBinaryVector$Accessor.getObject(NullableVarBinaryVector.java:1) > at org.apache.drill.exec.vector.complex.MapVector$Accessor.getObject(MapVector.java:313) > at org.apache.drill.exec.util.VectorUtil.showVectorAccessibleContent(VectorUtil.java:166) > at org.apache.drill.BaseTestQuery.printResult(BaseTestQuery.java:487) > at org.apache.drill.hbase.BaseHBaseTest.printResultAndVerifyRowCount(BaseHBaseTest.java:95) > at org.apache.drill.hbase.BaseHBaseTest.runHBaseSQLVerifyCount(BaseHBaseTest.java:91) > at org.apache.drill.hbase.TestHBaseFilterPushDown.testFilterPushDownMultiColumns(TestHBaseFilterPushDown.java:592) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at java.lang.reflect.Method.invoke(Method.java:606) > at java.lang.reflect.Method.invoke(Method.java:606) > {noformat} > See DRILL-TBD. -- This message was sent by Atlassian JIRA (v6.3.4#6332)