Return-Path: X-Original-To: apmail-drill-issues-archive@minotaur.apache.org Delivered-To: apmail-drill-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 3659D17D46 for ; Tue, 3 Feb 2015 21:55:34 +0000 (UTC) Received: (qmail 3468 invoked by uid 500); 3 Feb 2015 21:55:35 -0000 Delivered-To: apmail-drill-issues-archive@drill.apache.org Received: (qmail 3439 invoked by uid 500); 3 Feb 2015 21:55:35 -0000 Mailing-List: contact issues-help@drill.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@drill.apache.org Delivered-To: mailing list issues@drill.apache.org Received: (qmail 3425 invoked by uid 99); 3 Feb 2015 21:55:35 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 03 Feb 2015 21:55:35 +0000 Date: Tue, 3 Feb 2015 21:55:34 +0000 (UTC) From: "Chris Westin (JIRA)" To: issues@drill.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (DRILL-2151) VariableLengthVector.copyFromSafe() unnecessary sets the offsetVector MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/DRILL-2151?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14304085#comment-14304085 ] Chris Westin commented on DRILL-2151: ------------------------------------- Another minor improvement would be to only call from.offsetVector.getAccessor() once and to re-use the result. It looks like there are other similar repeated expressions as well. > VariableLengthVector.copyFromSafe() unnecessary sets the offsetVector > --------------------------------------------------------------------- > > Key: DRILL-2151 > URL: https://issues.apache.org/jira/browse/DRILL-2151 > Project: Apache Drill > Issue Type: Bug > Components: Execution - Data Types > Reporter: Deneche A. Hakim > Assignee: Deneche A. Hakim > Priority: Critical > Attachments: DRILL-2151.1.patch.txt > > > The following method defined in {{VariableLengthVectors.java}}: > {code} > public boolean copyFromSafe(int fromIndex, int thisIndex, ${minor.class}Vector from){ > int start = from.offsetVector.getAccessor().get(fromIndex); > int end = from.offsetVector.getAccessor().get(fromIndex+1); > int len = end - start; > > int outputStart = offsetVector.data.get${(minor.javaType!type.javaType)?cap_first}(thisIndex * ${type.width}); > > if(data.capacity() < outputStart + len) { > reAlloc(); > } > offsetVector.getMutator().setSafe(thisIndex + 1, outputStart + len); > from.data.getBytes(start, data, outputStart, len); > > offsetVector.getMutator().setSafe( (thisIndex+1) * ${type.width}, outputStart + len); > return true; > } > {code} > at the very end, it multiplies {{thisIndex+1}} by the type width, but the mutator will also multiply the index by the type width. This line should be removed because the offset is already set correctly in the same method. > One possible query where this actually causes problem is the one defined in DRILL-1588 (on TPCH SF 1): > {noformat} > // set slice target to 1 to ensure exchanges are used > 0: jdbc:drill:zk=local> alter session set `planner.slice_target` = 1; > +------------+------------+ > | ok | summary | > +------------+------------+ > | true | planner.slice_target updated. | > +------------+------------+ > 1 row selected (0.1 seconds) > 0: jdbc:drill:zk=local> select * from lineitem l left outer join orders o on (l.l_orderkey = o.o_orderkey) left outer join customer c on (l.l_orderkey = c.c_custkey) left outer join nation n on (l.l_partkey = n.n_nationkey) left outer join region r on (l.l_suppkey = r.r_regionkey) limit 10; > {noformat} > Here is a copy of the execution in the logs: > {noformat} > java.lang.IndexOutOfBoundsException: index: 60496, length: 4 (expected: range(0, 32768)) > at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:156) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final] > at io.netty.buffer.DrillBuf.chk(DrillBuf.java:178) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final] > at io.netty.buffer.DrillBuf.setInt(DrillBuf.java:473) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final] > at org.apache.drill.exec.vector.UInt4Vector$Mutator.set(UInt4Vector.java:359) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT] > at org.apache.drill.exec.vector.UInt4Vector$Mutator.setSafe(UInt4Vector.java:366) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT] > at org.apache.drill.exec.vector.VarBinaryVector.copyFromSafe(VarBinaryVector.java:259) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT] > at org.apache.drill.exec.vector.NullableVarBinaryVector.copyFromSafe(NullableVarBinaryVector.java:301) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT] > at org.apache.drill.exec.test.generated.HashJoinProbeGen28.projectBuildRecord(HashJoinProbeTemplate.java:718) ~[na:na] > at org.apache.drill.exec.test.generated.HashJoinProbeGen28.executeProbePhase(HashJoinProbeTemplate.java:173) ~[na:na] > at org.apache.drill.exec.test.generated.HashJoinProbeGen28.probeAndProject(HashJoinProbeTemplate.java:223) ~[na:na] > at org.apache.drill.exec.physical.impl.join.HashJoinBatch.innerNext(HashJoinBatch.java:227) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT] > at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT] > at org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:118) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT] > at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:67) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT] > at org.apache.drill.exec.physical.impl.partitionsender.PartitionSenderRootExec.innerNext(PartitionSenderRootExec.java:133) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT] > at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:57) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT] > at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:110) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT] > at org.apache.drill.exec.work.WorkManager$RunnableWrapper.run(WorkManager.java:254) [drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT] > at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_71] > at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_71] > at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71] > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)