drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Deneche A. Hakim (JIRA)" <j...@apache.org>
Subject [jira] [Created] (DRILL-2151) VariableLengthVector.copyFromSafe() unnecessary sets the offsetVector
Date Tue, 03 Feb 2015 21:50:34 GMT
Deneche A. Hakim created DRILL-2151:
---------------------------------------

             Summary: VariableLengthVector.copyFromSafe() unnecessary sets the offsetVector
                 Key: DRILL-2151
                 URL: https://issues.apache.org/jira/browse/DRILL-2151
             Project: Apache Drill
          Issue Type: Bug
          Components: Execution - Data Types
            Reporter: Deneche A. Hakim
            Assignee: Deneche A. Hakim
            Priority: Critical


The following method defined in {{VariableLengthVectors.java}}:

{code}
public boolean copyFromSafe(int fromIndex, int thisIndex, ${minor.class}Vector from){
    int start = from.offsetVector.getAccessor().get(fromIndex);
    int end =   from.offsetVector.getAccessor().get(fromIndex+1);
    int len = end - start;
    
    int outputStart = offsetVector.data.get${(minor.javaType!type.javaType)?cap_first}(thisIndex
* ${type.width});
    
    if(data.capacity() < outputStart + len) {
        reAlloc();
    }

    offsetVector.getMutator().setSafe(thisIndex + 1, outputStart + len);

    from.data.getBytes(start, data, outputStart, len);
    
    offsetVector.getMutator().setSafe( (thisIndex+1) * ${type.width}, outputStart + len);

    return true;
}
{code}

at the very end, it multiplies {{thisIndex+1}} by the type width, but the mutator will also
multiply the index by the type width. This line should be removed because the offset is already
set correctly in the same method.

One possible query where this actually causes problem is the one defined in DRILL-1588 (on
TPCH SF 1):
{noformat}
// set slice target to 1 to ensure exchanges are used
0: jdbc:drill:zk=local> alter session set `planner.slice_target` = 1;
+------------+------------+
|     ok     |  summary   |
+------------+------------+
| true       | planner.slice_target updated. |
+------------+------------+
1 row selected (0.1 seconds)
0: jdbc:drill:zk=local> select * from lineitem l left outer join orders o on (l.l_orderkey
= o.o_orderkey) left outer join  customer c on (l.l_orderkey = c.c_custkey) left outer join
 nation n on (l.l_partkey = n.n_nationkey) left outer join  region r  on (l.l_suppkey = r.r_regionkey)
limit 10;
{noformat}

Here is a copy of the execution in the logs:
{noformat}
java.lang.IndexOutOfBoundsException: index: 60496, length: 4 (expected: range(0, 32768))
at io.netty.buffer.DrillBuf.checkIndexD(DrillBuf.java:156) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
at io.netty.buffer.DrillBuf.chk(DrillBuf.java:178) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
at io.netty.buffer.DrillBuf.setInt(DrillBuf.java:473) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:4.0.24.Final]
at org.apache.drill.exec.vector.UInt4Vector$Mutator.set(UInt4Vector.java:359) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at org.apache.drill.exec.vector.UInt4Vector$Mutator.setSafe(UInt4Vector.java:366) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at org.apache.drill.exec.vector.VarBinaryVector.copyFromSafe(VarBinaryVector.java:259) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at org.apache.drill.exec.vector.NullableVarBinaryVector.copyFromSafe(NullableVarBinaryVector.java:301)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at org.apache.drill.exec.test.generated.HashJoinProbeGen28.projectBuildRecord(HashJoinProbeTemplate.java:718)
~[na:na]
at org.apache.drill.exec.test.generated.HashJoinProbeGen28.executeProbePhase(HashJoinProbeTemplate.java:173)
~[na:na]
at org.apache.drill.exec.test.generated.HashJoinProbeGen28.probeAndProject(HashJoinProbeTemplate.java:223)
~[na:na]
at org.apache.drill.exec.physical.impl.join.HashJoinBatch.innerNext(HashJoinBatch.java:227)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at org.apache.drill.exec.record.AbstractRecordBatch.next(AbstractRecordBatch.java:142) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at org.apache.drill.exec.physical.impl.validate.IteratorValidatorBatchIterator.next(IteratorValidatorBatchIterator.java:118)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:67) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at org.apache.drill.exec.physical.impl.partitionsender.PartitionSenderRootExec.innerNext(PartitionSenderRootExec.java:133)
~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:57) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:110) ~[drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at org.apache.drill.exec.work.WorkManager$RunnableWrapper.run(WorkManager.java:254) [drill-java-exec-0.8.0-SNAPSHOT-rebuffed.jar:0.8.0-SNAPSHOT]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_71]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_71]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message