drill-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Khurram Faraaz (JIRA)" <j...@apache.org>
Subject [jira] [Created] (DRILL-3849) IOB Exception : ORDER BY ROW_KEY over date_epoch_be encoded data
Date Tue, 29 Sep 2015 04:55:04 GMT
Khurram Faraaz created DRILL-3849:
-------------------------------------

             Summary: IOB Exception : ORDER BY ROW_KEY over date_epoch_be encoded data
                 Key: DRILL-3849
                 URL: https://issues.apache.org/jira/browse/DRILL-3849
             Project: Apache Drill
          Issue Type: Bug
          Components: Execution - Flow
    Affects Versions: 1.2.0
         Environment: 4 node cluster CentOS
            Reporter: Khurram Faraaz
            Assignee: Smidth Panchamia


Order by ROW_KEY that has DATE type data results in IOB Exception where queried from Drill.
Note that there is a duplicate in ROW_KEY.

{code}
Drill master commitID : b9afcf8f
[root@centos-01 rowkey_filter_pushdown]# hbase version
2015-09-29 04:50:04,510 INFO  [main] util.VersionInfo: HBase 0.98.12-mapr-1506-SNAPSHOT
2015-09-29 04:50:04,511 INFO  [main] util.VersionInfo: Subversion git://5e8a86c5195a/root/opensource/mapr-hbase-0.98/dl/mapr-hbase-0.98
-r 34eede8b74911af09661867c07440888b648c3ba
2015-09-29 04:50:04,511 INFO  [main] util.VersionInfo: Compiled by root on Tue Jul 21 00:24:58
UTC 2015
{code}

A regular select without order by on row_key gives correct results.

{code}
0: jdbc:drill:schema=dfs.tmp> SELECT CONVERT_FROM(BYTE_SUBSTR(row_key,1,8),'date_epoch_be')
AS RK, CONVERT_FROM(T.`colfam1`.`qual1`, 'UTF8') AS cf1 from dt_Tbl T;
+-------------+-------+
|     RK      |  cf1  |
+-------------+-------+
| 2014-08-25  | val1  |
| 2015-09-26  | val5  |
| 2015-09-28  | val4  |
| 2015-09-29  | val3  |
| 2015-09-29  | val2  |
+-------------+-------+
5 rows selected (0.79 seconds)
{code}

However when encoding 'date_epoch_be' is used in the query and ORDER BY is used on ROW_KEY
we see the IOB Exception.

{code}
0: jdbc:drill:schema=dfs.tmp> SELECT CONVERT_FROM(BYTE_SUBSTR(row_key,1,8),'date_epoch_be')
AS RK, CONVERT_FROM(T.`colfam1`.`qual1`, 'UTF8') AS cf1 from dt_Tbl T ORDER BY ROW_KEY;
Error: SYSTEM ERROR: IndexOutOfBoundsException: DrillBuf(ridx: 0, widx: 133, cap: 133/133,
unwrapped: SlicedByteBuf(ridx: 0, widx: 133, cap: 133/133, unwrapped: UnsafeDirectLittleEndian(PooledUnsafeDirectByteBuf(ridx:
0, widx: 0, cap: 392/392)))).slice(89, 64)

Fragment 2:0

[Error Id: bca3c7dd-e5cc-4a5a-8616-231fa73c73c9 on centos-02.qa.lab:31010] (state=,code=0)
0: jdbc:drill:schema=dfs.tmp>
{code}

Stack trace from drillbit.log for the failing query

{code}
2015-09-29 04:33:04,295 [29f5eb81-5696-49fd-b86e-788ed103fec9:frag:2:0] ERROR o.a.d.e.w.fragment.FragmentExecutor
- SYSTEM ERROR: IndexOutOfBoundsException: DrillBuf(ridx: 0, widx: 133, cap: 133/133, unwrapped:
SlicedByteBuf(ridx: 0, widx: 133, cap: 133/133, unwrapped: UnsafeDirectLittleEndian(PooledUnsafeDirectByteBuf(ridx:
0, widx: 0, cap: 392/392)))).slice(89, 64)

Fragment 2:0

[Error Id: bca3c7dd-e5cc-4a5a-8616-231fa73c73c9 on centos-02.qa.lab:31010]
org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: IndexOutOfBoundsException:
DrillBuf(ridx: 0, widx: 133, cap: 133/133, unwrapped: SlicedByteBuf(ridx: 0, widx: 133, cap:
133/133, unwrapped: UnsafeDirectLittleEndian(PooledUnsafeDirectByteBuf(ridx: 0, widx: 0, cap:
392/392)))).slice(89, 64)

Fragment 2:0

[Error Id: bca3c7dd-e5cc-4a5a-8616-231fa73c73c9 on centos-02.qa.lab:31010]
        at org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:524)
~[drill-common-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
        at org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:323)
[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
        at org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:178)
[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
        at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:292)
[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
        at org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
[drill-common-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_45]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_45]
        at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
Caused by: java.lang.IndexOutOfBoundsException: DrillBuf(ridx: 0, widx: 133, cap: 133/133,
unwrapped: SlicedByteBuf(ridx: 0, widx: 133, cap: 133/133, unwrapped: UnsafeDirectLittleEndian(PooledUnsafeDirectByteBuf(ridx:
0, widx: 0, cap: 392/392)))).slice(89, 64)
        at io.netty.buffer.DrillBuf.<init>(DrillBuf.java:115) ~[drill-java-exec-1.2.0-SNAPSHOT.jar:4.0.27.Final]
        at io.netty.buffer.DrillBuf.<init>(DrillBuf.java:105) ~[drill-java-exec-1.2.0-SNAPSHOT.jar:4.0.27.Final]
        at io.netty.buffer.DrillBuf.slice(DrillBuf.java:348) ~[drill-java-exec-1.2.0-SNAPSHOT.jar:4.0.27.Final]
        at org.apache.drill.exec.record.RecordBatchLoader.load(RecordBatchLoader.java:102)
~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
        at org.apache.drill.exec.physical.impl.unorderedreceiver.UnorderedReceiverBatch.next(UnorderedReceiverBatch.java:186)
~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
        at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:83) ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
        at org.apache.drill.exec.physical.impl.partitionsender.PartitionSenderRootExec.innerNext(PartitionSenderRootExec.java:145)
~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
        at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:73) ~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
        at org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:258)
~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
        at org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:252)
~[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
        at java.security.AccessController.doPrivileged(Native Method) ~[na:1.7.0_45]
        at javax.security.auth.Subject.doAs(Subject.java:415) ~[na:1.7.0_45]
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1566)
~[hadoop-common-2.5.1-mapr-1503.jar:na]
        at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:252)
[drill-java-exec-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
        ... 4 common frames omitted
{code}

Snippet used to insert data into base table

{code}
Date dt1= new Date(1408924800000L);
        byte[] rowKey1 = ByteBuffer.allocate(8) .putLong(dt1.getTime()).array();
        // getTime() will convert date into a `long'
        Put p = new Put(rowKey1);
        p.add(Bytes.toBytes("colfam1"),Bytes.toBytes("qual1"),Bytes.toBytes("val1"));
        table.put(p);

        Date dt2= new Date(1443500086094L);
        byte[] rowKey2 = ByteBuffer.allocate(8) .putLong(dt2.getTime()).array();
        // getTime() will convert date into a `long'
        p = new Put(rowKey2);
        p.add(Bytes.toBytes("colfam1"),Bytes.toBytes("qual1"),Bytes.toBytes("val2"));
        table.put(p);

        Date dt3= new Date(1443491446094L);
        byte[] rowKey3 = ByteBuffer.allocate(8) .putLong(dt3.getTime()).array();
        // getTime() will convert date into a `long'
        p = new Put(rowKey3);
        p.add(Bytes.toBytes("colfam1"),Bytes.toBytes("qual1"),Bytes.toBytes("val3"));
        table.put(p);

        Date dt4= new Date(1443413686094L);
        byte[] rowKey4 = ByteBuffer.allocate(8) .putLong(dt4.getTime()).array();
        // getTime() will convert date into a `long'
        p = new Put(rowKey4);
        p.add(Bytes.toBytes("colfam1"),Bytes.toBytes("qual1"),Bytes.toBytes("val4"));
        table.put(p);

        Date dt5= new Date(1443240886094L);
        byte[] rowKey5 = ByteBuffer.allocate(8) .putLong(dt5.getTime()).array();
        // getTime() will convert date into a `long'
        p = new Put(rowKey5);
        p.add(Bytes.toBytes("colfam1"),Bytes.toBytes("qual1"),Bytes.toBytes("val5"));
        table.put(p);
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message