drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jinfeng Ni (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-4686) Aggregation query over HBase table results in IllegalStateException: Failure while reading vector
Date Sat, 05 Aug 2017 00:01:25 GMT

    [ https://issues.apache.org/jira/browse/DRILL-4686?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16115140#comment-16115140
] 

Jinfeng Ni commented on DRILL-4686:
-----------------------------------

I was able to reproduce the problem on 1.11.0.  With the patch for DRILL-5466, the query seems
to run successfully consistently in multiple runs.

1. Prepare hbase table 
{code}
## hbase shell
create 'browser_action2', 'v', {SPLITS => ['0','1','2','3','4','5','6','7','8','9']}

put 'browser_action2', '1','v:e0', 'abc1';
put 'browser_action2', '2','v:e0', 'abc2';
put 'browser_action2', '3','v:e0', 'abc3';
put 'browser_action2', '4','v:e0', 'abc4';
put 'browser_action2', '5','v:e0', 'abc5';
put 'browser_action2', '6','v:e0', 'abc6';
put 'browser_action2', '7','v:e0', 'abc7';
put 'browser_action2', '8','v:e0', 'abc8';
put 'browser_action2', '9','v:e0', 'abc9';
put 'browser_action2', '10','v:e0', 'abc10';
{code}

2. Hit issue on 1.11.0 release.
{code}
## drill sqlline
select CONVERT_FROM(a.`v`.`e0`, 'UTF8') as k, count(a.`v`.`e0`) p from hbase.browser_action2
a where a.row_key > '0'  group by a.`v`.`e0`;
Error: SYSTEM ERROR: IllegalStateException: Failure while reading vector.  Expected vector
class of org.apache.drill.exec.vector.NullableIntVector but was holding vector class org.apache.drill.exec.vector.NullableVarBinaryVector,
field= $f0(VARBINARY:OPTIONAL) [$bits$(UINT1:REQUIRED), $f0(VARBINARY:OPTIONAL) [$offsets$(UINT4:REQUIRED)]]

Fragment 2:1
{code}

3. run successful with the patch of DRILL-5466.
{code}
select CONVERT_FROM(a.`v`.`e0`, 'UTF8') as k, count(a.`v`.`e0`) p from hbase.browser_action2
a where a.row_key > '0'  group by a.`v`.`e0`;
+--------+----+
|   k    | p  |
+--------+----+
| abc9   | 1  |
| abc7   | 1  |
| abc8   | 1  |
| abc2   | 1  |
| abc4   | 1  |
| abc3   | 1  |
| abc10  | 1  |
| abc1   | 1  |
| abc6   | 1  |
| abc5   | 1  |
+--------+----+
{code}

> Aggregation query over HBase table results in IllegalStateException: Failure while reading
vector
> -------------------------------------------------------------------------------------------------
>
>                 Key: DRILL-4686
>                 URL: https://issues.apache.org/jira/browse/DRILL-4686
>             Project: Apache Drill
>          Issue Type: Bug
>          Components: Query Planning & Optimization
>    Affects Versions: 1.7.0
>         Environment: 4 node cluster CentOS
>            Reporter: Khurram Faraaz
>
> Aggregation query over HBase table from Drill 1.7.0 returns IllegalStateException
> Drill version 1.7.0-SNAPSHOT,  commit ID : 09b26277
> {noformat}
> put 'browser_action2', '1','v:e0', 'abc1';
> put 'browser_action2', '2','v:e0', 'abc2';
> put 'browser_action2', '3','v:e0', 'abc3';
> put 'browser_action2', '4','v:e0', 'abc4';
> put 'browser_action2', '5','v:e0', 'abc5';
> put 'browser_action2', '6','v:e0', 'abc6';
> put 'browser_action2', '7','v:e0', 'abc7';
> put 'browser_action2', '8','v:e0', 'abc8';
> put 'browser_action2', '9','v:e0', 'abc9';
> put 'browser_action2', '10','v:e0', 'abc10';
> {noformat}
> {noformat}
> [root@centos-01 ~]# hbase shell
> HBase Shell; enter 'help<RETURN>' for list of supported commands.
> Type "exit<RETURN>" to leave the HBase Shell
> Version 1.1.1-mapr-1602-SNAPSHOT, r05ceb750d7ac9decac18e92650fedc0e86d85c7a, Mon Mar
28 18:32:45 UTC 2016
> Not all HBase shell commands are applicable to MapR tables.
> Consult MapR documentation for the list of supported commands.
> hbase(main):001:0> scan 'browser_action2'
> ROW                                COLUMN+CELL
>  1                                 column=v:e0, timestamp=1463589516782, value=abc1
>  10                                column=v:e0, timestamp=1463589516916, value=abc10
>  2                                 column=v:e0, timestamp=1463589516809, value=abc2
>  3                                 column=v:e0, timestamp=1463589516829, value=abc3
>  4                                 column=v:e0, timestamp=1463589516834, value=abc4
>  5                                 column=v:e0, timestamp=1463589516847, value=abc5
>  6                                 column=v:e0, timestamp=1463589516861, value=abc6
>  7                                 column=v:e0, timestamp=1463589516874, value=abc7
>  8                                 column=v:e0, timestamp=1463589516896, value=abc8
>  9                                 column=v:e0, timestamp=1463589516905, value=abc9
> 10 row(s) in 0.7970 seconds
> hbase(main):002:0>
> {noformat}
> {noformat}
> 0: jdbc:drill:schema=dfs.tmp> use hbase;
> +-------+------------------------------------+
> |  ok   |              summary               |
> +-------+------------------------------------+
> | true  | Default schema changed to [hbase]  |
> +-------+------------------------------------+
> 1 row selected (0.327 seconds)
> 0: jdbc:drill:schema=dfs.tmp> show tables;
> +---------------+---------------------+
> | TABLE_SCHEMA  |     TABLE_NAME      |
> +---------------+---------------------+
> | hbase         | browser_action2     |
> | hbase         | fewtypes_null       |
> | hbase         | fewtypes_null_hive  |
> | hbase         | hbase_student       |
> | hbase         | hbase_voter         |
> | hbase         | hbvoter             |
> | hbase         | hive_hbase          |
> | hbase         | student             |
> | hbase         | voter               |
> +---------------+---------------------+
> 9 rows selected (1.597 seconds)
> 0: jdbc:drill:schema=dfs.tmp> describe browser_action2;
> +--------------+------------+--------------+
> | COLUMN_NAME  | DATA_TYPE  | IS_NULLABLE  |
> +--------------+------------+--------------+
> | row_key      | ANY        | NO           |
> | v            | MAP        | NO           |
> +--------------+------------+--------------+
> 2 rows selected (1.669 seconds)
> 0: jdbc:drill:schema=dfs.tmp> select  CONVERT_FROM(a.`v`.`e0`, 'UTF8') as k from hbase.browser_action2
a;
> +--------+
> |   k    |
> +--------+
> | abc6   |
> | abc5   |
> | abc3   |
> | abc7   |
> | abc1   |
> | abc10  |
> | abc9   |
> | abc2   |
> | abc8   |
> | abc4   |
> +--------+
> 10 rows selected (0.924 seconds)
> 0: jdbc:drill:schema=dfs.tmp> select CONVERT_FROM(a.`v`.`e0`, 'UTF8') as k, count(a.`v`.`e0`)
p from hbase.browser_action2 a where a.row_key > '0'  group by a.`v`.`e0`;
> Error: SYSTEM ERROR: IllegalStateException: Failure while reading vector.  Expected vector
class of org.apache.drill.exec.vector.NullableIntVector but was holding vector class org.apache.drill.exec.vector.NullableVarBinaryVector,
field= $f0(VARBINARY:OPTIONAL)[$bits$(UINT1:REQUIRED), $f0(VARBINARY:OPTIONAL)[$offsets$(UINT4:REQUIRED)]]
> Fragment 2:3
> [Error Id: 5c0c8dc2-141b-4a2c-ba7e-008693fe4273 on centos-02.qa.lab:31010] (state=,code=0)
> 0: jdbc:drill:schema=dfs.tmp>
> {noformat}
> Explain plan for the above failing query
> {noformat}
> 0: jdbc:drill:schema=dfs.tmp> explain plan for select CONVERT_FROM(a.`v`.`e0`, 'UTF8')
as k, count(a.`v`.`e0`) p from hbase.browser_action2 a where a.row_key > '0'  group by
a.`v`.`e0`;
> +------+------+
> | text | json |
> +------+------+
> | 00-00    Screen
> 00-01      Project(k=[$0], p=[$1])
> 00-02        UnionExchange
> 01-01          Project(k=[CONVERT_FROMUTF8($0)], p=[$1])
> 01-02            HashAgg(group=[{0}], p=[$SUM0($1)])
> 01-03              Project($f0=[$0], p=[$1])
> 01-04                HashToRandomExchange(dist0=[[$0]])
> 02-01                  UnorderedMuxExchange
> 03-01                    Project($f0=[$0], p=[$1], E_X_P_R_H_A_S_H_F_I_E_L_D=[hash32AsDouble($0)])
> 03-02                      HashAgg(group=[{0}], p=[COUNT($0)])
> 03-03                        Project($f0=[ITEM($1, 'e0')])
> 03-04                          Scan(groupscan=[HBaseGroupScan [HBaseScanSpec=HBaseScanSpec
[tableName=browser_action2, startRow=0\x00, stopRow=, filter=null], columns=[`*`]]])
> {noformat}
> Java code used to create table in HBase
> {noformat}
> public class PutIntDataToHBase {
>     public static void main(String args[]) throws IOException {
>         Configuration conf = HBaseConfiguration.create();
>         conf.set("hbase.zookeeper.property.clientPort","5181");
>         HBaseAdmin admin = new HBaseAdmin(conf);
>         if (admin.tableExists("browser_action2")) {
>             admin.disableTable("browser_action2");
>             admin.deleteTable("browser_action2");
>         }
>         byte[][] SPLIT_KEYS = {{'0'},{'1'},{'2'},{'3'},{'4'},{'5'},{'6'},{'7'},{'8'},{'9'}};
>         HTableDescriptor tableDesc = new
>             HTableDescriptor(TableName.valueOf("browser_action2"));
>         tableDesc.addFamily(new HColumnDescriptor("v"));
>         admin.createTable(tableDesc,SPLIT_KEYS);
>     }
> }
> {noformat}
> Stack trace from drillbit.log
> {noformat}
> 2016-05-19 06:11:31,906 [28c2a6ec-c264-23a9-c1d3-d8d0d174ca57:frag:2:3] ERROR o.a.d.e.w.fragment.FragmentExecutor
- SYSTEM ERROR: IllegalStateException: Failure while reading vector.  Expected vector class
of org.apache.drill.exec.vector.NullableIntVector but was holding vector class org.apache.drill.exec.vector.NullableVarBinaryVector,
field= $f0(VARBINARY:OPTIONAL)[$bits$(UINT1:REQUIRED), $f0(VARBINARY:OPTIONAL)[$offsets$(UINT4:REQUIRED)]]
> Fragment 2:3
> [Error Id: 5c0c8dc2-141b-4a2c-ba7e-008693fe4273 on centos-02.qa.lab:31010]
> org.apache.drill.common.exceptions.UserException: SYSTEM ERROR: IllegalStateException:
Failure while reading vector.  Expected vector class of org.apache.drill.exec.vector.NullableIntVector
but was holding vector class org.apache.drill.exec.vector.NullableVarBinaryVector, field=
$f0(VARBINARY:OPTIONAL)[$bits$(UINT1:REQUIRED), $f0(VARBINARY:OPTIONAL)[$offsets$(UINT4:REQUIRED)]]
> Fragment 2:3
> [Error Id: 5c0c8dc2-141b-4a2c-ba7e-008693fe4273 on centos-02.qa.lab:31010]
>         at org.apache.drill.common.exceptions.UserException$Builder.build(UserException.java:543)
~[drill-common-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
>         at org.apache.drill.exec.work.fragment.FragmentExecutor.sendFinalState(FragmentExecutor.java:318)
[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
>         at org.apache.drill.exec.work.fragment.FragmentExecutor.cleanup(FragmentExecutor.java:185)
[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
>         at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:287)
[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
>         at org.apache.drill.common.SelfCleaningRunnable.run(SelfCleaningRunnable.java:38)
[drill-common-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_45]
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_45]
>         at java.lang.Thread.run(Thread.java:744) [na:1.7.0_45]
> Caused by: java.lang.IllegalStateException: Failure while reading vector.  Expected vector
class of org.apache.drill.exec.vector.NullableIntVector but was holding vector class org.apache.drill.exec.vector.NullableVarBinaryVector,
field= $f0(VARBINARY:OPTIONAL)[$bits$(UINT1:REQUIRED), $f0(VARBINARY:OPTIONAL)[$offsets$(UINT4:REQUIRED)]]
>         at org.apache.drill.exec.record.VectorContainer.getValueAccessorById(VectorContainer.java:290)
~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
>         at org.apache.drill.exec.record.RecordBatchLoader.getValueAccessorById(RecordBatchLoader.java:178)
~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
>         at org.apache.drill.exec.physical.impl.unorderedreceiver.UnorderedReceiverBatch.getValueAccessorById(UnorderedReceiverBatch.java:135)
~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
>         at org.apache.drill.exec.test.generated.PartitionerGen953$OutgoingRecordBatch.doSetup(PartitionerTemplate.java:64)
~[na:na]
>         at org.apache.drill.exec.test.generated.PartitionerGen953$OutgoingRecordBatch.initializeBatch(PartitionerTemplate.java:358)
~[na:na]
>         at org.apache.drill.exec.test.generated.PartitionerGen953.flushOutgoingBatches(PartitionerTemplate.java:163)
~[na:na]
>         at org.apache.drill.exec.physical.impl.partitionsender.PartitionerDecorator$FlushBatchesHandlingClass.execute(PartitionerDecorator.java:266)
~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
>         at org.apache.drill.exec.physical.impl.partitionsender.PartitionerDecorator.executeMethodLogic(PartitionerDecorator.java:138)
~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
>         at org.apache.drill.exec.physical.impl.partitionsender.PartitionerDecorator.flushOutgoingBatches(PartitionerDecorator.java:82)
~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
>         at org.apache.drill.exec.physical.impl.partitionsender.PartitionSenderRootExec.innerNext(PartitionSenderRootExec.java:183)
~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
>         at org.apache.drill.exec.physical.impl.BaseRootExec.next(BaseRootExec.java:94)
~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
>         at org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:257)
~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
>         at org.apache.drill.exec.work.fragment.FragmentExecutor$1.run(FragmentExecutor.java:251)
~[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
>         at java.security.AccessController.doPrivileged(Native Method) ~[na:1.7.0_45]
>         at javax.security.auth.Subject.doAs(Subject.java:415) ~[na:1.7.0_45]
>         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1595)
~[hadoop-common-2.7.0-mapr-1602.jar:na]
>         at org.apache.drill.exec.work.fragment.FragmentExecutor.run(FragmentExecutor.java:251)
[drill-java-exec-1.7.0-SNAPSHOT.jar:1.7.0-SNAPSHOT]
>         ... 4 common frames omitted
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message