drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vitalii Diravka (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-3290) Hive Storage : Add support for Hive complex types
Date Wed, 31 Aug 2016 11:09:20 GMT

    [ https://issues.apache.org/jira/browse/DRILL-3290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15451933#comment-15451933
] 

Vitalii Diravka commented on DRILL-3290:
----------------------------------------

Use case to reproduce it:
{code}CREATE TABLE `complex`(
   `col1` ARRAY<STRING>, 
   `col2` MAP<INT,STRING>, 
   `col3` STRUCT<c1:SMALLINT,c2:VARCHAR(30)>,
   `col4` UNIONTYPE<INT, BOOLEAN, STRING>)
ROW FORMAT DELIMITED 
  FIELDS TERMINATED BY ',' 
  COLLECTION ITEMS TERMINATED BY '&' 
  MAP KEYS TERMINATED BY '#'
  LINES TERMINATED BY '\n' STORED AS TEXTFILE;
{code}
Create a csv file:
{code}arr1&arr2,101#map1&102#map2,11&varchar_1,0&3.141459
arr3&arr4,103#map3&104#map4,12&varchar_2,1&true
arr3&arr4,103#map3&104#map4,12&varchar_2,2&Time to go home{code}

Load this data in hive table:
{code}LOAD DATA LOCAL INPATH '/home/dev/complex_data.csv' into table complex;{code}
Note: Assuming file is located at /home/dev/complex_data.csv


Result from hive:
{code}hive> select * from complex;
OK
["arr1","arr2"]	{101:"map1",102:"map2"}	{"c1":11,"c2":"varchar_1"}	{0:3}
["arr3","arr4"]	{103:"map3",104:"map4"}	{"c1":12,"c2":"varchar_2"}	{1:true}
["arr3","arr4"]	{103:"map3",104:"map4"}	{"c1":12,"c2":"varchar_2"}	{2:"Time to go home"}
Time taken: 0.065 seconds, Fetched: 3 row(s){code}
Result from dril:
{code}0: jdbc:drill:> use hive;
+-------+-----------------------------------+
|  ok   |              summary              |
+-------+-----------------------------------+
| true  | Default schema changed to [hive]  |
+-------+-----------------------------------+
1 row selected (0.252 seconds)
0: jdbc:drill:> select * from complex;
Error: SYSTEM ERROR: IndexOutOfBoundsException: index (4) must be less than size (4)


[Error Id: c6701738-0bd9-4702-a936-73f8a60f34fe on node1:31010] (state=,code=0)
0: jdbc:drill:> select col1 from complex;
Error: UNSUPPORTED_OPERATION ERROR: Unsupported Hive data type LIST. 
Following Hive data types are supported in Drill for querying: BOOLEAN, TINYINT, SMALLINT,
INT, BIGINT, FLOAT, DOUBLE, DATE, TIMESTAMP, BINARY, DECIMAL, STRING, and VARCHAR

Fragment 0:0

[Error Id: 824c9d6d-be1c-4707-8ff1-e8db03141f07 on node1:31010] (state=,code=0)
0: jdbc:drill:> select col2 from complex;
Error: UNSUPPORTED_OPERATION ERROR: Unsupported Hive data type MAP. 
Following Hive data types are supported in Drill for querying: BOOLEAN, TINYINT, SMALLINT,
INT, BIGINT, FLOAT, DOUBLE, DATE, TIMESTAMP, BINARY, DECIMAL, STRING, and VARCHAR

Fragment 0:0

[Error Id: 77f146fb-6102-43e0-bb39-3262e50d6357 on node1:31010] (state=,code=0)
0: jdbc:drill:> select col3 from complex;
Error: SYSTEM ERROR: IndexOutOfBoundsException: index (1) must be less than size (1)


[Error Id: e8a46ec1-b8d2-4fea-9eba-0cab20bd766e on node1:31010] (state=,code=0)
0: jdbc:drill:> select col4 from complex;
Error: UNSUPPORTED_OPERATION ERROR: Unsupported Hive data type UNION. 
Following Hive data types are supported in Drill for querying: BOOLEAN, TINYINT, SMALLINT,
INT, BIGINT, FLOAT, DOUBLE, DATE, TIMESTAMP, BINARY, DECIMAL, STRING, and VARCHAR

Fragment 0:0

[Error Id: 44d50ae2-21fc-44a0-924a-cc2a0a7c6cc1 on node1:31010] (state=,code=0){code}

> Hive Storage : Add support for Hive complex types
> -------------------------------------------------
>
>                 Key: DRILL-3290
>                 URL: https://issues.apache.org/jira/browse/DRILL-3290
>             Project: Apache Drill
>          Issue Type: Improvement
>          Components: Functions - Hive, Storage - Hive
>            Reporter: Rahul Challapalli
>            Assignee: Vitalii Diravka
>             Fix For: Future
>
>
> Improve the hive storage plugin to add support for complex types in hive. Below are the
complex types hive supports
> {code}
> ARRAY<data_type>
>  MAP<primitive_type, data_type>
> STRUCT<col_name : data_type [COMMENT col_comment], ...>
> UNIONTYPE<data_type, data_type, ...>
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message