drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rahul Challapalli (JIRA)" <j...@apache.org>
Subject [jira] [Created] (DRILL-4345) Hive Native Reader reporting wrong results for timestamp column in hive generated parquet file
Date Wed, 03 Feb 2016 00:16:40 GMT
Rahul Challapalli created DRILL-4345:
----------------------------------------

             Summary: Hive Native Reader reporting wrong results for timestamp column in hive
generated parquet file
                 Key: DRILL-4345
                 URL: https://issues.apache.org/jira/browse/DRILL-4345
             Project: Apache Drill
          Issue Type: Bug
          Components: Storage - Hive, Storage - Parquet
            Reporter: Rahul Challapalli
            Priority: Critical


git.commit.id.abbrev=1b96174

Below you can see different results returned from hive plugin and native reader for the same
table.

{code}
0: jdbc:drill:zk=10.10.100.190:5181> use hive;
+-------+-----------------------------------+
|  ok   |              summary              |
+-------+-----------------------------------+
| true  | Default schema changed to [hive]  |
+-------+-----------------------------------+
1 row selected (0.415 seconds)
0: jdbc:drill:zk=10.10.100.190:5181> select int_col, timestamp_col from hive1_fewtypes_null_parquet;
+----------+------------------------+
| int_col  |     timestamp_col      |
+----------+------------------------+
| 1        | null                   |
| null     | 1997-01-02 00:00:00.0  |
| 3        | null                   |
| 4        | null                   |
| 5        | 1997-02-10 17:32:00.0  |
| 6        | 1997-02-11 17:32:01.0  |
| 7        | 1997-02-12 17:32:01.0  |
| 8        | 1997-02-13 17:32:01.0  |
| 9        | null                   |
| 10       | 1997-02-15 17:32:01.0  |
| null     | 1997-02-16 17:32:01.0  |
| 12       | 1897-02-18 17:32:01.0  |
| 13       | 2002-02-14 17:32:01.0  |
| 14       | 1991-02-10 17:32:01.0  |
| 15       | 1900-02-16 17:32:01.0  |
| 16       | null                   |
| null     | 1897-02-16 17:32:01.0  |
| 18       | 1997-02-16 17:32:01.0  |
| null     | null                   |
| 20       | 1996-02-28 17:32:01.0  |
| null     | null                   |
+----------+------------------------+
21 rows selected (0.368 seconds)
0: jdbc:drill:zk=10.10.100.190:5181> alter session set `store.hive.optimize_scan_with_native_readers`
= true;
+-------+--------------------------------------------------------+
|  ok   |                        summary                         |
+-------+--------------------------------------------------------+
| true  | store.hive.optimize_scan_with_native_readers updated.  |
+-------+--------------------------------------------------------+
1 row selected (0.213 seconds)
0: jdbc:drill:zk=10.10.100.190:5181> select int_col, timestamp_col from hive1_fewtypes_null_parquet;
+----------+------------------------+
| int_col  |     timestamp_col      |
+----------+------------------------+
| 1        | null                   |
| null     | 1997-01-02 00:00:00.0  |
| 3        | 1997-02-10 17:32:00.0  |
| 4        | null                   |
| 5        | 1997-02-11 17:32:01.0  |
| 6        | 1997-02-12 17:32:01.0  |
| 7        | 1997-02-13 17:32:01.0  |
| 8        | 1997-02-15 17:32:01.0  |
| 9        | 1997-02-16 17:32:01.0  |
| 10       | 1900-02-16 17:32:01.0  |
| null     | 1897-02-16 17:32:01.0  |
| 12       | 1997-02-16 17:32:01.0  |
| 13       | 1996-02-28 17:32:01.0  |
| 14       | 1997-01-02 00:00:00.0  |
| 15       | 1997-01-02 00:00:00.0  |
| 16       | 1997-01-02 00:00:00.0  |
| null     | 1997-01-02 00:00:00.0  |
| 18       | 1997-01-02 00:00:00.0  |
| null     | 1997-01-02 00:00:00.0  |
| 20       | 1997-01-02 00:00:00.0  |
| null     | 1997-01-02 00:00:00.0  |
+----------+------------------------+
21 rows selected (0.352 seconds)
{code}

DDL for hive table :
{code}
create external table hive1_fewtypes_null_parquet (
      int_col int,
      bigint_col bigint,
      date_col string,
      time_col string,
      timestamp_col timestamp,
      interval_col string,
      varchar_col string,
      float_col float,
      double_col double,
      bool_col boolean
    )
stored as parquet
location '/drill/testdata/hive_storage/hive1_fewtypes_null';
{code}

Attached the underlying parquet file



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message