drill-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (DRILL-4373) Drill and Hive have incompatible timestamp representations in parquet
Date Thu, 13 Oct 2016 18:47:20 GMT

    [ https://issues.apache.org/jira/browse/DRILL-4373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15572829#comment-15572829
] 

ASF GitHub Bot commented on DRILL-4373:
---------------------------------------

Github user parthchandra commented on a diff in the pull request:

    https://github.com/apache/drill/pull/600#discussion_r83284350
  
    --- Diff: exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/writer/TestParquetWriter.java
---
    @@ -754,15 +764,45 @@ public void testImpalaParquetVarBinary_DictChange() throws Exception
{
         compareParquetReadersColumnar("field_impala_ts", "cp.`parquet/int96_dict_change.parquet`");
       }
     
    +  @Test
    +  public void testImpalaParquetBinaryTimeStamp_DictChange() throws Exception {
    +    try {
    +      test("alter session set %s = true", ExecConstants.PARQUET_READER_INT96_AS_TIMESTAMP);
    +      compareParquetReadersColumnar("field_impala_ts", "cp.`parquet/int96_dict_change.parquet`");
    --- End diff --
    
    This is not a good enough test. Both the baseline and test case queries will use the getDateTimeValueFromBinary
method and if there is a bug in that method, the test will still pass as both will produce
the same incorrect value. Better to compare with the actual baseline value in the file.


> Drill and Hive have incompatible timestamp representations in parquet
> ---------------------------------------------------------------------
>
>                 Key: DRILL-4373
>                 URL: https://issues.apache.org/jira/browse/DRILL-4373
>             Project: Apache Drill
>          Issue Type: Improvement
>          Components: Storage - Hive, Storage - Parquet
>    Affects Versions: 1.8.0
>            Reporter: Rahul Challapalli
>            Assignee: Karthikeyan Manivannan
>              Labels: doc-impacting
>             Fix For: 1.9.0
>
>
> git.commit.id.abbrev=83d460c
> I created a parquet file with a timestamp type using Drill. Now if I define a hive table
on top of the parquet file and use "timestamp" as the column type, drill fails to read the
hive table through the hive storage plugin



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message