Mailing-List: contact issues-help@drill.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@drill.apache.org
Date: Tue, 18 Oct 2016 15:04:58 +0000 (UTC)
From: "ASF GitHub Bot (JIRA)" <jira@apache.org>
To: issues@drill.apache.org
Message-ID: <JIRA.12937659.1454968250000.17147.1476803098580@Atlassian.JIRA>
In-Reply-To: <JIRA.12937659.1454968250000@Atlassian.JIRA>
References: <JIRA.12937659.1454968250000@Atlassian.JIRA> <JIRA.12937659.1454968250108@arcas>
Subject: [jira] [Commented] (DRILL-4373) Drill and Hive have incompatible
 timestamp representations in parquet
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Tue, 18 Oct 2016 15:05:05 -0000


    [ https://issues.apache.org/jira/browse/DRILL-4373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15585691#comment-15585691 ] 

ASF GitHub Bot commented on DRILL-4373:
---------------------------------------

Github user vdiravka commented on a diff in the pull request:

    https://github.com/apache/drill/pull/600#discussion_r83852721
  
    --- Diff: exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/writer/TestParquetWriter.java ---
    @@ -899,18 +883,21 @@ public void testLastPageOneNull() throws Exception {
             "cp.`parquet/last_page_one_null.parquet`");
       }
     
    -  private void compareParquetInt96Converters(String newInt96ConverterQuery,
    -      String oldInt96ConverterAndConvertFromFunctionQuery) throws Exception {
    -    testBuilder()
    -        .ordered()
    -        .sqlQuery(newInt96ConverterQuery)
    -        .optionSettingQueriesForTestQuery(
    -            "alter session set `%s` = true", ExecConstants.PARQUET_READER_INT96_AS_TIMESTAMP)
    -        .sqlBaselineQuery(oldInt96ConverterAndConvertFromFunctionQuery)
    -        .optionSettingQueriesForBaseline(
    -            "alter session set `%s` = false", ExecConstants.PARQUET_READER_INT96_AS_TIMESTAMP)
    -        .build()
    -        .run();
    +  private void compareParquetInt96Converters(String selection, String table) throws Exception {
    +    try {
    --- End diff --
    
    I refactored my helped method with more clear code.


> Drill and Hive have incompatible timestamp representations in parquet
> ---------------------------------------------------------------------
>
>                 Key: DRILL-4373
>                 URL: https://issues.apache.org/jira/browse/DRILL-4373
>             Project: Apache Drill
>          Issue Type: Improvement
>          Components: Storage - Hive, Storage - Parquet
>    Affects Versions: 1.8.0
>            Reporter: Rahul Challapalli
>            Assignee: Karthikeyan Manivannan
>              Labels: doc-impacting
>             Fix For: 1.9.0
>
>
> git.commit.id.abbrev=83d460c
> I created a parquet file with a timestamp type using Drill. Now if I define a hive table on top of the parquet file and use "timestamp" as the column type, drill fails to read the hive table through the hive storage plugin
> Implementation: 
> Added int96 to timestamp converter for both parquet readers and controling it by system / session option "store.parquet.int96_as_timestamp".
> The value of the option is false by default for the proper work of the old query scripts with the "convert_from TIMESTAMP_IMPALA" function.
> When the option is true using of that function is unnesessary and can lead to the query fail.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)