Return-Path: <issues-return-57416-archive-asf-public=cust-asf.ponee.io@hive.apache.org>
X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io
Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io
Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183])
	by cust-asf2.ponee.io (Postfix) with ESMTP id 99751200BC5
	for <archive-asf-public-internal@cust-asf2.ponee.io>; Mon,  7 Nov 2016 10:35:00 +0100 (CET)
Received: by cust-asf.ponee.io (Postfix)
	id 981A0160AEB; Mon,  7 Nov 2016 09:35:00 +0000 (UTC)
Delivered-To: archive-asf-public@cust-asf.ponee.io
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by cust-asf.ponee.io (Postfix) with SMTP id E339F160AF9
	for <archive-asf-public@cust-asf.ponee.io>; Mon,  7 Nov 2016 10:34:59 +0100 (CET)
Received: (qmail 19780 invoked by uid 500); 7 Nov 2016 09:34:59 -0000
Mailing-List: contact issues-help@hive.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:issues-help@hive.apache.org>
List-Unsubscribe: <mailto:issues-unsubscribe@hive.apache.org>
List-Post: <mailto:issues@hive.apache.org>
List-Id: <issues.hive.apache.org>
Reply-To: dev@hive.apache.org
Delivered-To: mailing list issues@hive.apache.org
Received: (qmail 19755 invoked by uid 99); 7 Nov 2016 09:34:59 -0000
Received: from arcas.apache.org (HELO arcas) (140.211.11.28)
    by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 07 Nov 2016 09:34:59 +0000
Received: from arcas.apache.org (localhost [127.0.0.1])
	by arcas (Postfix) with ESMTP id 03C2C2C0D55
	for <issues@hive.apache.org>; Mon,  7 Nov 2016 09:34:59 +0000 (UTC)
Date: Mon, 7 Nov 2016 09:34:59 +0000 (UTC)
From: "Vaibhav Gumashta (JIRA)" <jira@apache.org>
To: issues@hive.apache.org
Message-ID: <JIRA.13015740.1477558640000.207679.1478511299012@Atlassian.JIRA>
In-Reply-To: <JIRA.13015740.1477558640000@Atlassian.JIRA>
References: <JIRA.13015740.1477558640000@Atlassian.JIRA> <JIRA.13015740.1477558640012@arcas>
Subject: [jira] [Updated] (HIVE-15082) Hive-1.2 cannot read data from
 complex data types with TIMESTAMP column, stored in Parquet
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394
archived-at: Mon, 07 Nov 2016 09:35:00 -0000


     [ https://issues.apache.org/jira/browse/HIVE-15082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Vaibhav Gumashta updated HIVE-15082:
------------------------------------
    Priority: Blocker  (was: Major)

> Hive-1.2 cannot read data from complex data types with TIMESTAMP column, stored in Parquet
> ------------------------------------------------------------------------------------------
>
>                 Key: HIVE-15082
>                 URL: https://issues.apache.org/jira/browse/HIVE-15082
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: 1.2.0
>            Reporter: Oleksiy Sayankin
>            Assignee: Oleksiy Sayankin
>            Priority: Blocker
>         Attachments: HIVE-15082-branch-1.2.patch, HIVE-15082-branch-1.patch
>
>
> *STEP 1. Create test data*
> {code:sql}
> select * from dual;
> {code}
> *EXPECTED RESULT:*
> {noformat}
> Pretty_UnIQUe_StrinG
> {noformat}
> {code:sql}
> create table test_parquet1(login timestamp) stored as parquet;
> insert overwrite table test_parquet1 select from_unixtime(unix_timestamp()) from dual;
> select * from test_parquet1 limit 1;
> {code}
> *EXPECTED RESULT:*
> No exceptions. Current timestamp as result.
> {noformat}
> 2016-10-27 10:58:19
> {noformat}
> *STEP 2. Store timestamp in array in parquet file*
> {code:sql}
> create table test_parquet2(x array<timestamp>) stored as parquet;
> insert overwrite table test_parquet2 select array(login) from test_parquet1;
> select * from test_parquet2;
> {code}
> *EXPECTED RESULT:*
> No exceptions. Current timestamp in brackets as result.
> {noformat}
> ["2016-10-27 10:58:19"]
> {noformat}
> *ACTUAL RESULT:*
> {noformat}
> ERROR [main]: CliDriver (SessionState.java:printError(963)) - Failed with exception java.io.IOException:parquet.io.ParquetDecodingException: Can not read value at 0 in block -1 in file hdfs:///user/hive/warehouse/test_parquet2/000000_0
> java.io.IOException: parquet.io.ParquetDecodingException: Can not read value at 0 in block -1 in file hdfs:///user/hive/warehouse/test_parquet2/000000_0
> {noformat}
> *ROOT-CAUSE:*
> Incorrect initialization of {{metadata}} {{HashMap}} causes that it has {{null}} value in enumeration {{org.apache.hadoop.hive.ql.io.parquet.convert.ETypeConverter}} when executing following line:
> {code:java}
>   boolean skipConversion = Boolean.valueOf(metadata.get(HiveConf.ConfVars.HIVE_PARQUET_TIMESTAMP_SKIP_CONVERSION.varname));
> {code}
> in element {{ETIMESTAMP_CONVERTER}}.
> JVM throws NPE and parquet library can not read data from file and throws 
> {noformat}
> java.io.IOException:parquet.io.ParquetDecodingException: Can not read value at 0 in block -1 in file hdfs:///user/hive/warehouse/test_parquet2/000000_0
> {noformat}
> for its turn.
> *SOLUTION:*
> Perform initialization in separate method to skip overriding it with {{null}} value in block of code
> {code:java}
>   if (parent != null) {
>      setMetadata(parent.getMetadata());
>   }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)