hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-12767) Implement table property to address Parquet int96 timestamp bug
Date Mon, 06 Feb 2017 19:12:41 GMT

    [ https://issues.apache.org/jira/browse/HIVE-12767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15854565#comment-15854565
] 

Hive QA commented on HIVE-12767:
--------------------------------



Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12851198/HIVE-12767.5.patch

{color:green}SUCCESS:{color} +1 due to 6 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 10224 tests executed
*Failed tests:*
{noformat}
TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out) (batchId=235)
TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=100)
	[skewjoinopt15.q,auto_join18.q,list_bucket_dml_2.q,input1_limit.q,load_dyn_part3.q,union_remove_14.q,auto_sortmerge_join_14.q,auto_sortmerge_join_15.q,union10.q,bucket_map_join_tez2.q,groupby5_map_skew.q,join_reorder.q,sample1.q,bucketmapjoin8.q,union34.q]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_external_time] (batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_int96_timestamp] (batchId=33)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys]
(batchId=159)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] (batchId=223)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23] (batchId=223)
org.apache.hadoop.hive.llap.daemon.impl.TestTaskExecutorService.testWaitQueuePreemption (batchId=278)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3396/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3396/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3396/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 8 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12851198 - PreCommit-HIVE-Build

> Implement table property to address Parquet int96 timestamp bug
> ---------------------------------------------------------------
>
>                 Key: HIVE-12767
>                 URL: https://issues.apache.org/jira/browse/HIVE-12767
>             Project: Hive
>          Issue Type: Bug
>    Affects Versions: 1.2.1, 2.0.0
>            Reporter: Sergio Peña
>            Assignee: Barna Zsombor Klara
>         Attachments: HIVE-12767.3.patch, HIVE-12767.4.patch, HIVE-12767.5.patch, TestNanoTimeUtils.java
>
>
> Parque timestamps using INT96 are not compatible with other tools, like Impala, due to
issues in Hive because it adjusts timezones values in a different way than Impala.
> To address such issues. a new table property (parquet.mr.int96.write.zone) must be used
in Hive that detects what timezone to use when writing and reading timestamps from Parquet.
> The following is the exit criteria for the fix:
> * Hive will read Parquet MR int96 timestamp data and adjust values using a time zone
from a table property, if set, or using the local time zone if it is absent. No adjustment
will be applied to data written by Impala.
> * Hive will write Parquet int96 timestamps using a time zone adjustment from the same
table property, if set, or using the local time zone if it is absent. This keeps the data
in the table consistent.
> * New tables created by Hive will set the table property to UTC if the global option
to set the property for new tables is enabled.
> ** Tables created using CREATE TABLE and CREATE TABLE LIKE FILE will not set the property
unless the global setting to do so is enabled.
> ** Tables created using CREATE TABLE LIKE <OTHER TABLE> will copy the property
of the table that is copied.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message