hive-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hive QA (Jira)" <j...@apache.org>
Subject [jira] [Commented] (HIVE-22476) Hive datediff function provided inconsistent results when hive.fetch.task.conversion is set to none
Date Sat, 09 Nov 2019 06:30:00 GMT

    [ https://issues.apache.org/jira/browse/HIVE-22476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16970739#comment-16970739
] 

Hive QA commented on HIVE-22476:
--------------------------------



Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12985411/HIVE-22476.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 31 failed/errored test(s), 17669 tests executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[date_1] (batchId=90)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[date_udf] (batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[localtimezone2] (batchId=64)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[localtimezone] (batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[timestamptz] (batchId=71)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[timestamptz_1] (batchId=65)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[timestamptz_2] (batchId=88)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[timestamptz_3] (batchId=62)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_from_utc_timestamp] (batchId=93)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorized_date_funcs] (batchId=85)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[windowing_windowspec] (batchId=19)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druid_timestamptz2] (batchId=199)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_dynamic_partition]
(batchId=199)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_expressions] (batchId=199)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_floorTime] (batchId=199)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_test1] (batchId=199)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_test_insert] (batchId=199)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_test_ts] (batchId=199)
org.apache.hadoop.hive.cli.TestMiniHiveKafkaCliDriver.testCliDriver[kafka_storage_handler]
(batchId=300)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[default_constraint] (batchId=173)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_date_funcs]
(batchId=184)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[date_udf] (batchId=128)
org.apache.hadoop.hive.common.type.TestTimestampTZ.testConvertToUTC (batchId=310)
org.apache.hadoop.hive.common.type.TestTimestampTZ.testFromToInstant (batchId=310)
org.apache.hadoop.hive.common.type.TestTimestampTZ.testVariations (batchId=310)
org.apache.hadoop.hive.llap.security.TestLlapSignerImpl.testSigning (batchId=365)
org.apache.hadoop.hive.ql.exec.vector.expressions.TestVectorDateExpressions.testVectorUDFUnixTimeStamp
(batchId=340)
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testWriteSetTracking2 (batchId=354)
org.apache.hadoop.hive.ql.udf.generic.TestGenericUDFToUnixTimestamp.testDate (batchId=315)
org.apache.hadoop.hive.ql.udf.generic.TestGenericUDFToUnixTimestamp.testString (batchId=315)
org.apache.hadoop.hive.serde2.TestJsonSerDe.testTimestampWithLocalTimeZone (batchId=361)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/19362/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/19362/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-19362/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 31 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12985411 - PreCommit-HIVE-Build

> Hive datediff function provided inconsistent results when hive.fetch.task.conversion
is set to none
> ---------------------------------------------------------------------------------------------------
>
>                 Key: HIVE-22476
>                 URL: https://issues.apache.org/jira/browse/HIVE-22476
>             Project: Hive
>          Issue Type: Bug
>            Reporter: Slim Bouguerra
>            Assignee: Slim Bouguerra
>            Priority: Major
>         Attachments: HIVE-22476.patch
>
>
> The actual issue stems to the different date parser used by various part of the engine.
> Fetch task uses udfdatediff via {code} org.apache.hadoop.hive.ql.udf.generic.GenericUDFToDate{code}
while the vectorized llap execution uses {code}VectorUDFDateDiffScalarCol{code}.
> This fix is meant to be not very intrusive and will add more support to the GenericUDFToDate
by enhancing the parser.
> For the longer term will be better to use one parser for all the operators.
> Thanks [~Rajkumar Singh] for the repro example
> {code} 
> create external table testdatediff(datetimecol string) stored as orc;
> insert into testdatediff values ('2019-09-09T10:45:49+02:00'),('2019-07-24');
> select datetimecol from testdatediff where datediff(cast(current_timestamp as string),
datetimecol)<183;
> set hive.ferch.task.conversion=none;
> select datetimecol from testdatediff where datediff(cast(current_timestamp as string),
datetimecol)<183;
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message